[gdal-dev] Problem calling Python bindings' band.ReadAsArray() from ArcGIS 9.3 - SWIG issue?

Wed Jun 30 18:21:48 EDT 2010

Oops, I apologize; I mis-described the scenario slightly. I had previously
thought that the failure occurs only when calling band.WriteArray. In fact,
it happens during band.ReadAsArray. Sorry for the elementary mistake.

Anyway, you probably want a full traceback. I changed this:

data = band.ReadAsArray()

to this:

try:

    data = band.ReadAsArray()

except:

    import traceback

    gp.AddMessage(traceback.format_exc())

    raise

and now here is the output:

Executing: ArrayTest

Start Time: Wed Jun 30 18:19:12 2010

Running script ArrayTest...

Reading the data

Traceback (most recent call last):

  File "<string>", line 21, in <module>

  File "C:\Python25\lib\site-packages\osgeo\gdal.py", line 850, in
ReadAsArray

  File "C:\Python25\lib\site-packages\osgeo\gdal_array.py", line 239, in
BandReadAsArray

  File "C:\Python25\lib\site-packages\osgeo\gdal_array.py", line 88, in
BandRasterIONumPy

TypeError: in method 'BandRasterIONumPy', argument 1 of type
'GDALRasterBandShadow *'

<type 'exceptions.TypeError'>: in method 'BandRasterIONumPy', argument 1 of
type 'GDALRasterBandShadow *'

Failed to execute (ArrayTest).

End Time: Wed Jun 30 18:19:13 2010 (Elapsed Time: 1.00 seconds)

Thanks,

Jason

From: gdal-dev-bounces at lists.osgeo.org
[mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of Jason Roberts
Sent: Wednesday, June 30, 2010 6:16 PM
To: gdal-dev at lists.osgeo.org
Subject: [gdal-dev] Problem calling Python bindings' band.WriteArray() from
ArcGIS 9.3 - SWIG issue?

Greetings GDAL developers,

I am encountering what I suspect is a SWIG issue with the GDAL Python
bindings. I am using the bindings from Python scripts invoked by ArcGIS on
Windows. Starting with version 9.3, ArcGIS instantiates the Python
interpreter internally with Python's C API and executes scripts within the
ArcGIS process itself, rather than starting a worker python.exe process and
instructing it to run the script. They did this presumably because it
provides major performance advantages.

I have a script that calls band.WriteArray(). My problem is that I cannot
run this script more than once from an ArcGIS process without having to
restart it. The first time it runs fine. The second time it runs I get this
error:

<type 'exceptions.TypeError'>: in method 'BandRasterIONumPy', argument 1 of
type 'GDALRasterBandShadow *'

The script is very simple, shown below my signature below. To repro it, find
a Windows machine (any version) with ArcGIS 9.3 (I'm using 9.3.1 SP1),
Python 2.5 (I have 2.5.4), numpy (I have 1.3.0), and install the latest GDAL
and Python bindings from http://vbkto.dyndns.org/sdk/. I am using those from
http://vbkto.dyndns.org/sdk/release-1310-dev.zip from about a couple of
weeks ago.

You may recall that I previously expressed satisfaction that the GDAL team
was moving to SWIG 1.3.39 because it fixed a similar issue. See
http://www.mail-archive.com/gdal-dev@lists.osgeo.org/msg05108.html. I
believe this issue is similar to that one, but more complicated.

In short, I suspect the issue relates to the fact that GDAL's Python
extension modules _gdal.pyd and _gdal_array.pyd do not fully support what
ArcGIS is doing. Every time ArcGIS runs a Python script, it either
instantiates a new Python interpreter or instantiates a "sub-interpreter" of
the main interpreter. I'm not sure which. In any case, when the GDAL
extension modules are imported the second time, it appears that some SWIG
stuff is not properly set up between the two of them. When
_wrap_BandRasterIONumPy() in gdal_array_wrap.cpp performs its type checking,
it seems to be referencing the Python type structure instantiated from the
first import cycle. So even though the provided object is a
GDALRasterBandShadow instance, it was instantiated during the second import
cycle so SWIG considers it a different type.

I am not familiar enough with SWIG to know where to go from here. Is there
someone who understands the problem or at least advise me on what to do next
(e.g. Howard Butler)?

The ultimate consequence of this is that you cannot write rasters from an
ArcGIS 9.3 or later geoprocessing script using GDAL. You have to fall back
on the old, very slow method of writing using some other technique and
calling an ArcGIS tool to convert the data to the format you want.

Presumably this problem affects other Python-based infrastructures that use
sub-interpreters. I'm not sure what falls into that category.

Thanks very much for any help you can provide.

Jason

Here is the Python script:

# Instantiate ArcGIS geoprocessor so we can report messages.

import arcgisscripting

gp = arcgisscripting.create()

# Create an IMG raster of random float32 values, size 250x250, cell size 1.

if not gp.Exists('C:\\RandomData.img'):

    gp.AddMessage('Creating the IMG file')

    gp.CreateRandomRaster_management('C:\\', 'RandomData.img', 'UNIFORM', '0
0 250 250', 1)

# Import gdal, read the values and write them back.

from osgeo import gdal

from osgeo import gdalconst

dataset = gdal.Open('C:\RandomData.img', gdalconst.GA_Update)

band = dataset.GetRasterBand(1)

gp.AddMessage('Reading the data')

data = band.ReadAsArray()

gp.AddMessage('Writing the data')

band.WriteArray(data, 0, 0)

gp.AddMessage('Done')

It succeeds the first time:

Executing: ArrayTest

Start Time: Wed Jun 30 18:14:22 2010

Running script ArrayTest...

Reading the data

Writing the data

Done

Completed script ArrayTest...

Executed (ArrayTest) successfully.

End Time: Wed Jun 30 18:14:23 2010 (Elapsed Time: 1.00 seconds)

It fails the second and subsequent times:

Executing: ArrayTest

Start Time: Wed Jun 30 18:14:42 2010

Running script ArrayTest...

Reading the data

<type 'exceptions.TypeError'>: in method 'BandRasterIONumPy', argument 1 of
type 'GDALRasterBandShadow *'

Failed to execute (ArrayTest).

End Time: Wed Jun 30 18:14:42 2010 (Elapsed Time: 0.00 seconds)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100630/9906e6bb/attachment-0001.html