<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Verdana","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Greetings
GDAL developers,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I
am encountering what I suspect is a SWIG issue with the GDAL Python bindings. I
am using the bindings from Python scripts invoked by ArcGIS on Windows.
Starting with version 9.3, ArcGIS instantiates the Python interpreter internally
with Python’s C API and executes scripts within the ArcGIS process itself,
rather than starting a worker python.exe process and instructing it to run the
script. They did this presumably because it provides major performance
advantages.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I
have a script that calls band.WriteArray(). My problem is that I cannot run
this script more than once from an ArcGIS process without having to restart it.
The first time it runs fine. The second time it runs I get this error:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:#D70019'><type
'exceptions.TypeError'>: in method 'BandRasterIONumPy', argument 1 of type
'GDALRasterBandShadow *'<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>The
script is very simple, shown below my signature below. To repro it, find a
Windows machine (any version) with ArcGIS 9.3 (I’m using 9.3.1 SP1),
Python 2.5 (I have 2.5.4), numpy (I have 1.3.0), and install the latest GDAL
and Python bindings from <a href="http://vbkto.dyndns.org/sdk/">http://vbkto.dyndns.org/sdk/</a>.
I am using those from <a href="http://vbkto.dyndns.org/sdk/release-1310-dev.zip">http://vbkto.dyndns.org/sdk/release-1310-dev.zip</a>
from about a couple of weeks ago.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>You
may recall that I previously expressed satisfaction that the GDAL team was
moving to SWIG 1.3.39 because it fixed a similar issue. See <a
href="http://www.mail-archive.com/gdal-dev@lists.osgeo.org/msg05108.html">http://www.mail-archive.com/gdal-dev@lists.osgeo.org/msg05108.html</a>.
I believe this issue is similar to that one, but more complicated.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>In
short, I suspect the issue relates to the fact that GDAL’s Python
extension modules _gdal.pyd and _gdal_array.pyd do not fully support what
ArcGIS is doing. Every time ArcGIS runs a Python script, it either instantiates
a new Python interpreter or instantiates a “sub-interpreter” of the
main interpreter. I’m not sure which. In any case, when the GDAL
extension modules are imported the second time, it appears that some SWIG stuff
is not properly set up between the two of them. When _wrap_BandRasterIONumPy()
in gdal_array_wrap.cpp performs its type checking, it seems to be referencing
the Python type structure instantiated from the first import cycle. So even
though the provided object is a GDALRasterBandShadow instance, it was instantiated
during the second import cycle so SWIG considers it a different type.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I
am not familiar enough with SWIG to know where to go from here. Is there someone
who understands the problem or at least advise me on what to do next (e.g. Howard
Butler)?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>The
ultimate consequence of this is that you cannot write rasters from an ArcGIS 9.3
or later geoprocessing script using GDAL. You have to fall back on the old,
very slow method of writing using some other technique and calling an ArcGIS
tool to convert the data to the format you want.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Presumably
this problem affects other Python-based infrastructures that use sub-interpreters.
I’m not sure what falls into that category.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Thanks
very much for any help you can provide.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Jason<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Here
is the Python script:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>#
Instantiate ArcGIS geoprocessor so we can report messages.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>import
arcgisscripting<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>gp
= arcgisscripting.create()<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>#
Create an IMG raster of random float32 values, size 250x250, cell size 1.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>if
not gp.Exists('C:\\RandomData.img'):<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>
gp.AddMessage('Creating the IMG file')<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>
gp.CreateRandomRaster_management('C:\\', 'RandomData.img', 'UNIFORM', '0 0 250
250', 1)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>#
Import gdal, read the values and write them back.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>from
osgeo import gdal<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>from
osgeo import gdalconst<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>dataset
= gdal.Open('C:\RandomData.img', gdalconst.GA_Update)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>band
= dataset.GetRasterBand(1)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>gp.AddMessage('Reading
the data')<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>data
= band.ReadAsArray()<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>gp.AddMessage('Writing
the data')<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>band.WriteArray(data,
0, 0)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>gp.AddMessage('Done')<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>It
succeeds the first time:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:#404040'>Executing:
ArrayTest<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Start Time: Wed
Jun 30 18:14:22 2010<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Running script
ArrayTest...<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Reading the data<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Writing the data<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Done<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Completed script
ArrayTest...<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Executed
(ArrayTest) successfully.<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>End Time: Wed
Jun 30 18:14:23 2010 (Elapsed Time: 1.00 seconds)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>It
fails the second and subsequent times:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:#404040'>Executing:
ArrayTest<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Start Time: Wed
Jun 30 18:14:42 2010<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Running script
ArrayTest...<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>Reading the data<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:#D70019'><type
'exceptions.TypeError'>: in method 'BandRasterIONumPy', argument 1 of type
'GDALRasterBandShadow *'<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:#D70019'>Failed to
execute (ArrayTest).<o:p></o:p></span></p>
<p class=MsoNormal style='margin-left:7.5pt;text-autospace:none'><span
style='font-size:10.0pt;font-family:"Courier New";color:black'>End Time: Wed
Jun 30 18:14:42 2010 (Elapsed Time: 0.00 seconds)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>
</div>
</body>
</html>