NumPy Integration

[Frank Warmerdam]

Folks, 

Preliminary integration of GDAL and OpenEV with NumPy is now checked in.
The following shows a simple example of loading a GDAL supported file 
into a NumPy array (using the gdalnumeric.LoadFile() function), manipulating
it to produce a new array, and then opening them both as GDAL datasets for
display in OpenEV.

from Numeric import *
import gdalnumeric

array = gdalnumeric.LoadFile( "/u/data/mff/test.hdr" )
array2 = array * -1
app.file_open_by_name(gdalnumeric.GetArrayFilename(array))
app.file_open_by_name(gdalnumeric.GetArrayFilename(array))

Details on the methods available are currently only available in 
gdal.py, and gdalnumeric.py. Some issues are:

o The LoadFile(), and ReadAsArray() methods on the gdal.Dataset, and gdal.Band
currently make an intermediate binary string copy of the data read from the
GDAL file, before turning it into a NumPy array. This is potentially is
somewhat less efficient than creating the NumPy array directly but avoids
us needing to link directly against NumPy shared libraries. 

o GDAL supports NumPy arrays as a special format. The filename passed to the
gdal.Open() call should be "NUMPY:::0xhhhhhhhh" with the h's containing the
hex pointer value of the python array object. The 
gdalnumeric.GetArrayFilename() function can be used to create this for any
array. 

o GDAL adds a reference to the opened array, and releases it when the file
is closed. This should prevent memory leaks, or unexpected crashes, 
assuming the gdal handle gets closed properly.

o GDAL supports almost all the types of numeric python arrays, including 
float32, float64, int32, and complex types. It does not include int64,
float128, int8, "object" arrays, or char arrays. If this is a problem, 
I could have the OpenArray() method cast these other types, or event cast
the numeric ones in C. 

o GDAL support for numpy arrays doesn't involve any duplication of data. 
The gdal access is to the same underlying memory. It is possible to
update numpy arrays in-place with some (all?) ufuncs, which should be
reflected in the next read from GDAL. However, GDAL doesn't know when 
the data has changed, and neither does the application. OpenEV will tend
to cache data in the GvRaster, and won't realize if this has been 
invalidated. 

o I have been careful to make it so that numpy support in GDAL can be built 
without numeric python actually having to be installed on the build system,
or the system the binary is run on. However, attempting to load gdalnumeric,
or call numpy related methods on gdal.Dataset, or gdal.Band without having
numpy installed will result in a python exception at the point where
the attempt to "import Numeric" is made. 

I don't have any immediate plans for more work on numpy integration with 
GDAL. The next stage is integrating a python command tool in OpenEV, which
is tentatively Steve's job. However, even now it should be practical for 
anyone interested to try writing numpy scripts for image manipulation loading
and saving through GDAL. I would be happy to provide support to anyone wanting
to try. 

Best regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerda at h...
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush | Geospatial Programmer for Rent