[Gdal-dev] Thoughts about gdalnumeric.py, NumPy and SWIG bindings

[Mateusz Loskot]

Hi,

Recently, I'm doing some research in field of SWIG bindings for GDAL.
Also, I'm investigating NumPy package to include it in Python bindings.
I'm quite new to SWIG and GDAL bindings so I have to try to guess
what author had in mind regarding many things.
So, I'd like to share my experiences related to SWIG bindings for GDAL
and ask some questions.
In order to keep things simpler, I'll be mainly
referring to Python bindings.

There are two bindings to Python for GDAL:
- traditional pymod
- Next Generation bindings, based on SWIG

The traditional pymod includes gdalnumeric.py module that integrates
GDAL with Numeric package. The pymod uses older release of Numeric.
Recently, Numeric package was released as a part of a bigger
one called NumPy.

Currently, NG Python wrapper does not provide support of gdalnumeric.py
and Numeric/NumPy.

My MAIN task is to include gdalnumeric.py and NumPy support to NG Python
bindings to GDAL. And here are two possibilities:
- provide support of both: old Numeric and new NumPy package
- drop Numeric support and use only new NumPy package in gdalnumeric.py

The traditional pymod will stay untouched supporting only old Numeric.

The idea behind Numeric/NumPy support is to read/write GDAL
dataset from/to numeric array buffers.
Yesterday, Howard suggested to consider possibility to add
gdalnumeric.py features to other scripting (SWIG) bindings.
I think this is a good idea and would help to unify
SWIG based API to GDAL.

Here is small draft of the idea:

1. Split gdalnumeric.py from Python-specific stuff and form generic
API of gdalnumeric as a gdalnumeric.i file for SWIG.
2. Make a research to find NumPy counterparts in other scripting
languages.
3. Implement specific stuff for every scripting language in its
"private" .i file

I have no idea if this is feasible for all scripting
languages GDAL is going to support.
At the first stage, I'm going to focus on Python bindings.
Comments are welcome!

But there are some problems...

I analyzed the structure of the traditional pymod bindings and
the NG bindings for Python and there are some things
I'd need to explain.

First, as I see, gdalnumeric.py delegates calls to low-level
module _gdal.so.
All (almost) those functions called by gdalnumeric.py are
implemented in gdal.i file as manually-written wrappers
e.g.:
py_GDALReadRaster - called by gdalnumeric.BandReadAsArray

I also see, that gdalnumeric.py seems to be "incomplete"
in support of Numeric.
I mean, Numeric module is used only in few cases.
Also, PyObjectArray from Numeric (and NumPy too) is not
used by functions called from gdalnumeric.py.
My be my research was not deep enough.

As I talked with Frank on the #gdal, it seems to be reasonable to move
this wrappers from gdal.i file of the traditional pymod and paste them
to gdal_python.i file from SWIG Python bindings.

But there (in the traditional pymod sources) is another module called
numpydataset.cpp.

And here I'm confused a bit:
- gdalnumeric.py is called as a Numeric integration
module but does not use main features of Numeric
- numpydataset.cpp uses Numeric (NumPy) features extensively but I can't
find where features from this module is used in
the traditional pymod or NG Python.
When I grep for NUMPYDataset (class defined in numpydataset.cpp) I don't
get any new files besides numpydataset.cpp (in both Python bindings).

So, it's hard to me to discover current state of the work, what is used
what is deprecated, what is prepared for future usage.
And, if I can't find answer for those questions,
I don't know what should I port to add NumPy support to NG Python.
gdalnumeric.py with those manually-written wrappers from
gdal.i or add numpydataset.cpp + Proxy Class (Shadow) for Python?

Please, if anyone could explain me current situation,
confirm or negate what I've discovered for now,
I'll be very thankful.
I'm trying to find the right way to go with this task,
where to start, so I definitely need some startup
guidelines to proceed.

Thanks in advance
-- 
Mateusz Łoskot
http://mateusz.loskot.net