[Gdal-dev] Numeric/NumPy and gdalnumeric.py

[Mateusz Łoskot]

Hi,

Recently, Didrik Pinte reported a bug + patch related to gdalnumeric.py
and Numeric package:
http://bugzilla.remotesensing.org/show_bug.cgi?id=1166

Didrik also has posted some details about new issue:
http://lists.maptools.org/pipermail/gdal-dev/2006-May/008839.html

Today, I investigated the problem and I'd like to share what I've learned.

On the Numeric Python website (http://numeric.scipy.org) one can read:

"NumPy derives from the old Numeric code base and can be used as a
replacement for Numeric. It also adds the features introduced by
numarray and can also be used to replace numarray."

(Didrik also reported about it in bug #1166).

So, the main issue is about transition gdalnumeric.py to use new NumPy
module. As I talked with Frank on the #gdal channel, the idea is to not
transit to NumPy completely, but also support old Numeric package.

So, I applied Didrik's patch to gdalnumeric.py file and tried to run
GDAL autotest but it failed.
Test cases that failed are:
gdalautotest/gcore/tiff_write.py
gdalautotest/gcore/numpy_rw.py

So it seems the patch does not solve all problem.

The first problem I found is that type codes have changed in NumPy and
seems to be incompatible with Numeric (NumericTypeCodeToGDALTypeCode
function fails in gdalnumeric.py).
I searched through NumPy sources but I have to say that there are many
versions of type codes schemes, some are marked as deprecated, some are
left as for backward compatibility, etc.
Unfortunately, NumPy docs are not freely accessible
(http://www.tramy.us/guidetoscipy.html) so I couldn't check what codes
are recommended now.

Conclusion is that hosting support for both NumPy and Numeric can be
tricky because of differences on basic level, so a kind of proxy would
be needed. Also, the question is how critical is support for NumPy?

Second issue has been reported in Didrik's post
(http://lists.maptools.org/pipermail/gdal-dev/2006-May/008839.html).
I'd suppose it's related to differences between NumPy and Numeric
modules. So, untill gdalnumeric.py is not ported to NumPy, then it may
be difficult to use gdalnumeric.py with this module, instead Numeric
seems to be still recommended.

I asked Didrik for his script he was using to generated problematic tiff
raster, so then I'll be able to reproduce the problem step-by-step and
detect possible issues with different environments.

Comments are welcome.

Cheers
-- 
Mateusz Łoskot
http://mateusz.loskot.net