[gdal-dev] gdal_merge problems: Negative size passed to PyString_FromStringAndSize

[Even Rouault]

Le Wednesday 06 May 2009 16:48:06 Andreas Neumann, vous avez écrit :
> Hi,
> Filename: a25.tif
> File Size: 25472x25427x1
> Pixel Size: 42.431795 x -42.431795
> UL:(697837651.400476,244274906.596461)
> LR:(698918474.082743,243195993.344970)
> Copy 0,0,25472,25427 to 205747,211643,50722,50632.

--> here's the issue. a25.tif is resampled from 25472x25427 to a 50722x50632 
raster. But as gdal_merge.py just operates on full buffers, this results in 
trying to allocate 50722x50632 pixels = 2 568 156 304 bytes, which is above 
the largest 32 bit (signed) integer whose value is 2 147 483 647. So this 
value is interpreated as a negative size by Python...

To make the story short, you didn't do anything wrong. You've just hit a known 
limitation of gdal_merge.py. 

The issue is that the default pixel size used for the destination merged file 
is the pixel size of the first source raster processed, in your case 
21.309260 x -21.309260 for a01.tif. But a25.tif on which the error occurs has 
a pixel size of  42.431795 x -42.431795. So it must be nearly over-sampled by 
a factor of 2, so that accounts for his size going from 25472x25427 to a 
50722x50632.

There are several workarounds :
- use the -ps flag of gdal_merge.py to increase the pixel size of the 
destination raster. "-ps 30 30" would probably work (provided that you've 
enough memory)
- or add a preprocessing step that cuts a25.tif into 4 smaller pieces. This 
can be done with 4 calls to gdal_translate by using the -srcwin option
- or use gdalwarp to do the same thing. It has not the limitation of 
gdal_merge.py, but it is known to be much slower.

Patches to improve gdal_merge.py are welcome ;-)

> Traceback (most recent call last):
>   File "/usr/local/bin/gdal_merge.py", line 427, in <module>
>     fi.copy_into( t_fh, band, band, nodata )
>   File "/usr/local/bin/gdal_merge.py", line 227, in copy_into
>     nodata_arg )
>   File "/usr/local/bin/gdal_merge.py", line 47, in raster_copy
>     nodata )
>   File "/usr/local/bin/gdal_merge.py", line 83, in raster_copy_with_nodata
>     t_xsize, t_ysize )
>   File
> "/usr/lib/python2.5/site-packages/GDAL-1.6.0-py2.5-linux-x86_64.egg/osgeo/g
>dal.py", line 835, in ReadAsArray
>     buf_xsize, buf_ysize, buf_obj )
>   File
> "/usr/lib/python2.5/site-packages/GDAL-1.6.0-py2.5-linux-x86_64.egg/osgeo/g
>dal_array.py", line 133, in BandReadAsArray
>     buf_xsize, buf_ysize, datatype )
>   File
> "/usr/lib/python2.5/site-packages/GDAL-1.6.0-py2.5-linux-x86_64.egg/osgeo/g
>dal.py", line 760, in ReadRaster
>     return _gdal.Band_ReadRaster(*args, **kwargs)
> SystemError: Negative size passed to PyString_FromStringAndSize
> -------------------
>
> Is this a problem with my data, georeferencing or projection or is it a
> problem in gdal_merge?
>
> Thanks for any idea what might be wrong in this case!
>
> Andreas