[gdal-dev] Excessive memory usage in python script

[Jamie Adams]

Hello all,

I've got a python script that reads and extracts elevation subdatasets from
an HDF5 raster.  There are hundreds of these and I'm running into issues
where system memory usage grows increasingly until I get a system "memory
could not be read" error.  All the subdatasets are 600x600 and result in
1.4MB GeoTiffs.  Extracting 6 of these in a row, I get a memory usage as
high as 160MB.  I believe I made it through about 20 of these before the
system choked.

How can I get the memory usage under control?  I'm not concerned about a
minimal footprint, just one that won't crash python.  I've tried various
combinations of del statements at the end of the loop and they seem to make
no difference.  I've also tried removing the Numeric.multiply line just to
make sure I wasn't creating any circular references.  Not sure what else to
do, any advice appreciated.

-Jamie

On WinXP with FWTools version 2.10.

Here's a basic rundown of the script:

###########

for i in hdf_sub:
  outfile = i+'.tif'

  indataset = gdal.Open(i)
  inband = indataset.GetRasterBand(1)

  format = "GTiff"
  outdataset.driver.Create(outfile, inband.XSize, inband.YSize, 1,
gdal.GDT_Float32)
  outband = outdataset.GetRasterBand(1)

  ## Set the geotransform & projection ##
  ......blah...blah

  ## Read data into Numeric array
  temp_out = inband.ReadAsArray(0, 0, inband.XSize, inband.YSize)

  ## Make all values negative
  ## Tried without this line, made no difference
  temp_out = Numeric.multiply(temp_out, -1)

  ## The data is stored reversed on the y-axis, reverse and write
  outband.WriteArray(temp_out[::-1])

###########
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20080609/19339da1/attachment.html