[gdal-dev] memory leak in GRIB reader (with Python bindings)

[Chris Barker]

Hi folks,

I"m finding what appears to be a memory leak, using the GRIB reader,
with the python bindings.

What I'm trying to do is read the data one band at a time, then throw
it away and read the next band -- there are 1129 bands in the file at
hand, and I can't hold it all in memory (32 bit still...)

However, when I do this, memory use just keeps climbing.

Should the memory be freed? I would expect so.

I'm using RasterBand.ReadAsArray()

Is this a leak? or is supposed to keep it around in memory?

Either way, is there a way to force it to release that memory (I"m
already doing and exlicite del and gc.collect call, so I dont think
it's a python reference counting issue)

I've enclosed a simpel test script -- watch the memory climb.

The data file is to big (186MB) to enclose here, you can get it here:

http://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20120522/18/time_grib_01/ocnu5.01.2012052218.daily.grb2

If you want to give this a try.


(note -- Grib giving some pretty good compression -- this climbs to
2.3GB when I read it)

I could actually live with the 2.3GB -- but in my real use case, I'm
reading two of these at the same time, so I max out what I can do with
32 bit python...

GDAL 1.8.1
Python 2.7 (32 bit Intel)
OS-X 10.6
I think it's the Kyng Chaos build of GDAL.

Thanks,
   -Chris









-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov