[GRASS-dev] r.contour - G_malloc:out of memory

[Glynn Clements]

Henning Lorenz wrote:

> In Grass 6.3 on Mac OSX 10.5 I try to contour the following raster  
> file (from r.info):
> 
> Type of Map:  raster               Number of Categories: 255
> Data Type:    CELL
> Rows:         38002
> Columns:      50276
> Total Cells:  1910588552
> Projection: UTM (zone 40)
> N: 7949005.04157202    S:    7816000   Res: 3.49994847
> E:     523000    W: 347033.98326284   Res: 3.50000033
> Range of data:    min = 0  max = 255
> 
> with the region set to the raster file (from g.region -p):
> 
> GRASS 6.3.0 (NovZem(old)):~ > g.region -p
> projection: 1 (UTM)
> zone:       40
> datum:      wgs84
> ellipsoid:  wgs84
> north:      7949005.04157202
> south:      7816000
> west:       347033.98326284
> east:       523000
> nsres:      3.49994847
> ewres:      3.50000033
> rows:       38002
> cols:       50276
> cells:      1910588552
> 
> The file size on disk is 1.22 GB. I have 8 GB RAM, whereof 5929 MB  
> free when I execute r.contour on the command line. r.contour fails  
> with the following message after reading in ca. 25% of the data  
> (reproduceable):
> 
> GRASS 6.3.0 (NovZem(old)):~ > r.contour --overwrite  
> input=NovZem_S_highrres_red at enhanced output=NovZem_shoreline levels=1
> Reading data:
> r.contour(13162) malloc: *** mmap(size=405504) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
> ERROR: G_malloc: out of memory
> 
> This happens while 3020 MB memory are still free! On the other hand it  
> means that r.contour used ca. 3 GB of memory for load ca. 300 MB of  
> data from the disk, i.e. the whole 1.22 GB dataset would require ca.  
> 12 GB of free memory.
> 
> Is this behaviour expected?

Yes.

You have ~2G (2^31) cells. Each one is stored internally as a 64-bit
floating-point value[1], meaning that it would require ~16GiB of
memory.

Try reducing the region resolution.

Note that the file in question is compressed, and only uses 1 byte per
cell when decompressed.

[1] which is probably overkill; a 32-bit value would likely suffice. 
However, that might still be problematic in this particular case,
depending upon how much memory is required for other purposes.

-- 
Glynn Clements <glynn at gclements.plus.com>