[GRASS-user] NVIZ can't allocate enough memory

[Glynn Clements]

Daniel Lee wrote:

> ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

18446744071974792324 = 0xffffffff9899ac84

> Does anybody know what could be the problem? If I'm interpreting the bytes
> correctly, that'd be about 10^10 gigabytes, which several orders of
> magnitude larger than the rasters involved. Thanks!

A calculation has overflowed the range of a signed 32-bit integer
resulting in a negative value. This value has then been converted to a
signed 64-bit integer, and then to an unsigned 64-bit integer.

> Here's the output of g.region:

> rows:       32001
> cols:       20001
> cells:      640052001

At 4 bytes per cell, 640052001 cells = 2560208004 bytes.

2560208004 = 0x9899ac84

The maximum value representable by a 32-bit integer is 0x7fffffff =
2147483647.

Using an unsigned integer would eliminate the problem in this
particular case, but the region wouldn't need to be much larger in
order for that to be insufficient (specifically, 32001 x 33553 would
overflow).

The correct solution is to use "size_t" rather than "int". E.g. for
this specific case:

--- gsds.c	(revision 52015)
+++ gsds.c	(working copy)
@@ -481,7 +481,8 @@
 int gsds_alloc_typbuff(int id, int *dims, int ndims, int type)
 {
     dataset *ds;
-    int i, siz = 1;
+    int i;
+    size_t siz = 1;
 
     if ((ds = get_dataset(id))) {
 	/*

More generally, it's important that the conversion comes before the
mulitply; e.g.:

	size_t cells = (size_t) rows * cols;

will work but:

	size_t cells = rows * cols;

won't; the calculation will be perfomed using "int", overflow, then
the overflowed result will be promoted.

However: the number of instances of this particular issue in the GRASS
source code is probably somewhere between "dozens" and "hundreds", and
there's no systematic way to identify them. Running test cases with a
region of >= 2^32 cells (or even >= 2^29 cells, i.e. >= 2^31 bytes at
4 bytes per cell) would find a lot of them, but it requires a 64-bit
system with plenty of RAM, and it's going to be slow.

-- 
Glynn Clements <glynn at gclements.plus.com>