[GRASS-dev] Allocating memory for a huge array

[Glynn Clements]

Damiano Triglione wrote:

> in a Grass module I am developing, I would like to allocate a huge array. I thought to use something like:
> 
> unsigned int * H_Array;
> (...)
> H_Array=(unsigned int *)G_calloc(2^32, sizeof(unsigned int));
> 
> because in this way (unsigned int) with 32 bits I can manage 4 billions cells, instead of 2 billions related to 32 bits as (traditional signed) int.
> At compile-time, I receive a warning:
>  warning: passing arg 1 of `G_calloc' as signed due to prototype
> 
> Thus, I realize that I am not succeeding in what I want!
> Is there a different G_calloc function that provides a prototype suitable for my needs?

G_calloc() takes size_t arguments, and isn't the problem here.

There are two distinct problems with the line:

	H_Array=(unsigned int *)G_calloc(2^32, sizeof(unsigned int));

First, and most importantly, C's "^" operator performs bitwise
exclusive-or (XOR), not exponentiation. Maybe you meant:

	H_Array=(unsigned int *)G_calloc(1<<32, sizeof(unsigned int));
?

But even that is wrong; if you have a 32-bit "int" type (and almost
every OS in common use does), 1<<32 == 0 according to the rules of C. 
Even if size_t is a 64-bit type (which is quite likely on a 64-bit
system). The reason is that:

1. C types propagate from the bottom up; an expressions comprised
solely of "int" values will be truncated to an "int", even if the
result is assigned to a variable or parameter of a wider type (e.g. 
"long").

2. Undecorated integer literals have type "int"; if you want a long
value, you need e.g. 1L.

The following should work if your platform's "size_t" is 64 bits:

	H_Array=(unsigned int *)G_calloc(1L<<32, sizeof(unsigned int));

Finally:

1. As Paul has pointed out, the number of entries and their types are
unrelated.

2. If you are on a 32-bit platform, there is no way that you can have
an array containing 2^32 "int"s, or even 2^32 bytes; The process'
address space is limited to 2^32 bytes in total, and some of that is
already taken. Even if you have more than 4GiB or RAM and/or swap, you
can't have more than 4GiB for an individual process.

-- 
Glynn Clements <glynn at gclements.plus.com>