[GRASS-dev] Allocating memory for a huge array

[Damiano Triglione]

Dear Paul and Glenn,
yes: in order to be quick and short, I badly explained myself.
Your advices are all interesting and correct, but I need to be more precise 
about my case:

I do not want a Huge Array, but a standard array (S_Array) whose values are 
indices to a Huge Array (H_Array)!
Besides, 2^32 is not the value I typed in my source file, but it was an 
expression just to let you understand the value 4 billions.

Sorry for my faults!

So, my intention is, actually:

unsigned int * S_Array;
double * H_Array;
(...)
S_Array=(unsigned int *)G_calloc(100, sizeof(unsigned int));
H_Array=(double *)G_calloc(1L<<32, sizeof(double));


Does it make sense, now?
Damiano


----- Original Message ----- 
From: "Glynn Clements" <glynn at gclements.plus.com>
To: "Damiano Triglione" <damiano.triglione at polimi.it>
Cc: <grass-dev at grass.itc.it>
Sent: Friday, March 23, 2007 6:23 AM
Subject: Re: [GRASS-dev] Allocating memory for a huge array


>
> Damiano Triglione wrote:
>
>> in a Grass module I am developing, I would like to allocate a huge array. 
>> I thought to use something like:
>>
>> unsigned int * H_Array;
>> (...)
>> H_Array=(unsigned int *)G_calloc(2^32, sizeof(unsigned int));
>>
>> because in this way (unsigned int) with 32 bits I can manage 4 billions 
>> cells, instead of 2 billions related to 32 bits as (traditional signed) 
>> int.
>> At compile-time, I receive a warning:
>>  warning: passing arg 1 of `G_calloc' as signed due to prototype
>>
>> Thus, I realize that I am not succeeding in what I want!
>> Is there a different G_calloc function that provides a prototype suitable 
>> for my needs?
>
> G_calloc() takes size_t arguments, and isn't the problem here.
>
> There are two distinct problems with the line:
>
> H_Array=(unsigned int *)G_calloc(2^32, sizeof(unsigned int));
>
> First, and most importantly, C's "^" operator performs bitwise
> exclusive-or (XOR), not exponentiation. Maybe you meant:
>
> H_Array=(unsigned int *)G_calloc(1<<32, sizeof(unsigned int));
> ?
>
> But even that is wrong; if you have a 32-bit "int" type (and almost
> every OS in common use does), 1<<32 == 0 according to the rules of C.
> Even if size_t is a 64-bit type (which is quite likely on a 64-bit
> system). The reason is that:
>
> 1. C types propagate from the bottom up; an expressions comprised
> solely of "int" values will be truncated to an "int", even if the
> result is assigned to a variable or parameter of a wider type (e.g.
> "long").
>
> 2. Undecorated integer literals have type "int"; if you want a long
> value, you need e.g. 1L.
>
> The following should work if your platform's "size_t" is 64 bits:
>
> H_Array=(unsigned int *)G_calloc(1L<<32, sizeof(unsigned int));
>
> Finally:
>
> 1. As Paul has pointed out, the number of entries and their types are
> unrelated.
>
> 2. If you are on a 32-bit platform, there is no way that you can have
> an array containing 2^32 "int"s, or even 2^32 bytes; The process'
> address space is limited to 2^32 bytes in total, and some of that is
> already taken. Even if you have more than 4GiB or RAM and/or swap, you
> can't have more than 4GiB for an individual process.
>
> -- 
> Glynn Clements <glynn at gclements.plus.com>