[GRASS5] total cells overflow

Markus Neteler neteler at geog.uni-hannover.de
Thu Sep 7 07:10:46 EDT 2000


Hi Justin,

On Thu, Sep 07, 2000 at 04:48:33PM +0700, Justin Hickey wrote:
> Hi Markus
> 
> Markus Neteler wrote:
> > while defining a new location (a large one) I got:
> > 
> >   total rows:            58400
> >   total cols:            65600
> >   total cells:   -,463,927,296
> > 
> > total cells should be 3831040000
> 
> It appears that the total cells value is stored as a signed 32 bit
> integer. The largest value that can be stored in such a variable is 2.14
> * 10^9 (2.14 billion). Your value is greater than that so the only
> solution I can think of is either change the variable storing the total
> cells to an unsigned 32 bit number, which should be fine since negative
> cells are invalid. Or create a smaller map.
> 
> Hope this helps.
Probably :-) Well, the file is

src/libes/gis/edit_cellhd.c

line 401
    sprintf(buf,"%ld",    (long) cellhd->rows * cellhd->cols);
    G_insert_commas(buf);
    fprintf (stderr, "  total cells: %15s\n",    buf);

I feel it would not break anything if the variable type would be
changed.

Other comments here?

Markus

---------------------------------------- 
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo at geog.uni-hannover.de with
subject 'unsubscribe grass5'



More information about the grass-dev mailing list