[GRASSLIST:3613] r.mapcalc - zero point bug

Maciek Sieczka werchowyna at pf.pl
Mon Jun 7 15:55:31 EDT 2004


hi!

I suppose I found a bug in r.mapcalc
(but I hope I didn't :))

I'm still in Grass 5.03 so if this is already fixed in newer release please
forgive me for taking you time

the problem: when working with really big numbers r.mapcalc lies if you
don't add an ".0" after a number you input:

WRONG result when without the ".0" following the "636287352795314688":

    r.mapcalc
'187bf256=(187bf-220330.1520377753)/(636287352795314688-220330.1520377753)*2
55'

    r.univar 187bf256

    Number of cells (excluding NULL cells): 3256397
    Minimum: 0
    Maximum: 75562821610.9638366699
    Range:  7.55628e+10
    Arithmetic mean:  2.9251e+09
    Variance:  2.61519e+19
    Standard deviation: 5.1139e+09
    Variation coefficient: 174.828 %

FINE when the ".0" is added:

    r.mapcalc
'187bf256=(187bf-220330.1520377753)/(636287352795314688.0-220330.1520377753)
*255'

    r.univar 187bf256

    Number of cells (excluding NULL cells): 3256397
    Minimum: 0
    Maximum: 255
    Range:  255
    Arithmetic mean:  9.87126
    Variance:  297.83
    Standard deviation: 17.2577
    Variation coefficient: 174.828 %

FYI: the '187bf' raster is DCELL

another strange example:

    r.mapcalc
'187bf256=round((187bf-220330.1520377753)/(636287352795314688-220330.1520377
753)*255)'
    100%
    NOTE: overflow occured in the calculation
    r.info 187bf256 -r
    min=0
    max=2147479019

what may it be?

Maciek




More information about the grass-user mailing list