[GRASSLIST:3613] r.mapcalc - zero point bug
Maciek Sieczka
werchowyna at pf.pl
Mon Jun 7 15:55:31 EDT 2004
hi!
I suppose I found a bug in r.mapcalc
(but I hope I didn't :))
I'm still in Grass 5.03 so if this is already fixed in newer release please
forgive me for taking you time
the problem: when working with really big numbers r.mapcalc lies if you
don't add an ".0" after a number you input:
WRONG result when without the ".0" following the "636287352795314688":
r.mapcalc
'187bf256=(187bf-220330.1520377753)/(636287352795314688-220330.1520377753)*2
55'
r.univar 187bf256
Number of cells (excluding NULL cells): 3256397
Minimum: 0
Maximum: 75562821610.9638366699
Range: 7.55628e+10
Arithmetic mean: 2.9251e+09
Variance: 2.61519e+19
Standard deviation: 5.1139e+09
Variation coefficient: 174.828 %
FINE when the ".0" is added:
r.mapcalc
'187bf256=(187bf-220330.1520377753)/(636287352795314688.0-220330.1520377753)
*255'
r.univar 187bf256
Number of cells (excluding NULL cells): 3256397
Minimum: 0
Maximum: 255
Range: 255
Arithmetic mean: 9.87126
Variance: 297.83
Standard deviation: 17.2577
Variation coefficient: 174.828 %
FYI: the '187bf' raster is DCELL
another strange example:
r.mapcalc
'187bf256=round((187bf-220330.1520377753)/(636287352795314688-220330.1520377
753)*255)'
100%
NOTE: overflow occured in the calculation
r.info 187bf256 -r
min=0
max=2147479019
what may it be?
Maciek
More information about the grass-user
mailing list