[GRASS5] Re: [GRASS_DE] r.mapcalc round()

[Glynn Clements]

Markus Neteler wrote:

> However, something is odd with r.mapcalc in my opinion:
> 
> r.mapcalc x1=1.4
> EXECUTING x1 = ...  100%
> CREATING SUPPORT FILES FOR x1
> range: 1.3999999762 1.3999999762
> 
> r.stats x1
> r.stats:  100%
> 1.4-1.4
> 
> but:
> r.stats -1 x1
> 1.3999999762
> 1.3999999762
> 1.3999999762
> 1.3999999762
> 1.3999999762
> [...]
> 
>  -> not what I expect.

Note that 1/10 isn't exactly representable in binary, in the same way
that 1/3 isn't exactly representable in decimal. The above seems to
match the accuracy of the "float" data type.

> r.univar x1
> [...]
> Minimum: 1.3999999762
> Maximum: 1.3999999762
> Range:  0
> Arithmetic Mean:  1.4
> Variance:  -2.49502e-12
> awk: cmd. line:17: (FILENAME=- FNR=18894) warning: sqrt called with negative argument -2.49502e-12
> Standarddeviation: nan
> 
> -> huh!

This looks like one of those cases where one operand of a subtraction
has IEEE float (32-bit) or double (64-bit) precision while the other
has the FPU's internal precision (80-bit for x86).

Does this go away if you compile r.univar with "-ffloat-store"?

> Perhaps someone could explain this behaviour. Or is it a precision bug?

It looks like it.

-- 
Glynn Clements <glynn.clements at virgin.net>

---------------------------------------- 
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo at geog.uni-hannover.de with
subject 'unsubscribe grass5'