[GRASS5] Re: [GRASS_DE] r.mapcalc round()

[Markus Neteler]

On Tue, May 15, 2001 at 03:22:11PM +0100, Glynn Clements wrote:
> Markus Neteler wrote:
> > However, something is odd with r.mapcalc in my opinion:
> > 
> > r.mapcalc x1=1.4
> > EXECUTING x1 = ...  100%
> > CREATING SUPPORT FILES FOR x1
> > range: 1.3999999762 1.3999999762
> > 
> > r.stats x1
> > r.stats:  100%
> > 1.4-1.4
> > 
> > but:
> > r.stats -1 x1
> > 1.3999999762
> > 1.3999999762
> > 1.3999999762
> > 1.3999999762
> > 1.3999999762
> > [...]
> > 
> >  -> not what I expect.
> 
> Note that 1/10 isn't exactly representable in binary, in the same way
> that 1/3 isn't exactly representable in decimal. The above seems to
> match the accuracy of the "float" data type.
> 
> > r.univar x1
> > [...]
> > Minimum: 1.3999999762
> > Maximum: 1.3999999762
> > Range:  0
> > Arithmetic Mean:  1.4
> > Variance:  -2.49502e-12
> > awk: cmd. line:17: (FILENAME=- FNR=18894) warning: sqrt called with negative argument -2.49502e-12
> > Standarddeviation: nan
> > 
> > -> huh!
> 
> This looks like one of those cases where one operand of a subtraction
> has IEEE float (32-bit) or double (64-bit) precision while the other
> has the FPU's internal precision (80-bit for x86).
Aha, I see.
 
> Does this go away if you compile r.univar with "-ffloat-store"?

Well, r.univar is only a script using r.stats and awk. However, after
recompiling r.stats with above flag (adding 
EXTRA_CFLAGS=-ffloat-store
to the r.stats' Gmakefile) I can't see any difference.

> > Perhaps someone could explain this behaviour. Or is it a precision bug?
> 
> It looks like it.

If I shall try anything else...

 Markus

---------------------------------------- 
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo at geog.uni-hannover.de with
subject 'unsubscribe grass5'