[GRASS-dev] CELL/FCELL/DCELL [was: Re: r.mapcal rand() strangeness]

[Maciej Sieczka]

Below follow details about CELL and DCELL datatypes in GRASS. It would 
be good to have them summarrised in GRASS raster intro IMHO; + FCELL 
specific notes. I'm not competent - anybody please do.

questions by me, Glynn replies:

> CELL maps are limited to 32-bit integers. There doesn't seem much
> point in extending r.mapcalc to handle anything larger; too much work
> for too little reward.

>>>> something bizzare to me:
>>>>
>>>> $ r.mapcalc 'map=99999999999999999999999.0'
>>>> $ r.info -rt map
>>>> min=99999999999999991611392.000000
>>>> max=99999999999999991611392.000000
>>>> datatype=DCELL

>>> A double has a precision of ~16 decimal digits, which matches what you
>>> see above.

>> So processing any number with more than 16 decimal diggits in r.mapcalc 
>> must yield such "strange" values?

> The precision of a "double" corresponds to roughly 16 significant
> decimal digits (it's exactly 52 binary digits).
> 
> The problem arises when something (in this case, r.info) tries to
> print numbers where the number of digits to the left of the decimal
> point exceeds the precision.
> 
> If you use exponential notation, you can limit the number of digits to
> match the precision, so the problem doesn't arise.

>> And how many diggits after the decimal 
>> separator are safe?

> Floating point numbers have a fixed relative error rather than a fixed
> absolute error. The issue is the number of significant digits. The
> position of the decimal point doesn't matter.

>> Does that mean that actually in the raster the value is something else 
>> than what r.info reports?

> What is in the raster is a binary floating point value:
> 
> 	http://en.wikipedia.org/wiki/IEEE_floating-point
> 
> E.g. 3.40282e38 is stored as
> 
> 	(9007189542424620/(2^53))*(2^128)
> =	9007189542424620*(2^(128-53))
> =	340282000000000014192072600942972764160