[GRASS5] floating AI ASCII GRID import
Frank Warmerdam
fwarmerdam at gmail.com
Sat Nov 27 11:53:57 EST 2004
Maciek Sieczka wrote:
> The value range should be:
> min=176.945400
> max=284.015600
>
> And it is the case when 'r.in.arc type=DCELL' is used for importing.
>
> However, if 'r.in.gdal' or 'r.in.arc type=FCELL' is used then:
>
> min=176.945404
> max=284.015594
>
> I suppose there should be no difference. What is the reason?
Maciek,
Well, the obvious conclusion is that a FCELL, which I presume reads as a
4byte float gives
slightly less precision than DCELL which is an 8byte float. The change
in the seventh digit would
seem to be what we would expect from floating point precision limitations.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
More information about the grass-dev
mailing list