[GRASS5] Zoom

[Bernhard Reiter]

On Wed, Apr 24, 2002 at 05:55:29PM +0100, Glynn Clements wrote:
> Radim Blazek wrote:
> > while finishing d.zoom, 

Can you tell us, when you are finished?
Then Glynn can start branching.

I'am facing this problem:
> > n-s resol:  0.1
> > fprintf(stderr, "%.15f\n", window.ns_res);
> > prints:
> > 0.099999999627471
> > 
> > I expected 0.1. Can anyone help me, explain that?

Apart from Glynn's explaination, the relevant section in the python 
tutorial will enlight you about the "perils of floating point"
representation for background reading:

	http://www.python.org/doc/current/tut/node14.html

> Note that the resolution fields are ignored; they are generated by
> G_adjust_Cell_head() by dividing (north - south) and (east - west) by
> the number of rows and columns.
> 
> The subtractions will result in an increase in relative error. The
> absolute error is determined by the precision of north, south, east
> and west. The results of the subtractions will have the same absolute
> error, but the magnitudes will be smaller by factors of 51e6 (n-s) and
> 17e6 (e-w), resulting in equivalent increases in absolute error.
> 
> Double-precision FP has ~15 decimal digits of precision, of which ~7
> are being lost, leaving ~8 digits of precision; which corresponds to
> the above.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 248 bytes
Desc: not available
Url : http://lists.osgeo.org/pipermail/grass-dev/attachments/20020424/11d74f21/attachment.bin