[GRASS-dev] r.in.gdal fp precision loss

[Glynn Clements]

Markus Metz wrote:

> I think I understand. If the largest values are in the range of 2^6 -
> 2^7, the ULP would have a magnitude of 2^-17 ~= 7.629395e-06?

Yes.

> I am
> asking because lately I observed slightly corrupted vector coordinates
> in shapefiles created with recent ESRI software and suspect
> float-double conversion artifacts. For latlon coordinates, the largest
> values are in the range of 2^6 - 2^8, depending on the longitude.
> v.in.ogr could then, based on the largest coordinate values, suggest a
> threshold for snapping if topological errors are encountered.

If cartographic projections or datum transformations are involved,
there are likely to be more significant issues than floating-point
rounding errors.

The main problem is that, for a non-affine transformation, the
transformation of a line between two endpoints isn't the same as the
line between the transformed endpoints.

So e.g. if you have points A, B, and C, where C lies exactly on the
line AB, transforming the points to A', B', and C' typically won't
result in C' lying on the line A'B'.

-- 
Glynn Clements <glynn at gclements.plus.com>