[GRASS-user] Precision of raster/vector float values

[Markus Metz]

On Thu, Aug 23, 2012 at 11:52 PM, Glynn Clements
<glynn at gclements.plus.com> wrote:
>
> Markus Metz wrote:
>
>> > So if I understand you correctly: 1.0e-15 is the smallest number for
>> > values in attribute columns although the values in a DCELL raster
>> > can be  smaller? A bottleneck is v.what.rast which converts
>> > the queried raster values into text with 15 decimal points?
>> > So is there any (future) way to have similar precision in a DCELL
>> > raster and its corresponding point vector (r.to.vect - v.what.rast)?
>>
>> The short answer is no because binary values have to be converted to
>> text which for fp values is lossy.
>
> Note that the round-trip conversion isn't *inherently* lossy.
>
> It's possible to convert any binary floating-point value to decimal
> exactly.
>
> Arbitrary decimal values cannot be converted to (binary)
> floating-point values exactly, but decimal values which were generated
> from a floating-point value will convert back to the original value
> provided that sufficient decimal digits are used. The decimal
> representation doesn't have to be exact, only sufficiently accurate to
> be distinguishable from the adjacent floating-point values.
>
> Conversion to and from text is only lossy because the number of digits
> required to avoid this is excessive for any other purpose (double
> precision can represent the circumference of the earth to within 9
> nanometres; no actual measurement will be that precise).

So I misunderstood something about IEEE precision, sorry about the
confusion! Coming back to the original problem, it seems that it can
be solved for DCELL by using ".15g" instead of ".10f", i.e. use the
same formatting rules like e.g. r.what and d.what.rast. I tested and
the values uploaded to the database are correct with ".15g" for DCELL.

Markus M