[GRASS-user] Precision of raster/vector float values

[Markus Metz]

On Thu, Aug 23, 2012 at 10:35 AM, Johannes Radinger
<johannesradinger at gmail.com> wrote:
> Hi,
>
> I have a GRASS raster map which contains small values (e.g 2.5e-15) among
> others.

This is (dangerously) close to the IEEE limit of double precision
floating point values. If GRASS tests for fp precision, it regards
(absolute) values smaller than 1.0e-15 (GRASS_EPSILON) as zero.

> When I try to update a attribute column (Type  = Float) of a corresponding
> point vector (using v.what.rast), all small values turn in "0".
> Thus I am curious about the precision of the float values of a vector
> file which is probably different from the precision of raster floats. What
> are the smallest
> number that get correctly updated by v.what.rast?? This might be related to
> the capabilities
> of the underlying db-connection.

v.what.rast prints fp values with 10 decimal places. The equivalent
issue has aready been fixed in d.what.rast, now also for v.what.rast
in trunk r52857. For FCELL maps, it prints 7 decimal places, for DCELL
maps it prints 15 decimal places. Printing means here that fp values
are first converted to text, then uploaded to the attribute table.

Markus M

>
> Some additional info: I am working on Ubuntu 12.04 whit GRASS 6.5SVN. My
> point vector  is
> connected to an sqlite db - table (standard sqlite connection).
>
> Of course I thought about a work around (e.g. multiplication by a large
> scalar). Any other suggestions?
>
> Best regards,
>
> Johannes
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
>