[GRASS-user] Precision of raster/vector float values

Johannes Radinger johannesradinger at gmail.com
Thu Aug 23 05:40:41 PDT 2012


>> I have a GRASS raster map which contains small values (e.g 2.5e-15) among
>> others.
>
> This is (dangerously) close to the IEEE limit of double precision
> floating point values. If GRASS tests for fp precision, it regards
> (absolute) values smaller than 1.0e-15 (GRASS_EPSILON) as zero.
>
> v.what.rast prints fp values with 10 decimal places. The equivalent
> issue has aready been fixed in d.what.rast, now also for v.what.rast
> in trunk r52857. For FCELL maps, it prints 7 decimal places, for DCELL
> maps it prints 15 decimal places. Printing means here that fp values
> are first converted to text, then uploaded to the attribute table.
>

So if I understand you correctly: 1.0e-15 is the smallest number for
values in attribute columns although the values in a DCELL raster
can be  smaller? A bottleneck is v.what.rast which converts
the queried raster values into text with 15 decimal points?
So is there any (future) way to have similar precision in a DCELL
raster and its corresponding point vector (r.to.vect - v.what.rast)?

Or is there any other recommendable way to get a "list" resp. loopable
object that contains all raster cells (x,y and value) without loosing
the precision of the original raster?

/Johannes

> Markus M
>
>>
>> Some additional info: I am working on Ubuntu 12.04 whit GRASS 6.5SVN. My
>> point vector  is
>> connected to an sqlite db - table (standard sqlite connection).
>>
>> Of course I thought about a work around (e.g. multiplication by a large
>> scalar). Any other suggestions?
>>
>> Best regards,
>>
>> Johannes


More information about the grass-user mailing list