[GRASS-user] Precision of raster/vector float values

[Johannes Radinger]

On Thu, Aug 23, 2012 at 3:23 PM, Markus Metz
<markus.metz.giswork at gmail.com> wrote:
> On Thu, Aug 23, 2012 at 2:40 PM, Johannes Radinger
> <johannesradinger at gmail.com> wrote:
>>>> I have a GRASS raster map which contains small values (e.g 2.5e-15) among
>>>> others.
>>>
>>> This is (dangerously) close to the IEEE limit of double precision
>>> floating point values. If GRASS tests for fp precision, it regards
>>> (absolute) values smaller than 1.0e-15 (GRASS_EPSILON) as zero.
>>>
>>> v.what.rast prints fp values with 10 decimal places. The equivalent
>>> issue has aready been fixed in d.what.rast, now also for v.what.rast
>>> in trunk r52857. For FCELL maps, it prints 7 decimal places, for DCELL
>>> maps it prints 15 decimal places. Printing means here that fp values
>>> are first converted to text, then uploaded to the attribute table.
>>>
>>
>> So if I understand you correctly: 1.0e-15 is the smallest number for
>> values in attribute columns although the values in a DCELL raster
>> can be  smaller? A bottleneck is v.what.rast which converts
>> the queried raster values into text with 15 decimal points?
>> So is there any (future) way to have similar precision in a DCELL
>> raster and its corresponding point vector (r.to.vect - v.what.rast)?
>
> The short answer is no because binary values have to be converted to
> text which for fp values is lossy.
>
Thank you for the answer. Good to know.

> The precision of double values in attribute tables is probably
> dependent on the module loading values to the table and maybe also the
> db driver.
>
>>
>> Or is there any other recommendable way to get a "list" resp. loopable
>> object that contains all raster cells (x,y and value) without loosing
>> the precision of the original raster?
>
> Hmm. Something with r.mapcalc? Any *.[in|out].ascii will suffer from
> the lossy conversion of binary -> text -> binary values.

I also though about looping over the cells via accessing the raster
as numpy array in python. Anyway I have to look into that to get
the e.g. also x and y for each cell beside the value etc. I'll try it ...

>
> Also be aware of the difference between precision and accuracy. For
> example, floating point DEM can have a precision in the nanometer
> range, but its accuracy may be in the meter range.
>
> In your case, rescaling could help.

Thank you for that explanation. In my case I am actually working with a
"virtual probabilistic surface" rather than with real elevation values etc. Thus
generally all values of my maps are between 0 and 1 with partly very small
values. Of course in this case rescaling might help and I'll go for
that approach for the moment.

/johannes

>
> Markus
>
>>
>> /Johannes
>>
>>> Markus M
>>>
>>>>
>>>> Some additional info: I am working on Ubuntu 12.04 whit GRASS 6.5SVN. My
>>>> point vector  is
>>>> connected to an sqlite db - table (standard sqlite connection).
>>>>
>>>> Of course I thought about a work around (e.g. multiplication by a large
>>>> scalar). Any other suggestions?
>>>>
>>>> Best regards,
>>>>
>>>> Johannes