[GRASS-user] Precision of raster/vector float values

[Markus Metz]

On Fri, Aug 24, 2012 at 10:40 AM, Johannes Radinger
<johannesradinger at gmail.com> wrote:
> Hi,
>> So I misunderstood something about IEEE precision, sorry about the
>> confusion! Coming back to the original problem, it seems that it can
>> be solved for DCELL by using ".15g" instead of ".10f", i.e. use the
>> same formatting rules like e.g. r.what and d.what.rast. I tested and
>> the values uploaded to the database are correct with ".15g" for DCELL.
>>
>> Markus M
>
> sounds good if the origiinal problem can be solved for DCELL. If I
> understand you
> correctly this means that v.what.rast needs to be slighly changed?
> Do I need to do it myself or will there be an update in a new revision
> for 6.4.3 and/or 6.5?

In my tests, uploading e.g. "2.5e-5" instead of "0.000025" to
attribute tables works, but I am not sure if it works with all db
backends. I am also not sure if you will get the same value out after
some processing because sometimes "%lf" is used which prints by
default only 6 decimal places, I think. There are various binary <->
text conversions happening in the db library, and they do not seem to
be consistent. That is, depending on what you are going to do with the
attributes, it may or may not work. v.what and v.db.select should
work, though. v.what.rast is now fixed in all branches, r52869-71.

Markus M