[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

[GRASS GIS]

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
 Reporter:  hamish     |       Owner:  grass-dev@…              
     Type:  task       |      Status:  new                      
 Priority:  critical   |   Milestone:  6.4.4                    
Component:  Default    |     Version:  svn-develbranch6         
 Keywords:  precision  |    Platform:  All                      
      Cpu:  All        |  
-----------------------+----------------------------------------------------

Comment(by mmetz):

 Replying to [comment:23 hamish]:
 > Replying to [comment:21 mmetz]:
 > > Glynn: Binary to text conversions must use %.9g (float) and %.17g
 > > (double) in order to preserve fp values in a binary-decimal-binary
 > > round-trip, e.g. r.out.ascii followed by r.in.ascii.
 >
 > take care that preserving b-d-b round trip ,on a single platform, is not
 the only task or consideration.  For the r.out.ascii + r.in.ascii round
 trip it may well be appropriate, but while conceding that point I'd argue
 that r.*.bin or r.*.mat would be a better choice in those cases.

 Please add this information to the manuals if you think this is crucial.

 >
 > %.15,7g was chosen because it's perfectly reproducible

 No, it is not. %.17,9g is reproducible. See IEEE 754 standard.

 > and doesn't introduce useless .0000000000000001 crap

 On what platform do you get this crap? %.17,9g does not produce this crap.

 > into the data files which G_trim_decimal() can't handle.

 G_trim_decimal() is not needed for %.17,9g.

 > The main thing for us to focus on I think is all the remaining lossy %f
 and meaningless %.56f type stuff in the code

 Sure, therefore %.17,9g.

 > not splitting hairs over preserving quanta finer than GRASS_EPSILON.

 Huh? This thread is not about GRASS_EPSILON. GRASS_EPSILON is 1.0e-15. A
 perfectly valid range of double fp data is e.g. 1.0e-200 to 1.0e-300, way
 smaller than GRASS_EPSILON. You do not want to corrupt these data.

 BTW, I found that fp calculation errors are rather in the range of 1.0e-4
 to 1.0e-8.

 Markus M

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:25>
GRASS GIS <http://grass.osgeo.org>