[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

[GRASS GIS]

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
 Reporter:  hamish     |       Owner:  grass-dev@…              
     Type:  task       |      Status:  new                      
 Priority:  critical   |   Milestone:  6.4.4                    
Component:  Default    |     Version:  svn-develbranch6         
 Keywords:  precision  |    Platform:  All                      
      Cpu:  All        |  
-----------------------+----------------------------------------------------

Comment(by hamish):

 Replying to [comment:21 mmetz]:
 > Glynn: Binary to text conversions must use %.9g (float) and %.17g
 > (double) in order to preserve fp values in a binary-decimal-binary
 > round-trip, e.g. r.out.ascii followed by r.in.ascii.

 take care that preserving b-d-b round trip ,on a single platform, is not
 the only task or consideration.  For the r.out.ascii + r.in.ascii round
 trip it may well be appropriate, but while conceding that point I'd argue
 that r.*.bin or r.*.mat would be a better choice in those cases.

 %.15,7g was chosen because it's perfectly reproducible and doesn't
 introduce useless .0000000000000001 crap into the data files which
 G_trim_decimal() can't handle. For things like r.univar, r.info, and
 v.db.select the output is at least in part for human consumption; there's
 no practical need to expose the FP noise. The main thing for us to focus
 on I think is all the remaining lossy %f and meaningless %.56f type stuff
 in the code, not splitting hairs over preserving quanta finer than
 GRASS_EPSILON.


 wrt lib/gis/color_write.c, look closely & you'll see there is a +/-
 GRASS_EPSILON adjustment on the range to ensure that the rounding exceeds
 the range, and so you shouldn't ever get white spots at the peaks and
 pits.


 best,
 Hamish

 (this is something I hope python cleans up with 3.0)

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:23>
GRASS GIS <http://grass.osgeo.org>