[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

[GRASS GIS]

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
 Reporter:  hamish     |       Owner:  grass-dev@…              
     Type:  task       |      Status:  new                      
 Priority:  critical   |   Milestone:  6.4.4                    
Component:  Default    |     Version:  svn-develbranch6         
 Keywords:  precision  |    Platform:  All                      
      Cpu:  All        |  
-----------------------+----------------------------------------------------

Comment(by hamish):

 > > %.15,7g was chosen because it's perfectly reproducible
 >
 > No, it is not. %.17,9g is reproducible. See IEEE 754 standard.

 I didn't mean reproducible as far as the binary stored value round-trip
 was concerned, I meant reproducible as far as getting the same ascii
 result on two different hardware platforms.

 for example, getting r.md5sum (addon to aid the test_suite) to give the
 same (portable) result when checksumming r.out.ascii output when the data
 is stored as floating point &/or not strictly real (2.5000000). My
 understanding, for what it is, is that the least sig. digits can flicker
 depending on the CPU arch, perhaps the compiler, and programming
 language's implementation too.

 I can't remember which right now, but one of the ogr/gdal/proj4 input
 streams was introducing .00000000001 style artifacts, it's probably worth
 digging that up as a real-world test case example as
 there are probably two classes of this problem with the same symptom: one
 to do with GRASS's use of %f and atof() in places, and another to do with
 the output of input values which were malformed before they came into
 grass.


 Another interesting corner-case example to try with this is the earthquake
 demo from the grass-promo/tutorials/ svn, where the heavily logarithmic
 data scale within a single dataset really stretches what fits well in the
 ieee FP model, and so something more linear like magnitude is used to
 store it instead of raw extajoules.


 > A perfectly valid range of double fp data is e.g. 1.0e-200
 > to 1.0e-300, way smaller than GRASS_EPSILON. You do not want to
 > corrupt these data.

 (see the following post with the `2^1024` example where I corrected myself
 about that after seeing how you fixed the error in my earlier attempt to
 make sure the color min/max rule was bigger than the data range)


 regards,
 Hamish

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:26>
GRASS GIS <http://grass.osgeo.org>