[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

[GRASS GIS]

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
 Reporter:  hamish     |       Owner:  grass-dev@…              
     Type:  task       |      Status:  new                      
 Priority:  critical   |   Milestone:  6.4.4                    
Component:  Default    |     Version:  svn-develbranch6         
 Keywords:  precision  |    Platform:  All                      
      Cpu:  All        |  
-----------------------+----------------------------------------------------

Comment(by glynn):

 Replying to [comment:26 hamish]:

 > > No, it is not. %.17,9g is reproducible. See IEEE 754 standard.
 >
 > I didn't mean reproducible as far as the binary stored value round-trip
 was concerned, I meant reproducible as far as getting the same ascii
 result on two different hardware platforms.

 All common platforms use IEEE-754 representation, so I wouldn't expect
 differences due to hardware.

 > My understanding, for what it is, is that the least sig. digits can
 flicker depending on the CPU arch, perhaps the compiler, and programming
 language's implementation too.

 Any differences are due to software, not hardware. The main reasons for
 differences are:

 1. Whether the software produces the closest decimal value to the actual
 binary value, or the shortest decimal value which would convert to the
 actual binary value (those two aren't necessarily the same).

 2. The rounding mode used in the event of a tie. E.g. 3.0/16=0.1875; if
 that is displayed with 3 fractional digits, should the result be 0.187
 (round toward zero, round toward negative infinity) or 0.188 (round toward
 positive infinity, round toward nearest even value)?

 3. Bugs. Many programmers don't understand the details of floating-point,
 and don't particularly care whether the least-significant digits are
 "correct" (particularly as that would requiring which flavour of "correct"
 you actually want).

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:27>
GRASS GIS <http://grass.osgeo.org>