[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

GRASS GIS trac at osgeo.org
Wed Jun 25 01:14:11 PDT 2014


#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
 Reporter:  hamish     |       Owner:  grass-dev@…              
     Type:  task       |      Status:  new                      
 Priority:  critical   |   Milestone:  6.4.5                    
Component:  Default    |     Version:  svn-develbranch6         
 Keywords:  precision  |    Platform:  All                      
      Cpu:  All        |  
-----------------------+----------------------------------------------------
Changes (by hamish):

  * milestone:  6.4.4 => 6.4.5


Comment:

 Replying to [comment:31 mlennert]:
 > Hamish, what exactly needs to be continued here ?

 the systematic audit and repair of modules printf'ing double and single
 precision FP variables with inappropriate %.*g or %.*f, either arbitrarily
 fixed-precision or using a precision appropriate for the data type.


 > Is it fixing those modules on your list that haven't been fixed, yet ?

 yes, exactly. spin boxes and (eg rectifier) table throughout the wxGUI is
 related, but probably should have its own ticket.


 > Or is it the discussion on how to handle decimals ?

 I doubt that has an end, but at least we can all agree that %.52f is
 meaningless and %f is too lossy.

 The question of whether to output values with slight rounding, user
 selectable precision, or try for exact ascii representation of IEEE double
 FP is I think a case by case question.

 e.g. %.15g will avoid outputting coordinates like `278.700000000000045
 44.8999999999999986`, which is human-ugly and bloats the ascii filesize a
 lot, but reversible to the same IEEE double FP value.

 fwiw I think we can achieve good defaults so user selectable precision is
 rarely needed (success is if no one ever thinks to ask for it, since the
 right thing was done automatically), and there are times like the colr/
 file when %.15g max and min range rounded outwards by 1 epsilon step can
 tidy up the out of range trouble which happens from time to time.


 the bug is of high priority as data input/output needs to be flawless, or
 of such minor jitter that in most cases a nanometer here or there won't
 harm the overall result.


 regards,
 Hamish

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:32>
GRASS GIS <http://grass.osgeo.org>



More information about the grass-dev mailing list