[GRASS-dev] Re: [GRASS GIS] #335: export floats and doubles with correct precision

[GRASS GIS]

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter:  hamish       |       Owner:  grass-dev at lists.osgeo.org
      Type:  task         |      Status:  new                      
  Priority:  critical     |   Milestone:  6.4.0                    
 Component:  default      |     Version:  unspecified              
Resolution:               |    Keywords:  precision                
  Platform:  Unspecified  |         Cpu:  Unspecified              
--------------------------+-------------------------------------------------
Comment (by hamish):

 Hi,

 is there any reason not to change the following to %.15g?
 {{{
 raster/r.colors/
 rules.c:47:         sprintf(minstr, "%.25f", (double)min);
 rules.c:48:         sprintf(maxstr, "%.25f", (double)max);
 }}}
 AFAIK %.25f for a double (DCELL) is just reporting noise. It causes out-
 of-range white areas on a raster map because the 0%,100% color rules are
 no longer == the actual floating point value (50% chance it falls on the
 outside of the range)



 also, in lib/gis/color_write.c:
 {{{
 #define PRECISION 30
 #define THRESHOLD .0000000000000000000000000000005
 /* .5 * 10 ^(-30) */
 ...
 static int format_min(char *str, double dval)
 {
     double dtmp;

     sprintf(str, "%.*f", PRECISION, dval);
     G_trim_decimal(str);
     sscanf(str, "%lf", &dtmp);
     if (dtmp != dval) {         /* if  no zeros after decimal point were
 trimmed */
         sprintf(str, "%.*f", PRECISION, dval - THRESHOLD);
         /* because precision is probably higher than PRECISION */
     }

     return 0;
 }
 }}}

 (max is similar but adds the spillover threshold; but apparently this
 doesn't have the desired effect as 5e-31 is swept away in the noise
 anyway. ?)

 I blindly guess number originally came from an attempt to be a wee bit
 bigger than 1e-32. ????  (dates back to CVS rev1.1 so no help
 from the commit history)


 i.e. can we always assume DCELL|double to be the same bit depth, and %.15g
 the most we'll ever be able to squeeze from it?


 Hamish

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:6>
GRASS GIS <http://grass.osgeo.org>