[GRASS-dev] [GRASS GIS] #1747: v.rast.stats on dbf shapefile not enough precision even with DOUBLE PRECISION

[GRASS GIS]

#1747: v.rast.stats on dbf shapefile not enough precision even with DOUBLE
PRECISION
--------------------------+-------------------------------------------------
 Reporter:  vesnikos      |       Owner:  grass-dev@…              
     Type:  defect        |      Status:  new                      
 Priority:  normal        |   Milestone:  6.4.3                    
Component:  Vector        |     Version:  6.4.2                    
 Keywords:  v.rast.stats  |    Platform:  Linux                    
      Cpu:  x86-32        |  
--------------------------+-------------------------------------------------

Comment(by hamish):

 Replying to [comment:13 ElcoL]:
 > I reproduced the error with the NC dataset:

 ok, thanks, I can reproduce it now too.

 (best to work in a user mapset, and not modify the maps in PERMANENT, make
 a copy and work on the copy)


 It happens because the awk command specifically sets the print format to
 "%.2f".
 https://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/scripts/v.rast.stats/v.rast.stats#L363

 so you only get 2 digits after the decimal place.

 seems this came in with r48058 as part of a bulk merge with another
 (faster) version of the script, so in 6.4.2+ but not the old+slow 6.4.1
 version. I don't see a reason to keep it as %.2f instead of
 %.{something}g, any hints Otto?


 Hamish

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/1747#comment:14>
GRASS GIS <http://grass.osgeo.org>