[GRASS-dev] Re: [GRASS GIS] #420: r.quantile returns wrong values

[GRASS GIS]

#420: r.quantile returns wrong values
-----------------------+----------------------------------------------------
  Reporter:  jarekj71  |       Owner:  grass-dev at lists.osgeo.org
      Type:  defect    |      Status:  closed                   
  Priority:  minor     |   Milestone:  6.4.0                    
 Component:  default   |     Version:  svn-trunk                
Resolution:  invalid   |    Keywords:  r.quantile               
  Platform:  Linux     |         Cpu:  x86-32                   
-----------------------+----------------------------------------------------
Changes (by glynn):

  * status:  new => closed
  * resolution:  => invalid

Comment:

 Replying to [ticket:420 jarekj71]:
 > For spearfish database:
 > g.region res=30

 > r.quantile input=elevation.dem at PERMANENT
 > percentiles=0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999 bins=1000000
 > returns:

 The arguments to the percentiles= option are, well, percentiles, i.e. the
 values given are divided by 100 to get fractions. So 0.99 is 0.99% not
 99%.

 Using correct values gives results which essentially agree with R:
 {{{
 $ r.quantile input=elevation.dem at PERMANENT
 percentiles=0.1,1,10,25,50,75,90,99,99.9 bins=1000000
 Computing histogram
 Computing bins
 Binning data
 Sorting bins
 Computing quantiles
 0:0.100000:1080.000000
 1:1.000000:1099.000000
 2:10.000000:1156.000000
 3:25.000000:1200.000000
 4:50.000000:1316.000000
 5:75.000000:1488.000000
 6:90.000000:1621.000000
 7:99.000000:1745.000000
 8:99.900000:1816.000000
 }}}

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/420#comment:1>
GRASS GIS <http://grass.osgeo.org>