[GRASS-dev] [GRASS GIS] #420: r.quantile returns wrong values

[GRASS GIS]

#420: r.quantile returns wrong values
------------------------+---------------------------------------------------
 Reporter:  jarekj71    |       Owner:  grass-dev at lists.osgeo.org
     Type:  defect      |      Status:  new                      
 Priority:  minor       |   Milestone:  6.4.0                    
Component:  default     |     Version:  svn-trunk                
 Keywords:  r.quantile  |    Platform:  Linux                    
      Cpu:  x86-32      |  
------------------------+---------------------------------------------------
 For spearfish database:
 g.region res=30


 r.info -r -s map=elevation.dem at PERMANENT
 min=1066
 max=1840
 nsres=30
 ewres=30

 r.quantile input=elevation.dem at PERMANENT
 percentiles=0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999 bins=1000000
 returns:
 0:0.001000:1067.000000
 1:0.010000:1070.000000
 2:0.100000:1080.000000
 3:0.250000:1086.000000
 4:0.500000:1091.000000
 5:0.750000:1095.000000
 6:0.900000:1098.000000
 7:0.990000:1099.000000
 8:0.999000:1099.000000

 While equivalent command in R:

 library(spgrass6)
 readRAST6("elevation.dem", plugin=FALSE, useGDAL=TRUE)

 > summary(r at data$elevation.dem)
    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's
    1066    1200    1316    1354    1488    1840    3294



 quantile(na.omit(r at data$elevation.dem),c(0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999))


 returns:

     0.1%       1%      10%      25%      50%      75%      90%      99%
 1080.000 1099.000 1156.000 1200.000 1316.000 1488.000 1621.000 1745.000
    99.9%
 1815.684

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/420>
GRASS GIS <http://grass.osgeo.org>