[GRASS-dev] [GRASS GIS] #1668: r.regression.line F-test incorrect

[GRASS GIS]

#1668: r.regression.line F-test incorrect
-------------------------+--------------------------------------------------
 Reporter:  cmbarton     |       Owner:  grass-dev@…              
     Type:  defect       |      Status:  new                      
 Priority:  normal       |   Milestone:  7.0.0                    
Component:  Raster       |     Version:  unspecified              
 Keywords:               |    Platform:  Unspecified              
      Cpu:  Unspecified  |  
-------------------------+--------------------------------------------------

Comment(by mlennert):

 Replying to [ticket:1668 cmbarton]:
 > One of my students noticed that the "F-test" in r.regression.line does
 not seem to be calculating F, but instead calculating -(R squared).
 >
 > For example, from the Spearfish demo data,
 >
 > {{{
 >
 > r.regression.line map1=elevation.dem at PERMANENT map2=slope at PERMANENT
 > y = a + b*x
 >    a (Offset): -16.675093
 >    b (Gain): 0.020833
 >    R (sumXY - sumX*sumY/N): 0.481666
 >    N (Number of elements): 2611107
 >    F (F-test significance): -0.232002
 >    meanX (Mean of map1): 1353.724982
 >    sdX (Standard deviation of map1): 176.754565
 >    meanY (Mean of map2): 11.527723
 >    sdY (Standard deviation of map2): 7.645157
 >
 > }}}
 >
 > 0.481666^2 = 0.232002

 The values only seem the same because of rounding.

 However, the formula for calculating the statistic does not seem correct
 in the code. IIUC, instead of


 {{{
 F = R * R / (1 - R * R / count - 2);
 }}}

 I think it should be

 {{{
 F = R * R / (1 - R * R) / (count - 2);
 }}}

 but this should be checked by a statistician.

 Moritz

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/1668#comment:1>
GRASS GIS <http://grass.osgeo.org>