[gdal-dev] Possible statistics computation issue

[Alex Speder]

Greetings all,

I recently attempted to update a Java project from GDAL 1.8.1 to GDAL
1.9.  However, I noticed the statistics being computed on my output
rasters were off.  After doing some digging, it seems that something
changed in the HFA driver's handling of minimum and maximum values.
IN GDAL 1.8.1 I could set a Float32 raster's NoData value to
Float.NEGATIVE_INFINITY and it would work as expected, i.e. pixels
with NoData values were ignored.  This no longer appears to be the
case in GDAL 1.9.  I have only seen this behavior with
Float.NEGATIVE_INFINITY or Float.POSITIVE_INFINITY as the NoData
(although with positive infinity the problem is reversed).  Is this
the intended behavior?

I've tested this on both Windows XP (32 bit) and Linux (both 32 and 64
bit) with the same results.  Of possible note, the GTiff driver in
GDAL 1.8.1 seems to exhibit this behavior as well, which makes me
think the HFA change might be intended.

Please note that the sample GDAL output's test raster is different
from, but exhibits the same problem as, rasters created with the
sample code.   Sorry in advance for the wall of text.

Java code to generate such rasters
----------------------------------
public static void main(String[] args) {
  gdal.AllRegister();
  Dataset writeDs =
gdal.GetDriverByName("HFA").Create("path//to//file.img", 100, 100, 1,
gdalconst.GDT_Float32);
  Band writeBand = writeDs.GetRasterBand(1);
  writeBand.SetNoDataValue(Float.NEGATIVE_INFINITY);
  double[] data = new double[10000];
  final Random gen = new Random();
  for(int i = 0; i < data.length; i++) {
    int val = gen.nextInt(3);
    data[i] = val == 2 ? Float.NEGATIVE_INFINITY : val;
  }
  writeBand.WriteRaster(0, 0, 100, 100, data);
  writeBand.delete();
  writeDs.delete();
}

GDAL 1.8.1 output
-----------------
[ywil at eli bin]$ ./gdalinfo test.img -stats -hist
Driver: HFA/Erdas Imagine Images (.img)
Files: test.img
Size is 177, 207
[...snip...]
Band 1 Block=64x64 Type=Float32, ColorInterp=Undefined
 Description = Layer_1
 Minimum=-0.000, Maximum=1.000, Mean=0.015, StdDev=0.113
0...10...20...30...40...50...60...70...80...90...100 - done.
 256 buckets from -0.00195313 to 1.00195:
 34184 1775 0 0 424 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33
0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39
17 0 0 0 77 0 0 0 0 0 0 0 0 0 0 0 46
 NoData Value=-inf
 Metadata:
    LAYER_TYPE=athematic
    STATISTICS_MINIMUM=-1.8190514199101e-12
    STATISTICS_MAXIMUM=1
    STATISTICS_MEAN=0.015357797876846
    STATISTICS_STDDEV=0.11342873850486

GDAL 1.9 output
---------------
[ywil at eli ~]$ gdalinfo test.img -stats -hist
Driver: HFA/Erdas Imagine Images (.img)
Files: test.img
Size is 177, 207
[...snip...]
Band 1 Block=64x64 Type=Float32, ColorInterp=Undefined
 Description = Layer_1
 Minimum=-inf, Maximum=1.000, Mean=-inf, StdDev=-nan
0...10...20...30...40...50...60...70...80...90...100 - done.
 256 buckets from -inf to inf:
 36639 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
 NoData Value=-inf
 Metadata:
    LAYER_TYPE=athematic
    STATISTICS_MAXIMUM=1
    STATISTICS_MEAN=-inf
    STATISTICS_MINIMUM=-inf
    STATISTICS_STDDEV=-nan

--
Alex Speder
alex.speder at gmail.com