[gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header

[Peter Willis]

Hello,

When I run :

Gdalinfo -stats  <FILENAME>

One file (success) provides statistics calculated by excluding the no data values.
Ie: min=0.25 max=1234.0 mean=12.0

The other file (failed) provides statistics with -999 calculated into the stats.
Ie: min = -999.0 max=1234.0 mean=-993.0

I have looked at both binary files. Each of the two files contain the mask value.
ENVI loads both files and headers fine and reports the same no data value, for each file, in header info just fine.

I think you could probably model the problem by generating a much smaller file with multiple channels.

The steps happening are as follows:

1.) Original file (IN_IMG) is UTM projection ENVI format 49 bands BSQ, float32

2.) gdalwarp -of ENVI -t_srs "+proj=latlong +datum=WGS84" ${IN_IMG} ${OUT_IMG}

3.) put the data ignore value back in the file:
        echo "" >> $OUT_HDR
        echo "data ignore value = -9.99000000e+002" >> $OUT_HDR

4.) Call gdalinfo for number of bands in original file => $BANDS
5.) loop through bands seq 1 $BANDS => $BAND
          gdal_rasterize  -b $BAND -burn -999 -l ${LAYER_NAME}  ${MASK_SHP} ${OUT_IMG}
6.) gdalinfo -stats ${OUT_IMG}

At step #6,  gdalinfo -stats reports data stats with the -999 values calculated into the min/max/mean/stdev.


Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined
  Description = Band 2
  Min=-999.000 Max=12.250
  Minimum=-999.000, Maximum=12.250, Mean=-946.817, StdDev=222.404
  NoData Value=-999
  Metadata:
    STATISTICS_MINIMUM=-999
    STATISTICS_MAXIMUM=12.25
    STATISTICS_MEAN=-946.81678963104
    STATISTICS_STDDEV=222.4036660914


If I perform gdalinfo -stats on the original UTM file the min/max/mean/stdev  exclude -999 from calculations.    

Band 2 Block=1677x1 Type=Float32, ColorInterp=Undefined
  Description = Band Math (Band Math (Band Math (1996_104 Landsat-4/5 TM :Surface Temperature Sensor:6:LSatThermalCube_96-98.img):LSatThermalCube_96-98_float.img):LSatThermalCube_96-98_float_cmsk.img)
  Min=0.250 Max=12.250
  Minimum=0.250, Maximum=12.250, Mean=1.379, StdDev=0.995
  NoData Value=-999
  Metadata:
    STATISTICS_MINIMUM=0.25
    STATISTICS_MAXIMUM=12.25
    STATISTICS_MEAN=1.3787828985307
    STATISTICS_STDDEV=0.99545771926881


I hope this helps. 
Let me know if you need any more information.

Peter


-----Original Message-----
From: Even Rouault [mailto:even.rouault at mines-paris.org] 
Sent: October-19-12 2:08 PM
To: gdal-dev at lists.osgeo.org
Cc: Peter Willis
Subject: Re: [gdal-dev] gdalinfo -stats Misses invalid data value in ENVI header

Le vendredi 19 octobre 2012 22:57:13, Peter Willis a écrit :
> Hello,
> 
> The one listed as 'failed'  actually fails for me as it is sent .

Just to be sure we are talking about the same thing, what do you call "fail" 
exactly ? Is it that gdalinfo doesn't report "NoData Value=-999" for the bands ?

Just for reference, I've create a fake
"LSatThermalCube_96-98_float_cmsk_landmasked_ll.img" and  gdalinfo on it reports :

Driver: ENVI/ENVI .hdr Labelled
Files: LSatThermalCube_96-98_float_cmsk_landmasked_ll.img
       LSatThermalCube_96-98_float_cmsk_landmasked_ll.hdr
Size is 2178, 1841
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        TOWGS84[0,0,0,0,0,0,0],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9108"]],
    AUTHORITY["EPSG","4326"]]
Origin = (-124.258818822248998,51.670841529998398)
Pixel Size = (0.000337173258152,-0.000337173258152)
Metadata:
[.... cut ....]
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (-124.2588188,  51.6708415) (124d15'31.75"W, 51d40'15.03"N) Lower Left  (-124.2588188,  51.0501056) (124d15'31.75"W, 51d 3' 0.38"N) Upper Right (-123.5244555,  51.6708415) (123d31'28.04"W, 51d40'15.03"N) Lower Right (-123.5244555,  51.0501056) (123d31'28.04"W, 51d 3' 0.38"N)
Center      (-123.8916371,  51.3604735) (123d53'29.89"W, 51d21'37.70"N)
Band 1 Block=2178x1 Type=Float32, ColorInterp=Undefined
  Description = Band 1
  NoData Value=-999
Band 2 Block=2178x1 Type=Float32, ColorInterp=Undefined
  Description = Band 2
  NoData Value=-999
[... cut ... ]
Band 49 Block=2178x1 Type=Float32, ColorInterp=Undefined
  Description = Band 49
  NoData Value=-999