[gdal-dev] Reading signed 8-bit Erdas Imagine .img (HFA driver)

[Jason Roberts]

Frank,

Thanks for the pointer. The PIXELTYPE SIGNEDBYTE metadata looks like the
official way to detect signed 8-bit data. Unfortunately it does not seem to
work for me.

I used the ArcGIS 9.3.1 ASCII to Raster tool to convert the following ASCII
file to IMG format:

ncols 3               
nrows 3
xllcorner 0.0
yllcorner 0.0
cellsize 1.0
nodata_value 63
-127 63 126
-127 63 126
-127 63 126

The ArcGIS GUI claims the resulting IMG file is 8-bit signed with the NoData
value -128. I opened the file with GDAL 1.6.0:

>>> dataset = gdal.Open(r'C:\Temp\NoDataTest2\IMG\WithNoData\int8a.img')
>>> dataset.GetDriver().ShortName
'HFA'
>>> band = dataset.GetRasterBand(1)
>>> band.DataType
1
>>> data = band.ReadAsArray()
>>> data
array([[129, 128, 126],
       [129, 128, 126],
       [129, 128, 126]], dtype=uint8)
>>> band.GetMetadata()
{'LAYER_TYPE': 'athematic', 'STATISTICS_MAXIMUM': '126',
'STATISTICS_MINIMUM': '-127', 'STATISTICS_STDDEV': '109.55249883047',
'STATISTICS_HISTOBINVALUES': '3|3|', 'STATISTICS_MEAN': '-0.33333333333333',
'STATISTICS_MODE': '6.2891288834462e-247', 'STATISTICS_MEDIAN':
'6.2896744699688e-247'}

As you can see, there is no PIXELTYPE metadata entry for the band. Is that
entry only created when GDAL creates the file? In any case, it is possible
to detect that the true values fit in the int8 range, not the uint8 range:

>>> band.GetMinimum()
-127.0
>>> band.GetMaximum()
126.0

To get the true values, you can detect that band.GetMinimum() < 0 and cast
the data to int8:

>>> numpy.cast['int8'](data)
array([[-127, -128,  126],
       [-127, -128,  126],
       [-127, -128,  126]], dtype=int8)

The problem now is determining that -128 is the NoData value because GDAL
does not report it:

>>> band.GetNoDataValue()
0.0

0 is the lowest possible value in the uint8 range, so it sort of makes sense
to hard code the lowest possible int8 (-128) when band.GetNoDataValue()
returns 0. But I'm not sure what to do if another value was used.

Do you have any thoughts on how to improve this procedure? Thanks for any
advice you can provide.

Jason

-----Original Message-----
From: Frank Warmerdam [mailto:warmerdam at pobox.com] 
Sent: Tuesday, September 01, 2009 4:16 PM
To: Jason Roberts
Cc: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Reading signed 8-bit Erdas Imagine .img (HFA driver)

Jason Roberts wrote:
> GDAL only supports unsigned 8-bit integers (GDT_Byte). That is the 
> ultimate difficulty but it has been worked around before in other 
> drivers. Is there any possibility that the HFA driver could be changed 
> to work like the AIG driver, so that S8 data is returned as GDT_Int16 
> instead of GDT_Byte?
> 
> If not, how exactly does this driver support S8 .img files? If I had 
> some way of detecting that it was S8, I guess I could fix it up by 
> casting the array to int16 and subtracting 256 from all cells with 
> values > 127. But I know of no way to detect that it is S8 except by 
> calling an ArcGIS API or trying to parse .img format myself. Neither of 
> those are a good solution.

Jason,

GDAL does not treat signed bytes as a proper data type (with a GDT_* code)
but it does support setting the PIXELTYPE metadata item in the
IMAGE_STRUCTURE
domain to SIGNEDBYTE per RFC 14:

  http://trac.osgeo.org/gdal/wiki/rfc14_imagestructure

This gives application code an opportunity to do something special with the
data - such as casting to a signed byte type.  Some CreateCopy()
implementations for formats with signed byte support also look for this
metadata and translate the data as signed bytes.

We could in theory upgrade such bands to GDT_Int16 but it would require
quite a bit of messy code and it would not be in line with the approach
in other drivers.

Note, the HFA driver already returns the PIXELTYPE SIGNEDBYTE metadata
as do a few other drivers where appropriate.

Best regards,
-- 
---------------------------------------+------------------------------------
--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent