[gdal-dev] problem with ArcSDE raster (negative values)

[Kor de Jong]

Hi Bart,

On Tuesday 08 December 2009 12:33:10 bartvde at osgis.nl wrote:
> in ArcMap I see negative values, in GDAL all values seem to have a shift
> of 256, so -21 becomes 235. Nodata values are also off with 256, so -256
> in ArcMap becomes 0 in GDAL.

The pixel type of your input is char (signed 8 bit). The no-data value
is -256 (which cannot be represented by the signed 8 bit, btw).

For some reason your data gets converted from a signed 8 bit type to an
unsigned 8 bit type.

Looking at the trunk version of sderasterband.cpp I see that GDT_BYTE is
taken to be the equivalent of SE_PIXEL_TYPE_8BIT_S. If this means that
an input of signed 8 bit value is stored in an unsigned 8 bit value,
than that would explain what you are seeing. An alternative would be to
update this code to return GDT_Int16, which will leave your data intact.
Line 833.

Special care may have to be taken to handle the no-data values
correctly.

Probably the original author will have something to say about this.

Kor