[gdal-dev] problem with ArcSDE raster (negative values)

[Howard Butler]

On Dec 8, 2009, at 6:21 AM, Kor de Jong wrote:

> Hi Bart,
> 
> On Tuesday 08 December 2009 12:33:10 bartvde at osgis.nl wrote:
>> in ArcMap I see negative values, in GDAL all values seem to have a shift
>> of 256, so -21 becomes 235. Nodata values are also off with 256, so -256
>> in ArcMap becomes 0 in GDAL.
> 
> The pixel type of your input is char (signed 8 bit). The no-data value
> is -256 (which cannot be represented by the signed 8 bit, btw).
> 
> For some reason your data gets converted from a signed 8 bit type to an
> unsigned 8 bit type.
> 
> Looking at the trunk version of sderasterband.cpp I see that GDT_BYTE is
> taken to be the equivalent of SE_PIXEL_TYPE_8BIT_S. If this means that
> an input of signed 8 bit value is stored in an unsigned 8 bit value,
> than that would explain what you are seeing. An alternative would be to
> update this code to return GDT_Int16, which will leave your data intact.
> Line 833.

This is very likely the culprit.   Thanks Kor.  Please file a bug on this and I will try to apply it before the release.  I would note that I don't have SDE anymore to be able to test against, so I would have to depend on Bart's assessment of whether or not the issue is resolved.

Howard