[gdal-dev] problem with ArcSDE raster (negative values)

[bartvde at osgis.nl]

Hi Kor,

I tried this change but with strange results. Also in Mapserver it seems
that a GByte * is used for the buffer:

LoadGDALImage( GDALRasterBandH hBand, int iColorIndex,
               layerObj *layer,
               int src_xoff, int src_yoff, int src_xsize, int src_ysize,
               GByte *pabyBuffer, int dst_xsize, int dst_ysize );

Would this need a change as well?

Best regards,
Bart

> Hi Bart,
>
> On Tuesday 08 December 2009 12:33:10 bartvde at osgis.nl wrote:
>> in ArcMap I see negative values, in GDAL all values seem to have a shift
>> of 256, so -21 becomes 235. Nodata values are also off with 256, so -256
>> in ArcMap becomes 0 in GDAL.
>
> The pixel type of your input is char (signed 8 bit). The no-data value
> is -256 (which cannot be represented by the signed 8 bit, btw).
>
> For some reason your data gets converted from a signed 8 bit type to an
> unsigned 8 bit type.
>
> Looking at the trunk version of sderasterband.cpp I see that GDT_BYTE is
> taken to be the equivalent of SE_PIXEL_TYPE_8BIT_S. If this means that
> an input of signed 8 bit value is stored in an unsigned 8 bit value,
> than that would explain what you are seeing. An alternative would be to
> update this code to return GDT_Int16, which will leave your data intact.
> Line 833.
>
> Special care may have to be taken to handle the no-data values
> correctly.
>
> Probably the original author will have something to say about this.
>
> Kor
>
>
>