[gdal-dev] Motion: adopt RFC 87: Signed int8 data type for raster

[Idan Miara]

Just adding my two cents...

I highly support adding the signed int8 datatype - this would probably lead
to downstream projects that support different raster datatypes to
eventually fully support int8 rasters as a first class citizen.
Yes - downstream projects would need to invest some time adapting but the
result would be a PROPER support for int8 rasters.
 (I would imagine that searching the code for some key search words would
yield most places that need fixing),
Nobody is forced to upgrade a major version and sometimes a downstream
effort is required to support the evolution of the upstream project.

I was a maintainer of an app that used signed int8 data vastly and wanted
to incorporate save/load/calc rasters using GDAL.
After spending countless hours of trying to use the signed byte flag
properly in every case I ended up changing the logic of the application to
use only unsigned byte rasters since there were too many cases where the
signed byte flag was not honored or caused weird outputs (for instance, I
think that QGIS didn't fully support int8 rasters properly).

Even, Thanks for investing the time to make this right,
Idan

On Wed, 16 Nov 2022 at 21:39, Even Rouault <even.rouault at spatialys.com>
wrote:

>
>
> I think long term compatibility is a very desirable feature, and several
> applications just use GDAL as part of their code base without even being
> aware of what is happening in the GDAL development front. The same is true
> for several other libraries, given the fact the use of package managers
> (VCPKG, conan etc) are becoming more common. For some code bases, GDAL is
> just a small cog, and I believe it is very easy for the developers to
> simply download the next version from the package manager thinking that
> they are getting mostly bug fixes and not being aware of new silent
> incompatibilities.
>
> We try to avoid breakage or silent breakage when reasonable, but sometimes
> it is the best thing to do. People/software who depends on the past
> functionality will see the tests related to their code testing
> GetMetadataItem("PIXELTYPE", "IMAGE_STRUCTURE") == "SIGNEDBYTE" break and
> will probably figure out they should read GDAL release notes to understand
> what happens. It is not the first time nor the last one we will do that.
>
>
> Sorry, but I think it is funny that you mention removing GDT_Byte would
> have breaking implications, since I am expressing my reservations against a
> change that will have breaking implications. :)
>
> My perception is that > 90% of current uses of GDT_Byte are for the
> unsigned usage. Of course their might be some domains where this is not the
> case.
>
>
> I don't understand what limitations you are referring to. Perhaps the
> support was broken to some types of raster files?
>
> This is described in the RFC: " the absence of a proper data type means
> that it is easy to forget to test the PIXELTYPE=SIGNEDBYTE metadata item in
> all places where the value of pixels is taken into account. There were
> special cases for statistics computations, but most of the other code, such
> as the overview or warping computation code had no provision for it, and
> consequently mis-interpreted negative values in the range [-128,-1] as
> positive values in the range [128,255]." . So current support of signed
> 8-byte within GDAL was broken in a number of places, and the new GDT_Int8
> data type enables to fix it. One could argue that the past behavior of
> reporting a metadata item that altered the semantics of GDT_Byte was a
> silent breakage of the API promise of GDT_Byte being unsigned.
>
> In my field, signed 8-bit images representing categorical images are very
> common, given the fact they can be used fully decompressed and represent a
> null value/nodata in very convenient way (as a negative value).
>
> You should have a better experience with GDAL 3.7 then, pending perhaps a
> few changes in your code.
> Even
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20221117/f7426417/attachment.htm>