[gdal-dev] Motion: adopt RFC 87: Signed int8 data type for raster

[Hermann Rodrigues]

Kurt,

In the ideal world, everything has a unit test. In the real world, there
are a lot of applications that keep things turning around that were not
developed with the same level of care. And yet, we use them everyday. We
have unit tests and a lot of them, but not everybody or everything does.

The RFC approval was voted by a number of developers that I don't believe
represents the number of developers using GDAL as part of their
applications. Perhaps breaking chances should require a more significant
number of votes.

I dare to say that, if we are following this path, maybe GDAL should start
marketing itself as a set of command line tools and not as a library, given
that breaking compatibility just because we can is not something a library
developer should do, IMHO.

Best,

hermann

----------------------------------------------------------------------
Hermann Rodrigues
hermann.rodrigues at gmail.com
hermann at csr.ufmg.br
Twitter: @horodrigues | @dinamica_ego
Centro de Sensoriamento Remoto / UFMG
https://csr.ufmg.br | https://dinamicaego.com


On Wed, Nov 16, 2022 at 2:41 PM Kurt Schwehr <schwehr at gmail.com> wrote:

> Hermann,
>
> Speaking to GDAL being one (not very small) piece of an ecosystem of
> libraries and tools:
>
> This is exactly what unit tests are for.  A well done tool or library
> should have the tests that cover their critical uses of libraries they
> depend on.  It's often referred to as the "Beyonce rule": "If you liked it,
> you should have put a test on it." (c.f. here
> <https://www.oreilly.com/library/view/software-engineering-at/9781492082781/ch01.html>
> )
>
> Now is a great time to make sure that your workflow has a check for proper
> handling of 8-bit rasters.
>
> -kurt
>
> On Wed, Nov 16, 2022 at 11:39 AM Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>>
>>
>> I think long term compatibility is a very desirable feature, and several
>> applications just use GDAL as part of their code base without even being
>> aware of what is happening in the GDAL development front. The same is true
>> for several other libraries, given the fact the use of package managers
>> (VCPKG, conan etc) are becoming more common. For some code bases, GDAL is
>> just a small cog, and I believe it is very easy for the developers to
>> simply download the next version from the package manager thinking that
>> they are getting mostly bug fixes and not being aware of new silent
>> incompatibilities.
>>
>> We try to avoid breakage or silent breakage when reasonable, but
>> sometimes it is the best thing to do. People/software who depends on the
>> past functionality will see the tests related to their code testing
>> GetMetadataItem("PIXELTYPE", "IMAGE_STRUCTURE") == "SIGNEDBYTE" break and
>> will probably figure out they should read GDAL release notes to understand
>> what happens. It is not the first time nor the last one we will do that.
>>
>>
>> Sorry, but I think it is funny that you mention removing GDT_Byte would
>> have breaking implications, since I am expressing my reservations against a
>> change that will have breaking implications. :)
>>
>> My perception is that > 90% of current uses of GDT_Byte are for the
>> unsigned usage. Of course their might be some domains where this is not the
>> case.
>>
>>
>> I don't understand what limitations you are referring to. Perhaps the
>> support was broken to some types of raster files?
>>
>> This is described in the RFC: " the absence of a proper data type means
>> that it is easy to forget to test the PIXELTYPE=SIGNEDBYTE metadata item in
>> all places where the value of pixels is taken into account. There were
>> special cases for statistics computations, but most of the other code, such
>> as the overview or warping computation code had no provision for it, and
>> consequently mis-interpreted negative values in the range [-128,-1] as
>> positive values in the range [128,255]." . So current support of signed
>> 8-byte within GDAL was broken in a number of places, and the new GDT_Int8
>> data type enables to fix it. One could argue that the past behavior of
>> reporting a metadata item that altered the semantics of GDT_Byte was a
>> silent breakage of the API promise of GDT_Byte being unsigned.
>>
>> In my field, signed 8-bit images representing categorical images are very
>> common, given the fact they can be used fully decompressed and represent a
>> null value/nodata in very convenient way (as a negative value).
>>
>> You should have a better experience with GDAL 3.7 then, pending perhaps a
>> few changes in your code.
>> Even
>>
>> -- http://www.spatialys.com
>> My software is free, but my time generally not.
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20221116/58e9a6d3/attachment.htm>