<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal-compose;
font-family:"Arial",sans-serif;
color:black;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US">I concur that signed byte data is fairly common and we have to handle it in our Python application. The recent addition of Int64 types actually broke the
dictionary-based way we were converting between numpy and GDAL datatypes and we had to implement a more robust solution.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US">Is it the case that gdal.GetDataTypeName and gdal.GetDataTypeByName currently return the wrong values – they suggest np.int8 is equivalent to gdal.GDT_Byte
when it should be np.uint8. Hopefully adding a signed 8bit type should clear this up. Having to also check the PIXELTYPE metadata isn’t always convenient so it will be better to have a more consistent conversion.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US">Jon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div dir="ltr" style="mso-line-height-rule:exactly;-webkit-text-size-adjust:100%;font-size:1px;direction:ltr;"><table cellpadding="0" cellspacing="0" border="0" style="width:100%;font-size:1px;"><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;"><tr style="font-size:0;"><td align="left" style="padding:10px 0 0;vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;color:#4A4A49;font-style:normal;font-weight:700;white-space:nowrap;"><tr style="font-size:14.67px;"><td align="left" style="padding:0;vertical-align:top;font-family:Arial;">e: <span style="font-family:remialcxesans;font-size:1px;color:#FFFFFF;line-height:1px;"></span></td><td align="left" style="padding:0 0 0 4px;vertical-align:top;color:#F6A124;font-family:Arial;font-weight:400;"><a href="mailto:Jon.Morris@jbarisk.com" target="_blank" id="LPlnk689713" style="text-decoration:none;color:#F6A124;"><strong style="font-weight:400;">Jon.Morris@jbarisk.com</strong></a></td></tr><tr style="font-size:14.67px;"><td align="left" style="padding:0;vertical-align:top;font-family:Arial;">d:</td><td align="left" style="padding:0 0 0 4px;vertical-align:top;color:#F6A124;font-family:Arial;font-weight:400;">+44 (0)1756 587229<br /></td></tr><tr style="font-size:14.67px;"><td align="left" style="padding:0;vertical-align:top;font-family:Arial;">t: </td><td align="left" style="padding:0 0 0 4px;vertical-align:top;color:#F6A124;font-family:Arial;font-weight:400;"><a href="tel:+44%20(0)1756%20799919" target="_blank" id="LPlnk689713" style="text-decoration:none;color:#F6A124;"><strong style="font-weight:400;">+44 (0)1756 799919</strong></a></td></tr><tr style="font-size:0;"><td align="left" style="padding:0;vertical-align:top;"></td><td align="left" style="padding:0 0 0 4px;vertical-align:top;"></td></tr></table></td></tr></table></td></tr><tr style="font-size:14.67px;color:#000001;font-style:normal;font-weight:400;white-space:nowrap;"><td align="left" style="vertical-align:top;font-family:Arial;"><a href="http://www.jbarisk.com/" target="_blank" id="LPlnk689713" title="Click to visit our website" style="text-decoration:none;color:#F6A124;"><strong style="font-weight:400;">www.jbarisk.com</strong></a></td></tr><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;"><tr style="font-size:0;"><td align="left" style="padding:10px 0 0;vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;"><tr style="font-size:0;"><td rowspan="3" align="left" style="padding:0 0 5px;vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="width:100%;font-size:0;"><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;line-height:normal;"><tr style="font-size:0;"><td align="left" style="padding:0 0 11px;vertical-align:top;"><img src="cid:image553391.png@4E82FDB8.64F521A8" border="0" alt="" style="font-size:0;" /></td></tr></table></td></tr><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;"><tr style="font-size:0;"><td align="left" style="padding:0 0 12px;vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="height:30px;font-size:0;"><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;line-height:normal;"><tr style="font-size:0;"><td align="left" style="padding:0 4px 5px 0;vertical-align:top;"><a href="https://www.facebook.com/TheFloodPeople" target="_blank" id="LPlnk689713" style="text-decoration:none;"><img src="cid:image038714.png@0DBCD7A4.69C0B73E" width="34" border="0" title="Facebook" alt="Facebook" style="width:34px;min-width:34px;max-width:34px;font-size:12px;" /></a></td></tr></table></td><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;line-height:normal;"><tr style="font-size:0;"><td align="left" style="padding:0 4px 5px 0;vertical-align:top;"><a href="https://www.linkedin.com/company/jba-risk-management/" target="_blank" id="LPlnk689713" style="text-decoration:none;"><img src="cid:image994012.png@FF248314.7F2A6487" width="34" border="0" title="LinkedIn" alt="LinkedIn" style="width:34px;min-width:34px;max-width:34px;font-size:12px;" /></a></td></tr></table></td><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;line-height:normal;"><tr style="font-size:0;"><td align="left" style="padding:0 4px 5px 0;vertical-align:top;"><a href="https://twitter.com/JBARisk" target="_blank" id="LPlnk689713" style="text-decoration:none;"><img src="cid:image270771.png@837A7FC3.12A37DCE" width="34" border="0" title="Twitter" alt="Twitter" style="width:34px;min-width:34px;max-width:34px;font-size:12px;" /></a></td></tr></table></td><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;line-height:normal;"><tr style="font-size:0;"><td align="left" style="padding:0 4px 5px 0;vertical-align:top;"><a href="https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw" target="_blank" id="LPlnk689713" style="text-decoration:none;"><img src="cid:image039784.png@40351117.9AEAB049" width="34" border="0" title="YouTube" alt="YouTube" style="width:34px;min-width:34px;max-width:34px;font-size:12px;" /></a></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td><td style="padding:0;"> </td></tr><tr style="font-size:0;"><td style="padding:0;"> </td></tr><tr style="font-size:0;"><td style="padding:0;"> </td></tr></table></td></tr></table></td></tr><tr style="font-size:0;"><td align="left" style="vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="font-size:0;"><tr style="font-size:0;"><td align="left" style="padding:7px 0 5px;border-top:solid 1px #4A4A49;border-right:none;border-bottom:none;border-left:none;vertical-align:top;"><table cellpadding="0" cellspacing="0" border="0" style="white-space:normal;color:#444444;font-size:14.67px;font-family:Calibri,Arial,sans-serif;font-weight:400;font-style:normal;text-align:left;line-height:12px;"><tr style="font-size:9px;"><td style="font-family:Arial;"><span style="background-color:#FFFFFF;">All JBA Risk Management's email messages contain confidential information and are intended only for the individual(s) named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.</span><br /><span style="background-color:#FFFFFF;">Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.</span> <br /><span style="background-color:#FFFFFF;">JBA Risk Management Limited is registered in England, company number 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23 3FD, England</span><span style="color:#000001;">.</span><br /></td></tr></table></td></tr></table></td></tr></table></div><div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>On Behalf Of </b>Idan Miara<br>
<b>Sent:</b> 17 November 2022 08:47<br>
<b>To:</b> Even Rouault <even.rouault@spatialys.com><br>
<b>Cc:</b> gdal-dev@lists.osgeo.org<br>
<b>Subject:</b> Re: [gdal-dev] Motion: adopt RFC 87: Signed int8 data type for raster<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:13.0pt;background:#FFEB9C"><b><span style="color:#9C6500">CAUTION:</span></b><span style="color:black"> This email originated from outside of JBA and
</span><b><span style="color:#9C6500">contains one or more links</span></b><span style="color:black">. DO NOT click links unless you recognise the sender's email address and are absolutely certain that the content is safe.<br>
See the Phishing page on IMS on SP for more information about how to spot and report suspicious messages.<o:p></o:p></span></p>
</div>
<div>
<div style="border:solid #9C0006 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:13.0pt;background:#FFC7CE"><b><span style="color:#9C0006">WARNING: This email contains one or more attachments with links within the attachment(s)</span></b><span style="color:black">. Please take extra care when handling
the attachments.<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal">Just adding my two cents... <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I highly support adding the signed int8 datatype - this would probably lead to downstream projects that support different raster datatypes to eventually fully support int8 rasters as a first class citizen. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Yes - downstream projects would need to invest some time adapting but the result would be a PROPER support for int8 rasters. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> (I would imagine that searching the code for some key search words would yield most places that need fixing), <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Nobody is forced to upgrade a major version and sometimes a downstream effort is required to support the evolution of the upstream project.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I was a maintainer of an app that used signed int8 data vastly and wanted to incorporate save/load/calc rasters using GDAL. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">After spending countless hours of trying to use the signed byte flag properly in every case I ended up changing the logic of the application to use only unsigned byte rasters since there were too many cases where the signed byte flag was
not honored or caused weird outputs (for instance, I think that QGIS didn't fully support int8 rasters properly).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Even, Thanks for investing the time to make this right,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Idan<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, 16 Nov 2022 at 21:39, Even Rouault <<a href="mailto:even.rouault@spatialys.com">even.rouault@spatialys.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think long term compatibility is a very desirable feature, and several applications just use GDAL as part of their code base without even being aware of what is happening in the GDAL development front. The same is true for several other
libraries, given the fact the use of package managers (VCPKG, conan etc) are becoming more common. For some code bases, GDAL is just a small cog, and I believe it is very easy for the developers to simply download the next version from the package manager
thinking that they are getting mostly bug fixes and not being aware of new silent incompatibilities.<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal">We try to avoid breakage or silent breakage when reasonable, but sometimes it is the best thing to do. People/software who depends on the past functionality will see the tests related to their code testing GetMetadataItem("PIXELTYPE", "IMAGE_STRUCTURE")
== "SIGNEDBYTE" break and will probably figure out they should read GDAL release notes to understand what happens. It is not the first time nor the last one we will do that.<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sorry, but I think it is funny that you mention removing GDT_Byte would have breaking implications, since I am expressing my reservations against a change that will have breaking implications. :)
<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal">My perception is that > 90% of current uses of GDT_Byte are for the unsigned usage. Of course their might be some domains where this is not the case.<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't understand what limitations you are referring to. Perhaps the support was broken to some types of raster files?<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal">This is described in the RFC: " the absence of a proper data type means that it is easy to forget to test the PIXELTYPE=SIGNEDBYTE metadata item in all places where the value of pixels is taken into account. There were special cases for
statistics computations, but most of the other code, such as the overview or warping computation code had no provision for it, and consequently mis-interpreted negative values in the range [-128,-1] as positive values in the range [128,255]." . So current
support of signed 8-byte within GDAL was broken in a number of places, and the new GDT_Int8 data type enables to fix it. One could argue that the past behavior of reporting a metadata item that altered the semantics of GDT_Byte was a silent breakage of the
API promise of GDT_Byte being unsigned.<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal">In my field, signed 8-bit images representing categorical images are very common, given the fact they can be used fully decompressed and represent a null value/nodata in very convenient way (as a negative value).<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<p>You should have a better experience with GDAL 3.7 then, pending perhaps a few changes in your code.<o:p></o:p></p>
<p class="MsoNormal">Even<br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre><a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a><o:p></o:p></pre>
<pre>My software is free, but my time generally not.<o:p></o:p></pre>
</div>
<p class="MsoNormal">_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>