[gdal-dev] -scale default

[Even Rouault]

Michael,

It turns out that gdal_translate was using 255.999 since the origins in 
2002 : 
https://github.com/OSGeo/gdal/commit/47d859efb25da249b5d62eb8619c36134ae76572#diff-d57b4553312c88a736821448adc155e6797653b0d902bd42f9310572aa71f0b0R215

I suspect the reasoning was that, if you rounded values down (floor) and 
used 255 as the scaleDstMax, and you remapped [0,65535] to [0,255] for 
example, then 65534. / (65535. / 255) = 254.996.. would be rounded to 
254. So you would only get 255 as the output for 65535 as the input, 
which seems "unfair". But GDAL actually rounds to closest, so if using 
dstMax = 65535, all input values in range [65535 - 128, 65535] get 
scaled to 255, which is OK to me. Of course that last "bucket" and the 
first one at 0 (input values in [0,128]) are half-width of the ones in 
the middle (input values in [129,129+256] get scaled to 1) but I don't 
think we can do much about that (discrete values vs intervals)

One downside of using dstMax=255.999 is that if your input range is [0, 
255], then -scale was not the identity (254 got transformed to 253), 
which is kind of an issue! Hence I've changed dstMax to be 255 in 
https://github.com/OSGeo/gdal/pull/8367

I've also noticed during testing a more subtle consistency issue when if 
you did for example "gdal_translate autotest/gcore/data/byte.tif out.asc 
-scale 0 1 0 1.5 -of aaigrid", you would get values outside of the [0, 
255] range because the VRT machinery was ignoring the constraints of the 
VRTRasterBand data type (Byte), and was only honouring the buffer data 
type of the RasterIO() request from the AAIGrid driver (Int32, always 
used when translating from source bands having integer data types, but 
with the reasoning that if you query a Byte band then you would get 
always value in [0,255] range)

Even

Le 08/09/2023 à 04:20, Michael Sumner a écrit :
> Hi, I'm expecting `gdal_translate -scale` to emit values in the range 
> 0,255 but it seems to be targeting 0,256.  (All works as expected when 
> using explicit src_min src_max dst_min dst_max).
>
> To see the output range on a simple example (see code at link below in 
> case email garbles):
>
> from osgeo import gdal
> ds = gdal.Translate("/vsimem/scl.tif", 
> "autotest/gcore/data/float32.tif", options = "-scale")
> ds.GetRasterBand(1).ComputeRasterMinMax()
> ## (0.0, 255.99899291992188)
>
> Is that expected, am I missing something? I've tried variations on the 
> input values and output types.
>
> https://gist.github.com/mdsumner/ee4103d8616b9aa341e82c46b44a8c8c
>
> Cheers, Mike
>
>
> -- 
> Michael Sumner
> Research Software Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsumner at gmail.com
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20230908/f00614a9/attachment-0001.htm>