[gdal-dev] -scale default

[Michael Sumner]

nice, thanks so much for the extended description.

Cheers, Mike

On Fri, 8 Sept 2023, 21:34 Even Rouault, <even.rouault at spatialys.com> wrote:

> Michael,
>
> It turns out that gdal_translate was using 255.999 since the origins in
> 2002 :
> https://github.com/OSGeo/gdal/commit/47d859efb25da249b5d62eb8619c36134ae76572#diff-d57b4553312c88a736821448adc155e6797653b0d902bd42f9310572aa71f0b0R215
>
> I suspect the reasoning was that, if you rounded values down (floor) and
> used 255 as the scaleDstMax, and you remapped [0,65535] to [0,255] for
> example, then 65534. / (65535. / 255) = 254.996.. would be rounded to 254.
> So you would only get 255 as the output for 65535 as the input, which seems
> "unfair". But GDAL actually rounds to closest, so if using dstMax = 65535,
> all input values in range [65535 - 128, 65535] get scaled to 255, which is
> OK to me. Of course that last "bucket" and the first one at 0 (input values
> in [0,128]) are half-width of the ones in the middle (input values in
> [129,129+256] get scaled to 1) but I don't think we can do much about that
> (discrete values vs intervals)
>
> One downside of using dstMax=255.999 is that if your input range is [0,
> 255], then -scale was not the identity (254 got transformed to 253), which
> is kind of an issue! Hence I've changed dstMax to be 255 in
> https://github.com/OSGeo/gdal/pull/8367
>
> I've also noticed during testing a more subtle consistency issue when if
> you did for example "gdal_translate autotest/gcore/data/byte.tif out.asc
> -scale 0 1 0 1.5 -of aaigrid", you would get values outside of the [0, 255]
> range because the VRT machinery was ignoring the constraints of the
> VRTRasterBand data type (Byte), and was only honouring the buffer data type
> of the RasterIO() request from the AAIGrid driver (Int32, always used when
> translating from source bands having integer data types, but with the
> reasoning that if you query a Byte band then you would get always value in
> [0,255] range)
>
> Even
> Le 08/09/2023 à 04:20, Michael Sumner a écrit :
>
> Hi, I'm expecting `gdal_translate -scale` to emit values in the range
> 0,255 but it seems to be targeting 0,256.  (All works as expected when
> using explicit src_min src_max dst_min dst_max).
>
> To see the output range on a simple example (see code at link below in
> case email garbles):
>
> from osgeo import gdal
> ds = gdal.Translate("/vsimem/scl.tif", "autotest/gcore/data/float32.tif",
> options = "-scale")
> ds.GetRasterBand(1).ComputeRasterMinMax()
> ## (0.0, 255.99899291992188)
>
> Is that expected, am I missing something? I've tried variations on the
> input values and output types.
>
> https://gist.github.com/mdsumner/ee4103d8616b9aa341e82c46b44a8c8c
>
> Cheers, Mike
>
>
> --
> Michael Sumner
> Research Software Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsumner at gmail.com
>
> _______________________________________________
> gdal-dev mailing listgdal-dev at lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20230909/bd7d30a2/attachment.htm>