[gdal-dev] GDAL CLI converting to output datatype before performing calculation

[Rahkonen Jukka]

Hi,

I think I do not understand something simple, but how a byte type pixel can have one half as a value?

0.5 0.5 1

-Jukka Rahkonen-
________________________________________
Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org> käyttäjän Daniel Baston via gdal-dev <gdal-dev at lists.osgeo.org> puolesta
Lähetetty: Torstai 13. marraskuuta 2025 16.40
Vastaanottaja: Henry Walshaw <henry at floodmapp.com>
Kopio: gdal-dev at lists.osgeo.org <gdal-dev at lists.osgeo.org>
Aihe: Re: [gdal-dev] GDAL CLI converting to output datatype before performing calculation

Hi Henry,The input values should not be rounded before doing the calculation. I get a correct result with both 3.12 and 3.11.5:$ gdal raster create --size 1,1 --burn 0.1 --output-data-type Float64 depth.tif$ gdal raster calc -i "A=depth.tif" -o footprint_b.tif --calc "A > 0 ? 1 : 0" --nodata none --ot Byte$ gdal raster convert footprint_b.tif --output-format XYZ /vsistdout/ERROR 6: Read or update mode not supported on /vsistdout0.5 0.5 1DanOn Thu, Nov 13, 2025 at 12:49 AM Henry Walshaw via gdal-dev <gdal-dev at lists.osgeo.org> wrote:Hi all,In the docs for gdal raster calc (https://gdal.org/en/stable/programs/gdal_raster_calc.html#cmdoption-calc) it statesInput rasters will be converted to 64-bit floating point numbers before performing calculations.However I’ve found that when using an integer output datatype the base data is rounded to an integer before performing the calculation. e.g. converting a flood depth input.tif raster to a simple water / no water footprint:
gdal raster calc -i "A=depth.tif" -o footprint_b.tif --calc "A > 0 ? 1 : 0" --nodata none --ot Byte
The expected result for a value of (say) 0.1 is 1, but in the above calculation it comes out as 0. Obviously I can leave the output datatype alone so it stays the same as the input’s 64-bit float, but it seems unnecessary. Am I looking at a bug, or is this expected behaviour?Regards,Henry​_______________________________________________gdal-dev mailing listgdal-dev at lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev