[QGIS-Developer] Wrong results converting vectors to Float32 rasters

[Pedro Venâncio]

Hi Andrea,

Thank you very much for pointing me in the right direction! Now I
understand why, for instance, 0.25 was converted to 0.25 both in Float32
and Float64. That was really strange for me.

No doubt that all come from the conversion of floating point numbers.

However, this can be really tricky and lead to wrong results in a
processing like I was talking about, with reclassifications or so.

Is there any workaround that can be used by QGIS / GDAL? Maybe an option to
the user specify the number of decimal places? If this is not possible,
maybe making Float64 as default could be safer?

Even, what do you think?

Thank you very much!

Best regards,
Pedro



Andrea Giudiceandrea <andreaerdna at libero.it> escreveu no dia quarta,
28/10/2020 à(s) 16:57:

> Hi Pedro,
> "decimal" to float (IEEE754) conversion is unlikely an exact conversion:
>
> The most accurate representation of 0.22 in
> float single precision (32 bit) is  0.2199999988079071044921875
> float double precision (64 bit) is 0.220000000000000001110223024625
>
> So probably the different behavior is due to the way GDAL or QGIS chose to
> represent and handle these values using a finite number of decimal places
> rounding/truncating at (so it seems) 16 digits:
>
> 0.2199999988079071044921875 becomes 0.2199999988079071 and
> 0.220000000000000001110223024625 becomes 0.22
>
> Regards.
>
> Andrea
>
>
>
> --
> Sent from:
> http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20201029/2adb2225/attachment.html>