[Qgis-user] double, real, numeric - does it matter at all?

Tobias Wendorff tobias.wendorff at tu-dortmund.de
Wed Feb 20 04:31:05 PST 2019


Hi there,

I'm curious, which decimal number format is a good one to store
and calculate with. When using ESRI Shapefiles, dbf's "double" had
the highest precision. In PostgreSQL world, "numeric" is incredible.
But QGIS confuses me a bit.

It doesn't use the normal precision/scale format, but length/precision. I
know of IEEE 754 and the limitation of double precision. But IEEE
754-2008 also has quadruple precision. Geopackage can handle up to
64-bit, that means double precision. JSON, PostgreSQL and others
could handle unlimited precision.

Seems like QGIS is limited to double, whatever you choose as format
and data-provider. You're limited to 18 digits after the point, of
course, only 15 of them might be correct.

Why can you create numeric fields with 32 length and 30 precision,
QGIS can't handle them? Wouldn't it be more secure to limit it to
QGIS's real limitations to make the user not think, he gets a
higher one? Sure, I've been storing high precision numbers as
strings for ages (when the full precision needs to survive storing).

Check my report [1] for some example decimals. The bug reports
in there got fixed by Nyall incredible fast - thanks for that :D

Best regards,
Tobias

Refs:
[1] https://issues.qgis.org/issues/21316



More information about the Qgis-user mailing list