[Qgis-user] How To overcome the limitation of real data width

Dan Lyke danlyke at flutterby.com
Mon Jun 22 09:06:26 PDT 2015


On Mon, Jun 22, 2015 at 8:39 AM, zehari <mriduan62 at gmail.com> wrote:
> Is there any way to overcome the limitation of real data width (max 20
> digit).
> Thanks to anybody for this solution

It's worth noting that if the underlying data type is an 64 bit IEEE
floating point "real", in base 10 you're only getting 15-17
significant digits no matter how many digits you have in the user
interface.

If you need more accuracy than that you should probably use a BCD
type, and might want to carefully examine your data path, because it's
likely that you're fooling yourself about the acuracy of your data,
and how precision is being carried through your various computational
stages.



More information about the Qgis-user mailing list