[GRASS-dev] Python assertion error in GRASS GIS script

Nikos Alexandris nik at nikosalexandris.net
Sat Oct 13 15:20:26 PDT 2018

Markus wrote:

>> >1.000000000000001 is most probably a too small difference to 1
>> >floating point precision limits.


>> It is. But then again, how do I decide how "loose" it can be?
>> 1.01?
>> 1.001?
>> 1.0001?


>From experience I recommend 1 + 1e-6 which is a bit lower than the minimum
>single precision floating point limit.
>Instead of
>assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1"
>assert abs(fractions_sum - 1) < 1e-6, "Sum of fractions is != 1"

That's really smart.  The fractions are fragments from the same "sum",
so this assertion refers exactly to this: their "sum" being close to 1.

This is what I need!

>You should not go lower than 1e-15 because here you are hitting the double
>precision floating point limit.

Kudos, Nikos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20181014/3b5ad0fd/attachment.sig>

More information about the grass-dev mailing list