[GRASS-dev] Python assertion error in GRASS GIS script

[Nikos Alexandris]

Markus wrote:

>> >1.000000000000001 is most probably a too small difference to 1
>considering
>> >floating point precision limits.

Nikos:

>> It is. But then again, how do I decide how "loose" it can be?
>>
>> 1.01?
>> 1.001?
>> 1.0001?

Markus:

>From experience I recommend 1 + 1e-6 which is a bit lower than the minimum
>single precision floating point limit.
>
>Instead of
>
>assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1"
>
>try
>
>assert abs(fractions_sum - 1) < 1e-6, "Sum of fractions is != 1"

That's really smart.  The fractions are fragments from the same "sum",
so this assertion refers exactly to this: their "sum" being close to 1.

This is what I need!

>
>You should not go lower than 1e-15 because here you are hitting the double
>precision floating point limit.

Kudos, Nikos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20181014/3b5ad0fd/attachment.sig>