[GRASS-dev] Python assertion error in GRASS GIS script

[Markus Metz]

On Fri, Oct 12, 2018 at 11:22 PM Nikos Alexandris <nik at nikosalexandris.net>
wrote:
>
>
> Nikos wrote:
>
[...]
>
> >1.000000000000001 is most probably a too small difference to 1
considering
> >floating point precision limits.
>
> It is. But then again, how do I decide how "loose" it can be?
>
> 1.01?
> 1.001?
> 1.0001?

>From experience I recommend 1 + 1e-6 which is a bit lower than the minimum
single precision floating point limit.

Instead of

assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1"

try

assert abs(fractions_sum - 1) < 1e-6, "Sum of fractions is != 1"

You should not go lower than 1e-15 because here you are hitting the double
precision floating point limit.

Markus M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20181013/082ca9c5/attachment.html>