[postgis-devel] 1.3.4rc1

[Paul Ramsey]

Make it so, sounds rational to me.

P

On Sun, Oct 26, 2008 at 2:13 PM, Mark Cave-Ayland
<mark.cave-ayland at siriusit.co.uk> wrote:
> Mark Cave-Ayland wrote:
>
>> Okay, I've finally found a few moments to test on Win32. Compilation goes
>> fine, but regression blows up big style on the geojson test for which I've
>> attached the diff :(  It seems to be precision related, but there must be
>> something different in the code since the problem doesn't occur at all in
>> the rest of the tests.
>
> I've spent some time today looking at this and the problem is a symptom of
> different floating point precision(s) and the use of the %f format
> specifier.
>
> The first thing to note is that the differences in floating point
> implementations will always be present, no matter what we try and do. The
> trick, of course, is to restrict our regression tests to the lowest common
> precision denominator that will pass on all platforms.
>
> Currently the geojson code format identifiers look like this: %.*f where the
> * is used to indicate the number of decimal places. Now unfortunately this
> is incorrect because the precision of the floating point answer is related
> to the size of the mantissa (and hence the number of significant figures)
> rather than the number of decimal places.  So depending upon whether the
> calculation is done in hardware or software, the compiler options and even
> program order, the resulting output is likely to be different.
>
> The double-precision IEEE floating point format specifies a 64-bit number
> with a 53-bit mantissa, which translates into approximately 14 significant
> figures. So in order to solve this, I'd like to propose changing the meaning
> of the precision parameter from decimal places to significant figures, and
> changing the geojson output formats to %*f with a maximum precision of 14.
>
> In my testing here, this gives identical results between gcc and MingW for
> 99.9% of cases, with the small 0.1% being caused by MingW adding a trailing
> "0" to the output when rounding in some cases. I would suggest that if there
> are cases like this in the regression tests after conversion then we simply
> alter the input coordinates to prevent this from happening.
>
> So I'd like to go ahead and do this to 1.3 branch / trunk so that we can
> cleanly pass regression tests on at least gcc/msvcrt in order to get 1.3.4
> out of the door. Anyone have any major objections?
>
>
> ATB,
>
> Mark.
>
> --
> Mark Cave-Ayland
> Sirius Corporation - The Open Source Experts
> http://www.siriusit.co.uk
> T: +44 870 608 0063
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel
>