[postgis-devel] Inconsistencies in text output

[Martin Davis]

I care!   And I agree that this is an important thing to get right.

One thing that occurred to me is that the apparent semantic of the default
output precision is not that useful.  Right now, at least according to the
manual, the default output precision is explicitly specified as 15 decimal
digits.  This is not useful, since it takes no account of the actual
significant digits in the number being output.

More useful IMO is that the default (if the precision is not provided) is
to output a number with say 15 *significant* digits (i.e. including all
digits in the number).  This means of course that the number of *decimal
places* can vary between numbers, depending on their magnitude.  But if the
user wanted a fixed number of decimal places they could have specified it
explicitly.

This means that the default does not drop (or add) any digits.  Also, I
believe this matches the semantics of most programming language floating
point output routines.

And is it the case that this is how the routines work right now?

On Fri, Apr 17, 2020 at 6:16 AM <rmrodriguez at carto.com> wrote:

> Hi all,
>
> I wanted to mention 2 things:
> First, I'm slightly worried that nobody cares about this. It makes me
> think that nobody cares about having a correct output for text output
> functions.
>
> Second, fixing the functions to output the requested (15 in most
> cases) decimal digits breaks the regression tests of any dependent
> project in many places. Not only that, but I think that having 15
> being the default doesn't make any sense as we ourselves are using 12
> digits (#define FP_TOLERANCE 1e-12) for tolerance and, for the common
> CRSs, it's enough precision to pinpoint a hair in my head. So if we
> decide to go that route it should be accompanied by a change in the
> default precision to something that makes more sense (8?).
>
> Any comments? If not I'll go trust my gut as soon as I have a sane CI
> passing (right now there appears to be multiple differences depending
> on the OS).
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20200417/ecec1fcb/attachment.html>