[postgis-devel] Inconsistencies in text output

[rmrodriguez at carto.com]

Hi all,

I wanted to mention 2 things:
First, I'm slightly worried that nobody cares about this. It makes me
think that nobody cares about having a correct output for text output
functions.

Second, fixing the functions to output the requested (15 in most
cases) decimal digits breaks the regression tests of any dependent
project in many places. Not only that, but I think that having 15
being the default doesn't make any sense as we ourselves are using 12
digits (#define FP_TOLERANCE 1e-12) for tolerance and, for the common
CRSs, it's enough precision to pinpoint a hair in my head. So if we
decide to go that route it should be accompanied by a change in the
default precision to something that makes more sense (8?).

Any comments? If not I'll go trust my gut as soon as I have a sane CI
passing (right now there appears to be multiple differences depending
on the OS).

On Mon, Apr 13, 2020 at 12:05 PM <rmrodriguez at carto.com> wrote:
>
> > - The current documentation and design intent is to provide the ability to specify Max DECIMAL Digits, so changing this would be a breaking change
>
> It would be a breaking documentation change, but normally not a
> behaviour breaking change. There are 100x more changes in behaviour
> when complying with the documentation.
>
> > - It is not useful to specify the Max TOTAL Digits, since without knowing the magnitude of the numbers this leads to unknown and possibly varying precision of output numbers
>
> Well, we already have varying precision due to the use of floating
> point. The nth decimal digit has more precision the closer to 0 you
> are, so is and the higher the integer part, the less meaningful
> decimal digits you have so it helps to remove digits that don't carry
> information.
>
> > +1 to correctly respecting the limit on Max DECIMAL Digits
> > -2 to changing the semantics to Max TOTAL Digits
>
> I'm fine going either way, I just want to make sure everyone
> understands the decision before we commit to one. If I was starting
> from scratch, I'd choose to return the shortest representation
> maintaining round trip safety but that would be an even greater
> breaking change.
>
> I'm going to wait a few extra days to see if there are any more
> comments before committing to a final solution.
>
> --
> Raúl Marín Rodríguez
> carto.com



-- 
Raúl Marín Rodríguez
carto.com