<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Apr 18, 2020 at 12:22 PM <<a href="mailto:rmrodriguez@carto.com">rmrodriguez@carto.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, Apr 18, 2020 at 6:18 PM Martin Davis <<a href="mailto:mtnclimb@gmail.com" target="_blank">mtnclimb@gmail.com</a>> wrote:<br>
<br>
> When you say "limit" do you mean truncate or round? I don't think truncation is a good approach (based on what every other precision-limiting system I have seen does).<br>
> If you mean round then I *think* your suggestion is essentially the same as mine.<br>
<br>
Ideally, rounding.<br></blockquote><div><br></div><div>Ah, that's good. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
> I would say precision 0..8 is most useful; beyond that not as much.<br>
><br>
> But surely there has to be some transition point at which rounding stops and "full" precision is used?<br>
<br>
Yes, but rounding might stop at different points<br></blockquote><div><br></div><div>My idea is that 15 decimal digits is about the limit of DP precision. Below that rounding would be noticeable, beyond it it would not be. It could be 14 or 13 instead, I think. The key point is that the current default will produce full precision.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
>> - There is no artificial distinction between < 15 and > 15, which<br>
>> eliminates inconsistencies. Note that in your proposal you would get<br>
>> odd things like this:<br>
>> - 0.300000011920928955078125 with precision 10 -> 0.3000000119<br>
>> - 0.300000011920928955078125 with precision 20 -> 0.3 (since 0.3 has<br>
>> the same double representation)<br>
><br>
><br>
> I don't see how 0.300000011920928955078125 has the same DP representation as 0.3. DP has about 16 digits of precision, so surely the first number would be represented as (close to) 0.300000011920929 ?<br>
<br>
Sorry, I took a 32bit float example (from tryu) so it doesn't work<br>
with 64bits. Let's see a 64 bit double example:<br>
```<br>
<br>
postgis=# Select ST_AsText(ST_MakePoint(0,<br>
0.299999999999999988897769753748), 30);<br>
st_astext<br>
<br>
POINT(0 0.299999999999999988897769753748)<br>
(1 row)<br>
<br>
postgis=# Select ST_AsText(ST_MakePoint(0, 0.3), 30);<br>
st_astext<br>
<br>
POINT(0 0.299999999999999988897769753748)<br>
(1 row)<br>
```<br></blockquote><div><br></div><div>Okay, that makes more sense. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
In both those examples, for anything except precision 0, IMO the<br>
output should be POINT(0 0.3). Why? Because both<br>
0.29999999999999998889776975374 and 0.3 have the same double<br>
representation (in hex 0x3fd3333333333333) so when you import 0.3 you<br>
are getting the same value, thus it makes sense to output the shortest<br>
representation in all cases.<br></blockquote><div><br></div><div>Yes, and I think that will happen with the proposed scheme? </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
>From what I understood of your proposal, there would be a precision<br>
(17+) where you would get more digits beyond 0.3. In my proposal you<br>
would never get any more digits.<br></blockquote><div><br></div><div>No, at 15 and above requested digits you would get "full precision" and no more.</div><div><br></div><div>The intent is NOT to allow the user to force a meaningless precision to be output. There are two goals:</div><div><br></div><div>a) allow output of numbers with a textual precision that matches the known DP precision. For instance, if ST_SnapToGrid(geom, .01) is used then output should only show 2 decimal digits (even if roundoff actually causes a bit of drift in the internal representation.</div><div>b) allow output of full (round-trippable) precision, with that as the default</div><div><br></div><div>Part of the motivation for (a) is that I hoping that "soon" PostGIS will gain the ability to execute some operations to a specified precision (in particular overlay, and also a better geometry rounding capability) . This will make moving data in out of PostGIS more robust, since it will prevent creating geometry in PostGIS that carries excess precision and then becomes invalid when exported at a lower precision. </div><div> </div><div>Ideally we should capture all this as an RFC, and then (if implemented) document it in the manual (with examples).</div></div></div>