[postgis-users] Does ST_AsText truncate or round
Andy Anderson
aanderson at amherst.edu
Fri May 16 14:38:31 PDT 2008
Sorry, given the subject I should have said that they -will- round
off/half up (format ~ %field_width.precision):
%6.6f %6.5f %6.2g %6.1g
0.000014 0.00001 1.4e-05 1e-05
0.000015 0.00002 1.6e-05 2e-05
0.000016 0.00002 1.6e-05 2e-05
Note, however, that rounding-versus-truncating won't mean much at the
limits of precision (16 digits for double), because that last digit
is soemtimes inaccurate anyway:
http://en.wikipedia.org/wiki/Double_precision
I don't know if the choice of a precision of 15 is just to be careful
or is part of a spec.
-- Andy
P.S. the code for the above table:
#include <stdio.h>
main()
{
double round_down = 0.000014;
double round_up = 0.000016;
double round_half = 0.000015;
char format[] = "%6.6f %6.5f %6.2g %6.1g\n";
(void) puts(format);
(void) printf(format, round_down, round_down, round_down,
round_down);
(void) printf(format, round_half, round_half, round_up,
round_half);
(void) printf(format, round_up, round_up, round_up, round_up);
}
On May 16, 2008, at 4:56 PM, Obe, Regina wrote:
> Thanks Andy and Greg for the clarification,
>
> Sorry for being so dense.
>
> So I take it sprintf would never ever do any rounding (they merely
> print
> % significant digits). So g and f always truncate but g can flip to
> scientific if < .001
>
> http://www.cplusplus.com/reference/clibrary/cstdio/sprintf.html
>
> Thanks,
> Regina
>
>
>
>
> -----Original Message-----
> From: postgis-users-bounces at postgis.refractions.net
> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
> Andy
> Anderson
> Sent: Friday, May 16, 2008 4:36 PM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] Does ST_AsText truncate or round
>
> f & g are output formatting codes, f is the normal double-precision
> format, while g produces scientific notation (mmmE+xx) if the number
> is < 0.0001. The 8 and 15 represent the number of significant digits
> to print (for g).
>
> -- Andy
>
> On May 16, 2008, at 3:33 PM, Obe, Regina wrote:
>> First I can't remember what g means. I remember seeing fs and gs
>> somewhere before so I can only assume that f means round and g means
>> truncate?
>>
>> -----Original Message-----
>> From: postgis-users-bounces at postgis.refractions.net
>> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
>> Paul
>> Ramsey
>> Sent: Friday, May 16, 2008 10:39 AM
>> To: PostGIS Users Discussion
>> Subject: Re: [postgis-users] Does ST_AsText truncate or round
>>
>> It all falls down into this function:
>>
>> write_double(double val){
>> ensure(32);
>> if (lwgi)
>> sprintf(out_pos,"%.8g",val);
>> else
>> sprintf(out_pos,"%.15g",val);
>> to_end();
>> }
>>
>>
>>
>> On Fri, May 16, 2008 at 3:23 AM, Paragon Corporation <lr at pcorp.us>
>> wrote:
>>> Slight clarification about what I am trying to solve.
>>>
>>> After edits I am coming back with lines that used to be intersecting
>> no
>>> longer intersecting and that still have the same AsText
>> representation.
>>>
>>> My hope is to reduce the precision of my geometry in the database so
>> the
>>> edits don't break the topology of my lines by doing something like
>> this
>>>
>>>
>>> Update sometable
>>> SET the_geom = ST_SetSRID(ST_AsText(the_geom), mysrid)
>>>
>>>
>>> But I fear that if AsText is doing a rounding rather than a
>> truncation, that
>>> I run the risk of breaking things that used to intersect. I am more
>>> concerned about under intersecting than over intersecting.
>>> Although I
>>> probably haven't thought this out enough so maybe it's a non-issue.
>>>
>>> Thanks,
>>> Regina
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: postgis-users-bounces at postgis.refractions.net
>>> [mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
>> Paragon
>>> Corporation
>>> Sent: Friday, May 16, 2008 12:01 AM
>>> To: 'PostGIS Users Discussion'
>>> Subject: [postgis-users] Does ST_AsText truncate or round
>>>
>>> Hopefully this is a simple question. I know that ST_AsText
>>> returns a
>>> geometry that is not necessarily as prescise as what is actually
>> stored in
>>> the geometry field. I have an editor that relies on WKT
>> representation.
>>>
>>> Anyrate just wanted to know if the ST_AsText does a rounding of the
>> points
>>> or it does a truncation of the points. Also would be nice to
>>> know if
>> I
>>> could control the precision of this since I will need to truncate my
>> actual
>>> geometry accordingly so that both are in agreement.
>>>
>>> Thanks,
>>> Regina
>>>
>>>
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users at postgis.refractions.net
>>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>>>
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users at postgis.refractions.net
>>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>> -----------------------------------------
>> The substance of this message, including any attachments, may be
>> confidential, legally privileged and/or exempt from disclosure
>> pursuant to Massachusetts law. It is intended
>> solely for the addressee. If you received this in error, please
>> contact the sender and delete the material from any computer.
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
> -----------------------------------------
> The substance of this message, including any attachments, may be
> confidential, legally privileged and/or exempt from disclosure
> pursuant to Massachusetts law. It is intended
> solely for the addressee. If you received this in error, please
> contact the sender and delete the material from any computer.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
More information about the postgis-users
mailing list