[postgis-tickets] [PostGIS] #3929: Unexpected rounding from ST_SnapToGrid

[PostGIS]

#3929: Unexpected rounding from ST_SnapToGrid
-----------------------+---------------------------
  Reporter:  michaudm  |      Owner:  pramsey
      Type:  defect    |     Status:  closed
  Priority:  medium    |  Milestone:  PostGIS 2.4.2
 Component:  postgis   |    Version:  2.4.x
Resolution:  wontfix   |   Keywords:
-----------------------+---------------------------
Changes (by pramsey):

 * status:  new => closed
 * resolution:   => wontfix


Comment:

 Unfortunately this seems a little too deep to do anything about. We're
 seeing here the fact that decimal floats don't always have exact analogues
 in IEEE double precision (binary) storage. Here's a simple C program that
 demonstrates exactly the effect you're seeing.
 {{{
 static void test_rint()
 {
     char str[] = "311.4";
     double d1, d2, d3;
     double g = 0.1;

     sscanf(str, "%lf", &d1);
     printf("d1: %g\n", d1);

     d2 = rint(d1 / g) * g;
     printf("d2: %g\n", d2);
     printf("d2-d1: %g\n", d2-d1);

     d3 = rint(d2 / g) * g;
     printf("d3-d2: %g\n", d3-d2);
 }
 }}}
 The results are
 {{{
 d1: 311.4
 d2: 311.4
 d2-d1: 5.68434e-14
 d3-d2: 0
 }}}
 Basically the way scanf is forming 311.4 is slightly different from the
 version that the math operations of dividing and then remultiplying the
 number come up with. In order to "snap to a grid" all we're doing is
 scaling up numbers until the grid is represented by integral values, then
 rounding, so exactly the operation you're seeing in this example.

--
Ticket URL: <https://trac.osgeo.org/postgis/ticket/3929#comment:1>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.