[postgis-tickets] [PostGIS] #4600: ST_TileEnvelope default tile is imprecise

[PostGIS]

#4600: ST_TileEnvelope default tile is imprecise
------------------------+---------------------------
 Reporter:  Algunenano  |      Owner:  Algunenano
     Type:  defect      |     Status:  assigned
 Priority:  medium      |  Milestone:  PostGIS 3.0.1
Component:  postgis     |    Version:  3.0.x
 Keywords:              |
------------------------+---------------------------
 When checking the initial tile I get integer precision, which is odd to
 say the least:

 {{{
 select ST_AsText(ST_TileEnvelope(0, 0, 0));
                                                  st_astext
 ------------------------------------------------------------------------------------------------------------
  POLYGON((-20037510 -20037510,-20037510 20037510,20037510
 20037510,20037510 -20037510,-20037510 -20037510))
 (1 row)
 }}}


 Carto's function seems to do this ok:
 {{{
 Select ST_AsText(cdb_xyz_extent(0, 0, 0));
 st_astext
 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  POLYGON((-20037508.3427892 20037508.3427892,-20037508.3427892
 -20037508.3427892,20037508.3427892 -20037508.3427892,20037508.3427892
 20037508.3427892,-20037508.3427892 20037508.3427892))
 }}}

 This seems related to the limited precision of floats, which is how bbox
 are stored.


 The problem is that this imprecision is found in the base tile, so its
 error is translated to all other tiles and becomes evident in any tile
 around the borders.

 The difference is low for 0/0/0:
 {{{
 Select ST_Area(ST_Intersection(ST_TileEnvelope(0, 0, 0), cdb_xyz_extent(0,
 0, 0))) / ST_Area(cdb_xyz_extent(0, 0, 0));
  ?column?
 ----------
         1
 (1 row)
 }}}


 But it's not for certain lower tiles:
 {{{
 Select ST_Area(ST_Intersection(ST_TileEnvelope(18, 262143, 0),
 cdb_xyz_extent(262143, 0, 18))) / ST_Area(cdb_xyz_extent(262143, 0, 18));
       ?column?
 --------------------
  0.9784369447535859
 (1 row)
 }}}

 That's over 2% error.


 Although it isn't great, I suggest recalculating the bbox of the input
 geometry the precision of a 64b floating point (vs 32b).
 With that, the same tile (18/0/0) is  `0.999999996832113` equivalent,
 which is what we would expect since CARTO's function uses a slightly
 different geometry and algorithm.

-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/4600>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.