[postgis-tickets] [PostGIS] #4646: Broken pointer arithmetic in gserialized_cmp leads crash/wrong results during ORDER BY

[PostGIS]

#4646: Broken pointer arithmetic in gserialized_cmp leads crash/wrong results
during ORDER BY
-----------------------+---------------------------
  Reporter:  dkvash    |      Owner:  pramsey
      Type:  defect    |     Status:  new
  Priority:  critical  |  Milestone:  PostGIS 2.5.4
 Component:  postgis   |    Version:  2.4.x
Resolution:            |   Keywords:
-----------------------+---------------------------
Changes (by Algunenano):

 * version:  2.5.x => 2.4.x


Comment:

 2.4 is also flawed:
 {{{
         if (
                 sz1 > 16 && // 16 is size of EMPTY, if it's larger - it
 has coordinates
                 sz2 > 16 &&
                 *(uint32_t*)(g1->data) == POINTTYPE &&
                 *(uint32_t*)(g2->data) == POINTTYPE &&
                 !FLAGS_GET_BBOX(g1->flags) &&
                 !FLAGS_GET_GEODETIC(g1->flags) &&
                 !FLAGS_GET_BBOX(g2->flags) &&
                 !FLAGS_GET_GEODETIC(g2->flags)
         )
 }}}

 We shouldn't be dereferencing g1->data directly before checking if there
 is a bbox (which is checked after). Let's reorder that and use
 `gserialized_get_type` which already knows about the arithmetics.

 Also, 3.0+ is not affected as that function was refactored.

-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/4646#comment:2>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.