[postgis-tickets] [PostGIS] #4233: Undefined behaviour in gserialized_spgist_picksplit_nd (CREATE INDEX)

[PostGIS]

#4233: Undefined behaviour in gserialized_spgist_picksplit_nd (CREATE INDEX)
------------------------+---------------------------
 Reporter:  Algunenano  |      Owner:  Algunenano
     Type:  defect      |     Status:  assigned
 Priority:  high        |  Milestone:  PostGIS 3.0.0
Component:  postgis     |    Version:  trunk
 Keywords:              |
------------------------+---------------------------
 Detected running test `regress_spgist_index_nd` with `-fsanitize=undefined
 -fsanitize-undefined-trap-on-error`:

 {{{
 Core was generated by `postgres: raul postgis_reg [local] CREATE IN'.
 Program terminated with signal SIGILL, Illegal instruction.
 #0  0x00007ff58cf98e8f in gserialized_spgist_picksplit_nd
 (fcinfo=0x7ffc205e4f60) at gserialized_spgist_nd.c:335
 335                             if (GIDX_GET_MAX(box, i) != FLT_MAX)
 (gdb) bt
 #0  0x00007ff58cf98e8f in gserialized_spgist_picksplit_nd
 (fcinfo=0x7ffc205e4f60) at gserialized_spgist_nd.c:335
 #1  0x000056216c80ee51 in FunctionCall2Coll (flinfo=0x7ff5f56a515c,
 collation=1, arg1=33, arg2=33) at fmgr.c:1145
 #2  0x000056216c3f07c9 in doPickSplit (index=<optimized out>,
 state=<optimized out>, current=<optimized out>, parent=<optimized out>,
     newLeafTuple=<optimized out>, level=<optimized out>,
 isNulls=<optimized out>, isNew=<optimized out>) at spgdoinsert.c:829
 #3  spgdoinsert (index=<optimized out>, state=<optimized out>,
 heapPtr=0x56216dd7cefc, datum=140694363145508, isnull=false)
     at spgdoinsert.c:2077
 #4  0x000056216c3ec6c9 in spgistBuildCallback (index=0x7ff58d1e31f0,
 htup=<optimized out>, values=0x7ffc205e58e0, isnull=0x7ffc205e58c0,
     tupleIsAlive=<optimized out>, state=0x7ffc205e5a80) at spginsert.c:57
 #5  0x000056216c4374fe in IndexBuildHeapRangeScan
 (heapRelation=0x7ff58d1d0d30, indexRelation=0x7ff58d1e31f0,
 indexInfo=0x56216dd7c950,
     allow_sync=<optimized out>, anyvisible=false, start_blockno=<optimized
 out>, numblocks=4294967295,
     callback=0x56216c3ec680 <spgistBuildCallback>,
 callback_state=0x7ffc205e5a80, scan=0x56216dd7ceb0) at index.c:2944
 #6  0x000056216c436d53 in IndexBuildHeapScan (heapRelation=0x56216dcf6401,
 indexRelation=0x1, indexInfo=0x21, allow_sync=33,
     callback=0x7ff5f56a515c, callback_state=0x56216dbcd010, scan=0x0) at
 index.c:2458
 }}

 Does
 https://github.com/postgis/postgis/blob/126b97b077a1ee512a6de71931fb04de58546154/postgis/gserialized_spgist_nd.c#L335
 need to use `j` instead of `i`? If so, let's consider using a more
 descriptive name (`tupleIterator`, `dimension`...).

 Related to #4230

 Tested with PG11 and Postgis trunk.

-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/4233>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.