[postgis-tickets] [PostGIS] #4246: Undefined behaviour in define_plane

[PostGIS]

#4246: Undefined behaviour in define_plane
------------------------+---------------------------
 Reporter:  Algunenano  |      Owner:  pramsey
     Type:  defect      |     Status:  new
 Priority:  high        |  Milestone:  PostGIS 3.0.0
Component:  postgis     |    Version:  trunk
 Keywords:              |
------------------------+---------------------------
 Reproducible with regress test:
 {{{
 SELECT '3dDistancetest6',
         ST_3DDistance(a,b) FROM (
         SELECT 'LINESTRING(1 1 1 , 2 2 2)'::geometry as a, 'POLYGON((0 0
 0, 2 2 2, 3 3 3, 0 0 0))'::geometry as b) as foo;
 }}}

 Clang sanitizer:
 {{{
 #0  define_plane (pa=0x55f1780347c8, pl=<optimized out>) at
 measures3d.c:1146
 1146            if((pa->npoints-1)==3) /*Triangle is special case*/
 (gdb) bt
 #0  define_plane (pa=0x55f1780347c8, pl=<optimized out>) at
 measures3d.c:1146
 #1  0x00007f8574ef911b in lw_dist3d_line_poly (line=<optimized out>,
 poly=<optimized out>, dl=0x7ffda6b15110) at measures3d.c:672
 #2  0x00007f8574ef8d2d in lw_dist3d_distribute_bruteforce (lwg1=0x1,
 lwg2=<optimized out>, dl=<optimized out>) at measures3d.c:549
 #3  0x00007f8574ef8454 in lw_dist3d_recursive (lwg1=0x55f178034700,
 lwg2=0x55f178034760, dl=0x7ffda6b15110) at measures3d.c:466
 #4  0x00007f8574ef86e2 in lwgeom_mindistance3d_tolerance
 (lw1=0x55f178034700, lw2=0x55f178034760, tolerance=0) at measures3d.c:376
 #5  lwgeom_mindistance3d (lw1=0x55f178034700, lw2=0x55f178034760) at
 measures3d.c:355
 #6  0x00007f8574e97283 in LWGEOM_mindistance3d (fcinfo=0x55f1780329e0) at
 lwgeom_functions_basic.c:928
 #7  0x000055f176357250 in ExecInterpExpr (state=<optimized out>,
 econtext=<optimized out>, isnull=0x7ffda6b152df) at execExprInterp.c:678
 #8  0x000055f1764235ae in ExecEvalExprSwitchContext (state=<optimized
 out>, econtext=0x11, isNull=0xc40ba7bf2bc10000) at
 ../../../../src/include/executor/executor.h:303
 #9  evaluate_expr (expr=<optimized out>, result_type=701,
 result_typmod=-1, result_collation=0) at clauses.c:4900
 }}}

 The debugger is pointing to the line 1146 but when stepping trough the
 code it's crashing around:
 {{{
 1183                    sumx+=(v.x/vl);
 }}}

 So it's probably a division by zero.

-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/4246>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.