[geos-devel] [GEOS] #850: Segfault in geos::algorithm::ConvexHull::preSort with extreme outlier

[GEOS]

#850: Segfault in geos::algorithm::ConvexHull::preSort with extreme outlier
------------------------+--------------------------
 Reporter:  sgillies    |      Owner:  geos-devel@…
     Type:  defect      |     Status:  new
 Priority:  major       |  Milestone:  3.6.3
Component:  Default     |    Version:  3.5.1
 Severity:  Unassigned  |   Keywords:
------------------------+--------------------------
 First reported in https://github.com/Toblerity/Shapely/issues/555, I've
 been able to reproduce it with GEOS 3.5.1 from Debian Stretch.

 The user's input has an extreme outlier, the first point:

 {{{
 [(280756800.63603467, 7571780.5096410504),
  (-0.00043553364940478493, -1.1745985126662545e-05),
  (-0.0040809829767810965, -0.00011006076189068978),
  (-0.0041201583341660313, -0.00011111728913462023),
  (-0.006976907320408115, -0.00018816146492247227),
  (-0.0069397726510486172, -0.00018715997340633273),
  (-0.0074676533800189931, -0.000201396483469504),
  (-0.13462489887442128, -0.0036307230426676734),
  (-0.010721780626750072, -0.00028915762480866283),
  (-0.010775949783764172, -0.00029061852246303201),
  (-0.011934357539045426, -0.0003218598289746266),
  (-0.019390152385490519, -0.00052293649740946452),
  (-0.016403812662021146, -0.00044239736574681491),
  (-0.013937679796751739, -0.00037588778618408299),
  (-0.0073628397580766435, -0.00019856974598662623),
  (-0.0013082267409651623, -3.5281801617658642e-05),
  (-0.0019059940589774278, -5.14030956166791e-05)]
 }}}

 {{{
 (gdb) run test.py
 Starting program: /tmp/venv/bin/python3 test.py
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

 Program received signal SIGSEGV, Segmentation fault.
 0x00007ffff5240204 in ?? () from /usr/lib/x86_64-linux-
 gnu/libgeos-3.5.1.so
 (gdb) up
 #1  0x00007ffff5240a22 in
 geos::algorithm::ConvexHull::preSort(std::vector<geos::geom::Coordinate
 const*, std::allocator<geos::geom::Coordinate const*> >&) ()
    from /usr/lib/x86_64-linux-gnu/libgeos-3.5.1.so
 }}}

 If the outlier is removed from the input, the convex hull computation
 succeeds.

--
Ticket URL: <https://trac.osgeo.org/geos/ticket/850>
GEOS <http://trac.osgeo.org/geos>
GEOS (Geometry Engine - Open Source) is a C++ port of the Java Topology Suite (JTS).