[geos-devel] [GEOS] #865: GEOSClipByRect failure

[GEOS]

#865: GEOSClipByRect failure
------------------------+---------------------------
 Reporter:  komzpa      |       Owner:  geos-devel@…
     Type:  defect      |      Status:  closed
 Priority:  major       |   Milestone:  3.6.3
Component:  Default     |     Version:  3.6.2
 Severity:  Unassigned  |  Resolution:  worksforme
 Keywords:              |
------------------------+---------------------------

Comment (by Algunenano):

 Trying to analyze the endless loop. Running the query from Postgis
 reaches:
 {{{
 #1  0x00007ff35817a014 in geos::operation::intersection::distance
 (rect=..., ring=std::vector of length 2, capacity 2 = {...})
     at RectangleIntersectionBuilder.cpp:247
 }}}

 With the following values:
 {{{
 (gdb) p ring
 $21 = std::vector of length 2, capacity 2 = {{static _nullCoord = {static
 _nullCoord = <same as static member of an already seen type>,
       x = nan(0x8000000000000), y = nan(0x8000000000000), z =
 nan(0x8000000000000)}, x = 3.0481343214686657e-14, y = -1,
     z = nan(0x8000000000000)}, {static _nullCoord = {static _nullCoord =
 <same as static member of an already seen type>,
       x = nan(0x8000000000000), y = nan(0x8000000000000), z =
 nan(0x8000000000000)}, x = 3.0481259877257472e-14, y = -1,
     z = nan(0x8000000000000)}}
 (gdb) p ring.size()
 $22 = 2
 (gdb) p ring[1].x
 $24 = 3.0481259877257472e-14
 (gdb) p ring[1].y
 $25 = -1
 (gdb) p ring[0].x
 $26 = 3.0481343214686657e-14
 (gdb) p ring[0].y
 $27 = -1
 }}}

 And the rectangle:
 {{{
 (gdb) p rect
 $28 = (const geos::operation::intersection::Rectangle &) @0x7ffe21add4f8:
 {xMin = 3.0481343214686657e-14, yMin = -20000000,
   xMax = 20000000, yMax = -1}
 }}}


 This then calls:
 {{{
 geos::operation::intersection::distance (rect=...,
 x1=3.0481259877257472e-14, y1=-20000000, x2=3.0481343214686657e-14, y2=-1)
     at RectangleIntersectionBuilder.cpp:187
 }}}


 The setup:
 {{{
 (gdb) p rect.position(3.0481259877257472e-14, -20000000)
 $30 = geos::operation::intersection::Rectangle::Outside
 (gdb) p rect.position(3.0481343214686657e-14, -1)
 $31 = geos::operation::intersection::Rectangle::TopLeft

 }}}

 Which means that `pos & endpos` will always be zero, thus never reaching
 the `break` statement and looping until the end of time.


 I don't know GEOS codebase to add a test based on this, so I decided to
 follow the same idea as Kopmza and create a test directly with
 `GEOSClipByRect`. The issue is that using geometry::toText() is removing
 some significant bytes and the test doesn't fail.
 To avoid this I patched the function to always print 30 digits and took
 the output: Its here
 (https://gist.github.com/Algunenano/045dfc55f2b530df0fed12beb2d4db23) and
 its pretty long.

 With that test my installation enters the infinite loop. Setup:
 {{{
 $ uname -a
 Linux Mordor 4.17.8-1-ARCH #1 SMP PREEMPT Wed Jul 18 09:56:24 UTC 2018
 x86_64 GNU/Linux
 $ clang -v
 clang version 6.0.1 (tags/RELEASE_601/final)
 Target: x86_64-pc-linux-gnu
 Thread model: posix
 InstalledDir: /usr/bin
 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-linux-
 gnu/8.1.1
 Found candidate GCC installation: /usr/bin/../lib64/gcc/x86_64-pc-linux-
 gnu/8.1.1
 Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.1
 Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.1.1
 Selected GCC installation: /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.1.1
 Candidate multilib: .;@m64
 Candidate multilib: 32;@m32
 Selected multilib: .;@m64
 $ ./configure --prefix=/usr CC=clang CXX=clang++
 CPPFLAGS="-D_FORTIFY_SOURCE=2" CFLAGS="-pipe -fno-plt -Wall -Wextra -g3
 -gdwarf-4 -fno-omit-frame-pointer" CXXFLAGS="-pipe -fno-plt -Wall -Wextra
 -g3 -gdwarf-4 -fno-omit-frame-pointer" LDFLAGS="-Wl,-O1,--sort-common
 ,--as-needed,-z,relro,-z,now"
 }}}

 I've also tested with GCC 8.1.1 (same flags, only changing clang -> gcc,
 clang++ -> g++) and it also happens. Before I was trying with optimization
 flags (`-march=native -mtune=native -O3 -Wall -Wextra -g3 -gdwarf-4 -fno-
 omit-frame-pointer`) and it was also happening, but the inlining made it
 pretty hard to debug the issue.

 Finally, PR with the test: https://github.com/libgeos/geos/pull/109

-- 
Ticket URL: <https://trac.osgeo.org/geos/ticket/865#comment:2>
GEOS <http://trac.osgeo.org/geos>
GEOS (Geometry Engine - Open Source) is a C++ port of the Java Topology Suite (JTS).