[geos-devel] dynamic_cast may yield null pointers in AbstractSTRtree::query and AbstractSTRtree::itemsTree on OS X

[Sean Gillies]

Hi and Happy New Year, all.

I found time recently to make notes on a frustrating issue that I'm having
with GEOS in the Shapely binary wheels for OS X that I'm publishing on
PyPI. I'm not certain whether there's a GEOS bug, a dynamic loader bug, a
ctypes bug, misuse of computers on my part, or all four. I'd really
appreciate a bit of advice from anyone with more C++ and Mach-O wisdom that
would rule out a few of these possibilities.

Here's the gist of it: Shapely uses dlopen (via Python's ctypes) to load
libgeos_c.dylib at run time. The GEOS C++ library (libgeos-3.6.2.dylib, for
example) is a dependent library of libgeos_c.dylib. I have been
distributing the GEOS libs with my Shapely binaries for convenience of
users, but Shapely also works with GEOS installed to the standard places on
your system. I've never experienced or seen report of a problem with
Shapely's loading of the GEOS libraries *in isolation* that hasn't been
fixed.

My Fiona package for Python also depends on libgeos_c.dylib, but in the
more familiar way: it's a dependent library of GDAL, which is loaded when
Fiona's C extension modules are loaded in Python.

If we import fiona in a Python script and then import shapely from a binary
wheel that includes GEOS libraries, the script will abort in either
AbstractSTRtree::query or AbstractSTRtree::itemsTree because the dynamic
casts yield null pointers. As far as I can tell, loading two copies of the
C++ GEOS library is where the trouble starts. Is the trouble in the loader,
the library code, or my builds? I do not know.

There's no problem on Linux. The Linux library loader may be more foolproof
or the library code might be compiled more correctly in a way that I don't
yet see. I'm using the following flags in my builds – the dual architecture
build is the only thing that seems unusual to me.

environment =
    MACOSX_DEPLOYMENT_TARGET=10.9
configure-options =
    CFLAGS="-arch i386 -arch x86_64 -O2 -Wl,-S -Wall -Wstrict-prototypes"
    CXXFLAGS="-arch i386 -arch x86_64 -O2 -Wl,-S -Wall -Wstrict-prototypes"
    LDFLAGS="-arch i386 -arch x86_64"

The thing that makes me suspect that there is a localized bug in GEOS is
that loading the library twice doesn't lead to failures in computing areas,
lengths, predicates, or WKT serializations. Only in the AbstractSTRtree
module as far as I can tell.

I've made a ticket at https://trac.osgeo.org/geos/ticket/848 and have
attached a script that reproduces the problem as well as demonstrating that
other GEOS modules are unaffected.

It's interesting, but mostly baffling and humbling to go so deep into the
weeds of dynamic library loading. I'm over my head here and super grateful
for insights and discussion.

-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geos-devel/attachments/20180103/aeba3217/attachment.html>