<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I've come up with a quicky&dirty script
<a class="moz-txt-link-freetext" href="https://github.com/rouault/symbol_clash_detector/blob/master/symbol_clash_detector.py">https://github.com/rouault/symbol_clash_detector/blob/master/symbol_clash_detector.py</a>
that works only for Linux and attempts to find C symbol clashes in
the .so files loaded in a process (C++ is trickier because due to
template & inline code, functions from the standard library
are duplicated in C++ libraries)<br>
</p>
<p>When run on Conda gdal-3.6.0, it outputs:<br>
</p>
<p>$ PYTHONPATH=/home/even/symbol_clash_detector python -c "from
osgeo import gdal; import symbol_clash_detector"<br>
WARNING: libVersionPoint present in
/home/even/miniconda3/envs/test/lib/libplc4.so is also present in
/home/even/miniconda3/envs/test/lib/libnspr4.so<br>
WARNING: locale_charset present in
/home/even/miniconda3/envs/test/lib/libiconv.so.2.6.1 is also
present in /home/even/miniconda3/envs/test/lib/libcharset.so.1.0.0<br>
WARNING: libVersionPoint present in
/home/even/miniconda3/envs/test/lib/libplds4.so is also present in
/home/even/miniconda3/envs/test/lib/libnspr4.so<br>
</p>
<p>I'm not knowledgeable enough to assess if those 3 instances are
problematic (symbols with same name and content are OK), hopefully
not, as there are more or less system libraries.</p>
<p>When run on my dev GDAL build with drivers using proprietary
SDKs, it does find real symbol clashes that I was aware of, due to
those SDKs embedding symbol libraries (like libxml2 in FileGDB
SDK, lcms in ECW SDK, ...)</p>
<div class="moz-cite-prefix">Le 21/11/2022 à 13:37, Even Rouault a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:2ff04cc3-b8d9-d918-78c7-cd7fddc9071a@spatialys.com">
<p>Paul,</p>
<p>Looking at
<a class="moz-txt-link-freetext"
href="https://stackoverflow.com/questions/63161532/python-segmentation-fault-when-using-zipfile-while-gdal-package-loaded"
moz-do-not-send="true">https://stackoverflow.com/questions/63161532/python-segmentation-fault-when-using-zipfile-while-gdal-package-loaded</a></p>
<p>shows an interesting thing in a gdb backtrace</p>
<pre class="lang-py s-code-block"><code class="hljs language-python"><span class="hljs-comment">#0 0x00007ffff7e3aa50 in free () from /usr/lib/libc.so.6</span>
<span class="hljs-comment">#1 0x00007ffff485ae0f in inflateReset2 () from /usr/lib/libcfitsio.so.9</span>
<span class="hljs-comment">#2 0x00007ffff39ec1a4 in inflateInit2_ () from /usr/lib/libz.so.1
</span></code>
<code class="hljs language-python"><span class="hljs-comment">
</span></code></pre>
<p>One can see that libz inflateInit2_() ends up calling
libcfitsio's inflateReset2(), but libcfitsio is not a dependency
of libz ! That's the reverse. Which suggests that some builds of
libcfitsio use an internal outdated copy of libz, without symbol
renaming. Looking at conda-forge cfitsio recipe I see
<a class="moz-txt-link-freetext"
href="https://github.com/conda-forge/cfitsio-feedstock/blob/main/recipe/unvendor_zlib.patch"
moz-do-not-send="true">https://github.com/conda-forge/cfitsio-feedstock/blob/main/recipe/unvendor_zlib.patch</a>
(added per <a class="moz-txt-link-freetext"
href="https://github.com/conda-forge/cfitsio-feedstock/pull/22"
moz-do-not-send="true">https://github.com/conda-forge/cfitsio-feedstock/pull/22</a>)
which changes cfitsio to use conda-forge zlib, to avoid such
issue.<br>
</p>
<p>With latest conda-forge gdal, requests, openpyxl, I can't
trigger crashes with the reproducers of the various mentioned
tickets.</p>
<p>I'd suggest you run your code under gdb or Valgrind and look if
there are not funny symbol calls like the above.</p>
<p>Also if use pip module, avoid using their binary wheels. For
example to make sure that pyproj's embedded PROJ doesn't
conflict with the PROJ used by GDAL, to avoid similar issues
where you'd have 2 versions of the same lib in the same process.<br>
</p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 21/11/2022 à 12:40, Paul Harwood a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CAE8nN5OPu4Wri2WyeLn92BcN5dkbJfo1-LG5RAX5QrdKmtwY_g@mail.gmail.com">
<div dir="ltr">This may not be a GDAL bug and as such I have not
raised it as an issue but I wanted to put it here for anyone
else who comes across the same problem.<br>
<br>
There does appear to be some interference between, at least
the conda distribution of, GDAL and something else within
Python causing a segmentation fault.<br>
<br>
----------------------------------------
<div>SYMPTOMS<br>
<div><br>
</div>
<div>I have a Python application that uses GDAL (and PDAL
Python and MDAL Python and PyProj) wrapped in Kivy - so it
is relatively complicated.<br>
<br>
I was just clearing tech debt / updating etc. GDAL to 3.6
but also the other apps to latest versions (including
kivy).</div>
<div><br>
</div>
<div>BEFORE - if I tried to open an unrecognised file in
GDAL I got an Exception (using UseExceptions() ). This is
related to Python Duck Typing</div>
<div><br>
</div>
<div>AFTER - I get a seg fault and no debug info.<br>
<br>
This happened in exactly the same way on two machines -
one Mac and one Windows - so I don't think it is a bad
config. Attempts to create a minimum example do not work -
suggesting that it is a complicated interaction between
the various components<br>
<br>
I am using conda as the environment manager.</div>
<div><br>
</div>
<div>------------------------------------------</div>
<div><br>
</div>
<div>ULTIMATE(-ish) SOLUTION</div>
</div>
<div><br>
</div>
<div>After getting some diagnostic data and some
experimentation, it turned out that the fault is coming in
the error handling. I managed to avoid the segmentation
fault by creating and registering a dedicated error handler
for GDAL.<br>
<br>
--------------------------------------------<br>
DEEPER DISCUSSION</div>
<div><br>
</div>
<div>I found some discussion from a couple of years ago about
a problem that sounds similar (see <a
href="https://github.com/conda-forge/gdal-feedstock/issues/365"
class="moz-txt-link-freetext" moz-do-not-send="true">https://github.com/conda-forge/gdal-feedstock/issues/365</a>)
- it was not solved but was not very reproducible so was
closed.<br>
<br>
This suggests an interference between requests and gdal -
depending on the order in which they are imported. Certainly
- kivy (or more precisely kivyMD) is using requests but I
was not able to solve the problem by changing import orders
(kivy has a lot of asynch and event based things happening
and the app is very modular so it is not that simple).<br>
<br>
However - if the interference is in the error handler - then
that would explain why it was difficult to reproduce (need
to hit the right exception in the right way).<br>
<br>
Not sure if we can fix this one - but if anyone else does
come across a similar problem I would suggest to try and
create a custom error handler to see if the problem is
there.</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:gdal-dev@lists.osgeo.org" moz-do-not-send="true">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com" moz-do-not-send="true">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>