[gdal-dev] Test problem After Merge

[Even Rouault]

Andrew,

interestingly adding the debug traces I suggested to you to the GDAL CI 
MacOS 14 build witch no longer worked turns out to reveal the exact same 
issue as yours! (funny that I didn't even consider doing that myself ! 
interesting case where helping others actually is helping oneself :-))  
_gdal.cpython-312-darwin.so was also linking against libgdal.so.34 (GDAL 
3.8.5) instead of libgdal.so.35 (libgdal 3.10dev). The reason is that 
this CI configuration uses conda-forge to install dependencies, and it 
accidentally installed libgdal itself, instead of just its dependencies 
(it add a "conda install --only-deps libgdal libgdal-arrow-parquet" 
line, but I missed that libgdal *is* a dependency of 
libgdal-arrow-parquet... )

Looking at the compilation log, I see:

arm64-apple-darwin20.0.0-clang++ -bundle -undefined dynamic_lookup 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib 
-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs 
-Wl,-rpath,/Users/runner/miniconda3/envs/test/lib 
-L/Users/runner/miniconda3/envs/test/lib -ftree-vectorize -fPIC 
-fstack-protector-strong -O2 -pipe -isystem 
/Users/runner/miniconda3/envs/test/include -D_FORTIFY_SOURCE=2 -isystem 
/Users/runner/miniconda3/envs/test/include 
build/temp.macosx-11.0-arm64-cpython-312/extensions/gdal_wrap.o 
-L/Users/runner/work/gdal/gdal/build -lgdal -o 
build/lib.macosx-11.0-arm64-cpython-312/osgeo/_gdal.cpython-312-darwin.so

So something in the Python extension building logic (conda 
customizations?) adds this -L/Users/runner/miniconda3/envs/test/lib 
where the "libgdal" from conda-forge is found, which apparently takes 
precedence over the "-L/Users/runner/work/gdal/gdal/build -lgdal" 
towards the end of the line, which is the bit that links against the 
libgdal of the build tree.

For my CI case removing libgdal from the installed conda-forge packages 
fixed the issue, but it would be nice if we could avoid/detect the issue 
in the first place. If so, that would likely be some magic to add into 
swig/python/setup.py.in. It might be tricky to do so though. Like 
detecting -Larg in the linking options and looking for a 
libgdal.so/dylib in them... ? Yuck. Or perhaps do some postprocessing 
check that the resulting extension .so links against the expected libgdal?

Even

Le 30/04/2024 à 17:41, Andrew Bell a écrit :
> On Tue, Apr 30, 2024 at 9:18 AM Even Rouault 
> <even.rouault at spatialys.com> wrote:
>
> Even,
>
>     Also try in a Python interpreter "from osgeo import gdal" to see
>     if the exception is more verbose. If your GDAL lib links against
>     libraries that are not in the default library search path, GDAL
>     command line utilities might still be able to find them, but
>     loading libgdal through the Python bindings might fail (at least
>     that happens to me on Linux, cf
>     https://github.com/OSGeo/gdal/pull/9783)
>
>     Also try otool -L on libgdal.dylib and on
>     _gdal.cpython-312-darwin.so <http://gdal.cpython-312-darwin.so>
>
> This was indeed the issue. I didn't think there was a version of GDAL 
> in my environment, but it must have gotten in by error at some point. 
> When the SWIG library _gdal...so was created, a dependency was created 
> for the system-located library (version 34) rather than the one in the 
> build (version 35). I'd have to do more research to figure out how to 
> rectify this, if desired.
>
> Thanks for your help. The python failure message wasn't too helpful on 
> its own.
>
> -- 
> Andrew Bell
> andrew.bell.ia at gmail.com

-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240430/7fa347ab/attachment.htm>