[gdal-dev] Python Wheels for gdal

[Even Rouault]

> There are existing tools. Multibuild uses
> https://github.com/pypa/auditwheel. It says in the project README that the
> result is "like static linking"

>From what I understood, this is far from being static linking. It just plays 
with the RPATH to make sure that the .so from the wheel .libs directory are 
loaded instead of other from the system paths. But if within the same process, 
you have symbols with same name coming from different .so (like libhdf5 coming 
from rasterio wheel and another one coming from h5py), then game over.

Playing with my idea of symbol renaming, I've discovered this jewel:
https://github.com/lief-project/LIEF
It has a super simple Python API to manipulate ELF, PE and Mach-O objects. I 
gave it a try with Linux ELF and rasterio wheels

Small python script and instructions at:
https://gist.github.com/rouault/5e92eb046b7c451b8b0071e9d765c9de

The result is a patched rasterio wheel where .so files in rasterio/.libs/ have 
all their exported symbols prefixed with _rasterio_whl_  and the rasterio/*.so 
files are modified to import those _rasterio_whl_ symbols. So we have in 
theory full symbol isolation. The resulting rasterio seems to be functional 
(at least rasterio.open() works), although I didn't have an initial situation 
where I could make it crash when loading gdal python bindings or h5py. Would 
be cool if someone that has a crashing setup could confirm that this actually 
fixes it.

But... there is a downside preventing this to be used in production. lief has 
a bug / inefficency: the symbol renaming creates .so larger than expected. In 
particular, with the netcdf lib that jumps from 1.9 MB to 52 MB !
Reported as https://github.com/lief-project/LIEF/issues/379

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com