[gdal-dev] Python Wheels for gdal

[Joaquim Manuel Freire Luís]

Jason,

From your description you may be interested to try the GDAL that is shipped with the GMT installer. I haven't touched on the symbol names ofc but either the GDAL dll and all of its dependencies have different names (suffixed with a "_w64"), which avoids the name clashing that you refer to.
The problem is that I don't build the Python wrapper. 

Joaquim

-----Original Message-----
From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> On Behalf Of Jason Roberts
Sent: Friday, January 31, 2020 3:28 PM
To: Even Rouault <even.rouault at spatialys.com>; gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Python Wheels for gdal

As a 15-year user of GDAL's Python bindings and a long time lurker on this list, I just wanted to thank all of you who are contemplating the Python packaging problem (and thank you for GDAL in general). I maintain a small package of Python-based utilities used by marine ecologists, mainly in an academic research setting, usually accessed from ArcGIS though GUI wrappers. Installation is one of the biggest obstacles for these users. Anything you do to further improve it (e.g. pip install gdal) would be very much appreciated.

I also wanted to beg that you include Windows in your contemplations. On Windows, there is the classic "DLL Hell" problem similar to the symbol clashing problem you've been discussing for Linux. You guys are probably more familiar with it than me at this point, but the basic issue used to be that on Windows, a given process could only have one DLL with a given file name loaded. Because GDAL dynamically links to so many libraries, it can be a problem to access it from complicated programs, such as ArcGIS (which has its own private copy of GDAL), or in concert with other libraries that might have common dependencies (e.g. on netCDF, JPEG, etc.), at least when all of the DLLs have the same names. Anyway, this stuff seems somewhat tangential to the Python packaging problem, but any improvements you can make in to this stuff is also very appreciated.

Thanks again and all the best,

Jason

> -----Original Message-----
> From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> On Behalf Of Even 
> Rouault
> Sent: Friday, January 31, 2020 9:53 AM
> To: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] Python Wheels for gdal
> 
> On vendredi 31 janvier 2020 15:14:39 CET Mateusz Loskot wrote:
> > On Fri, 31 Jan 2020 at 10:58, Christoph Paulik 
> > <cpaulik at vandersat.com>
> wrote:
> > > My initial motivation was that I would like `pip install gdal` to 
> > > just work.
> > I do share your view.
> > The pip install is always the canonical Pythonic way for me and I'd 
> > be very happy myself to be able to install GDAL that way.
> >
> > > I guess that it is unrealistic to fix any of that, so let's get 
> > > back on topic.
> > My point is that, IMO, such initiative is best to be run as a 
> > project independent from GDAL development, outside GDAL repository, 
> > for practical reasons.
> 
> Except that if we wanted "pip install gdal" to include wheels, that 
> should be done as part of the gdal Pyython package that is managed in 
> swig/python of GDAL repository. Something outside should use another 
> package name
> 
> <brainstorming>
> Thinking about Sean's very valid point about wheels and symbol clashes 
> when loading different version of the same library (or perhaps even 
> the same version twice), I'm wondering if using symbol versioning, at 
> least on Linux, wouldn't solve this. Imagine that you'd recompile GDAL 
> and all its dependencies such that all exported symbols (functions & 
> global variables), with some "gdal_wheel" marker, then those libraries 
> would be effectively private to the Python package. I know that at 
> some past point (GDAL 1.8 I think), Debian packaged GDAL with 
> versionned symbols (from a distribution point of view, versionned 
> symbols aren't necessarily that great from what I read, but for the 
> use case we discuss here, that could perhaps be an option).
> 
> Hopefully, there would be some existing tools to automate this. At 
> linking time. Or perhaps as a post processing stage, messing directly 
> with the ELF format.
> 
> I could imagine thought that symbol renaming/versionning wouldn't fly 
> very well with dlopen()/dlsym() use that libraries could make. But I 
> don't think this is extensive used in the GDAL use case. On top of my 
> head, a few cases might be:
> * typically the old way GDAL loaded PROJ4. But no longer an issue with 
> GDAL 3, or can be avoided with GDAL 2.x as well when building with 
> --with-
> [static-]proj[4])
> * the OGDI library loading its plugin. But OGDI is sufficiently a odd 
> beast that we don't need/want this in a general purpose wheel.
> </brainstorming>
> 
> Even
> 
> --
> Spatialys - Geospatial professional services
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__www.spatialys.com&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_
> gZ4a
> dc&r=cJfJ4ejc1xbg_qb47Pg1OoRq1GfFGvWbDD2PT7-
> fBKk&m=aoKWdL0rF70E3ttLt_aZ96nhL_Cdj1rsNXRsMBVM_9M&s=7T6raBfaPIXagvEPC
> CZDh
> nZMGlqHNWWjv1rdKxYLKEw&e=
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__lists.osgeo.org_mailman_listinfo_gdal-
> 2Ddev&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=cJfJ4ej
> c1xb
> g_qb47Pg1OoRq1GfFGvWbDD2PT7-
> fBKk&m=aoKWdL0rF70E3ttLt_aZ96nhL_Cdj1rsNXRsMBVM_9M&s=Hgl9GE67s6g8AcPh3
> VnFC
> QzQkhgnjRBdszdhn338ddM&e=
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev