<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<br>
<blockquote type="cite"
cite="mid:55CE6F36-F0CA-483F-83BF-2B67E88A3FB7@hobu.co">
<pre class="moz-quote-pre" wrap="">
Now that the project has made it to CMake, I wonder if there is enthusiasm to break apart GDAL's build system to take advantage of its plugin capability in a default fashion to cut down on the massiveness of our default stuff-you-actually-want build situation. People rightly complain that libgdal.so links too much stuff, and in most cases, people don't need it all.
What if we were to default to having drivers be free-standing CMake projects that installed versioned plugin shared objects into known locations that could be activated at library load (or use) time? GDAL has this dynamic plugin capability right now, but it is not widely used, and build configuration is not so convenient for its usage. Plugins would allow each of these drivers to link to their esoteric dependencies without polluting the common library, and the situation we have now, where multiple drivers have similar sets of dependencies, could be handled in some kind of hierarchical fashion based on common dependencies.
For deployments like rasterio's wheels, external GDAL plugins could be loaded at runtime by the rasterio libgdal.so, and rasterio wouldn't need to be responsible for building a libgdal.so that provided kitchen sink capabilities. People could show up with their own driver plugins that included dependencies as needed.
GDAL's reputation of being hard to build and deploy is well earned and deserved. IMO, the CMake configuration has increased its build speed, system flexibility, and commonality on supported platforms, but it hasn't addressed a core challenge which is GDAL links to the world one DLL at a time. I don't know if the project wants to prioritize this topic relative to other issues, but we have some tools and resources available to us to improve the situation if so.</pre>
</blockquote>
<p>Some thoughts/questions regarding this:</p>
<p>- you definitely don't want to build all drivers (that can be
built as plugins) as plugins. This works (this is actually my dev
setup, to ensure that the core exports all the symbols needed for
drivers built as plugins), but I've found there's a measurable
penalty in doing so (300 ms to load the 127 plugins on my system,
ie each GDAL command line invokation will at least take 300 ms,
which might be a significant penalty in some workflows) . So <span
class="sig-name descname"><span class="pre">GDAL_ENABLE_PLUGINS_NO_DEPS=YES
(cf
<a class="moz-txt-link-freetext" href="https://gdal.org/development/building_from_source.html#cmdoption-arg-GDAL_ENABLE_PLUGINS_NO_DEPS-BOOL">https://gdal.org/development/building_from_source.html#cmdoption-arg-GDAL_ENABLE_PLUGINS_NO_DEPS-BOOL</a>)
is typically discouraged. But 10 or so plugins should be
fine.</span></span></p>
<p><span class="sig-name descname"><span class="pre">- you can
"emulate" building a single driver as plugin, by re-running a
GDAL build with just the dependencies of that driver, </span></span><span
class="n">and setting -DGDAL_ENABLE_DRIVER_xxxx_PLUGIN</span>=ON.
The only build artifact of interest in that use case is the
[gdal|ogr]_XXX.so/dll corresponding to that driver. The main
inconvenience I can see with this current approach is that you
have to pay the price for the core lib to be rebuilt, whereas with
the free-standing CMake project, you'd point to the installed GDAL
headers & lib</p>
<p>- by free-standing CMake project, what do you mean ? Couldn't
that just the existing frmts/XXXX/CMakeLists.txt file that would
detect it is called as the top level CMakeLists.txt (probably by
checking ${CMAKE_CURRENT_SOURCE_DIR} == ${PROJECT_SOURCE_DIR}) and
then change its logic to find libgdal and its dependencies ? And
users would "cmake -S frmts/XXXX -B plugin_XXXX" to configure just
for that plugin.</p>
<p>- I don't understand the "where multiple drivers have similar
sets of dependencies, could be handled in some kind of
hierarchical fashion based on common dependencies". why is a
special case needed if several drivers share the same dependencies
? Or maybe you were thinking to drivers depending on other drivers
? That situation can indeed happen currently, but when it happens,
such drivers aren't allowed to be built as plugins. They have to
be built with libgdal. Both because it makes the build system
simpler, and also to avoid deployment complication where you'd
have to know that gdal_XXXX.so links against gdal_YYYY.so.<br>
</p>
<p><span class="sig-name descname"><span class="pre"></span></span></p>
<p><span class="sig-name descname"><span class="pre">- we would need
to keep the current monolithic CMake project approach working,
at the very least for people doing static builds, but even for
people doing dynamic builds as it might be still be more
practical if you can install all the dependencies at once.
Having the standalone capability in addition would be an extra
complexity, so we need to be sure it addresses real use cases,
and perhaps restrict it to a few select drivers. Obvious
candidates for standalone CMake projects would likely be for
the few drivers that have proprietary dependencies (ECW,
JP2KAK, MrSID, Oracle, ...) and aren't shipped by FOSS binary
distributions.<br>
</span></span></p>
<p><span class="sig-name descname"><span class="pre">- offering the
possibility of building just a driver as a plugin in a unit
way would require people to build against the same GDAL
headers as libgdal. So people would have to download the same
version of the GDAL sources as the one used by the libgdal
provided by rasterio + having the required SDK + installing
the libs of the SDK in a path where Python/rasterio can find
them. Is it in the expected cultural background of rasterio
users to do that on their own ? <br>
</span></span></p>
<p><span class="sig-name descname"><span class="pre">Even<br>
</span></span></p>
--
<pre class="moz-signature" cols="72"><a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>