[gdal-dev] About CMake build again

[Kurt Schwehr]

+1 to what Howard said.

Two thoughts... (and I'm not saying CMake is any worse than autoconf)

Alternative build systems:

I co-maintain (mostly just me) a bazel build of GDAL (including autotest2,
expat, geos, hdf4, hdf5, hdfeos, kakadu, libgeotiff, libgif, libpng,
libtiff, netcdf, jpeg, proj, sqlite, zlib, and some other stuff).  It's
exhausting, but it's critical for my business needs. I'm happy to share
parts of it if anyone wants.  bazel has lots of great features.  It scales
well with parallelization (> 5K build targets with 10's of thousands of
source files), does ci well, lets me query the tree (love the DAG
<https://en.wikipedia.org/wiki/Directed_acyclic_graph>), and I get to "live
at head".  I've added a few patches to GDAL to make it easier for my
parallel universe, but the burden is on me and the other folks in that same
space, not the core of GDAL.  bazel isn't about the flexibility that many
packagers of GDAL want and sometimes require.

Make drivers plugins:

You talk about the issues of homebrew and crashing with changes due to
other libs:  That's a design tradeoff that homebrew took and why I'm still
living in fink for that kind of world (bias: I have fink commit access).  A
massively plugin world is going to make things even more messy for folks
would do actually make sure that requirements are met.  In fink, we try
(yeah, it's hard) to force / check ABI compatibility so that you get those
types of issues less often.  But that has little to do with the build
system.  I worry that more plugs means that end users will have more
targets for disaster and packagers have more nodes to worry about.  I
suggest watching CppCon 2017: Titus Winters “C++ as a "Live at Head"
Language” <https://www.youtube.com/watch?v=tISy7EJQPzI> and thinking hard
about the ABI issues he discusses.

I did separate out most of port/ into a separate library in my bazel world
to speed up and reduce the footprint of some autotest2 tests
<https://github.com/schwehr/gdal-autotest2/tree/master/cpp/port>.  It would
be nice to be able to layer more.  e.g.  I'd love to have a
GDALCoreRegister and GDALAllRegister, where core was the minimum set of
drivers and GDALAllRegister lived in a separate lib with the rest of the
drivers.  But plugins?  Can homebrew have a binary cache like
fink/macports?  Or does the design preclude that?


On Wed, Nov 29, 2017 at 9:47 AM, Howard Butler <howard at hobu.co> wrote:

> Larry,
>
> I agree that CMake brings some nice and useful features to the table. We
> use it for PDAL, but we started out with CMake so there was no migration
> that needed to be made.
>
> CMake for GDAL is a project sustainment issue much more than a
> technology switch and a patch. A person or a group of persons needs to
> step forward to provide the patches that implement CMake support, and
> then that group needs to stick around and keep it up in parallel to the
> existing build systems for a while (three major releases?) as all of the
> issues with it are identified and resolved. It would be a slog. That
> group has to want it bad to bring it over the finish line.
>
> That effort would have to compete against doing nothing, which every
> packaging system that packages GDAL has already plumbed and stitched
> around to make work. It is a tough sale. Does CMake have so many
> compelling features, protections, and conveniences to make it worth it?
> The project needs to answer that question for itself by trying it in
> parallel with what it is already doing.
>
> Finally, CMake is aggressive about deprecating features, its language is
> often just as arbitrary as the much hated autotools/automake stack, and
> it is now on its third attempt to regularize dependency detection for
> projects.  The deprecations mean significant configuration churn, and
> the dependency detection one is probably the most painful -- especially
> for projects that have been around a long time. These challenges mean
> trading one set of known annoyances with the current build stack for an
> unknown-to-the-project set. The problem is that a try-it-and-see cost to
> get started with such a substantial project as GDAL is very high.
>
> Howard
>
>
> On 11/28/17 11:03 PM, Larry Shaffer wrote:
> > Hi All,
> >
> > Apologies for top-posting, albeit the thread has stopped for ~month.
> >
> > While I have not contributed that much to the GDAL project over the
> years,
> > I have build it a LOT (mostly on Win and macOS [0]). Switching to CMake
> > would be a good future-proof, cross-platform move, even if it is overly
> > verbose at times.
> >
> > My concern, like others noted here, is that it will not necessarily be
> > greatly advantageous over the current build scripts. Unless... such an
> > endeavor leveraged some of CMake's more useful features, namely ease of
> > defining inter-library pre- or post-build-time dependencies, and external
> > projects.
> >
> > *Make drivers plugins*
> > One of my biggest frustrations with building (and rebuilding) libgdal
> comes
> > from the default opportunistic way it finds and links against external
> libs
> > when configuring. Sure driver building can be controlled with options,
> but
> > why do I need to do this to build the *base* library? If libgdal was
> rarely
> > built and all drivers loaded as drivers, programs like QGIS wouldn't
> crash
> > because some obscure driver's lib changed (happens often with Homebrew).
> > This prompts a required rebuild of libgdal again.
> >
> > I realize that this will take some extra work, and loading all drivers as
> > plugins may reduce some performance metric(s). The resulting CMake
> > configure options, though, could be specific to driver, not library, e.g.
> > -D GDAL_MRSID=ON would generate that plugin, building only the main
> libgdal
> > binary and any other dependent lib or driver plugins (none here) as
> needed.
> > Build time would be minimal, with the plugin immediately available for
> > distribution packaging.
> >
> > Surely I'm not the only person who would love to see this, and CMake
> > porting would be the ideal time to craft this.
> >
> > *External projects*
> > While Borsch has some awesome features, it currently requires out-of-tree
> > resources. If building GDAL with all dependencies local, like may be
> > required for a full static iOS build, is a goal, then the existing CMake
> > ExternalProject feature can be leveraged. A fine example of this is the
> > SuperBuild aspect of Orfeo ToolBox [1]. Such a setup could existing
> in-tree
> > for GDAL, though there is obvious overhead in maintaining
> > ExternalProject build scripts for many libs; so, maybe only for a
> standard
> > set makes sense.
> >
> >
> > Btw, CMake works very well on macOS and is my preferred build tool,
> > especially when used with Ninja generator.  :^)
> >
> > [0] https://github.com/OSGeo/homebrew-osgeo4mac/tree/master/Formula
> > [1] https://github.com/orfeotoolbox/OTB/tree/develop/SuperBuild
> >
> > Regards,
> >
> > Larry Shaffer
> > Dakota Cartography
> > Black Hills, South Dakota
> > ----------------------------------
> > Boundless Desktop and QGIS Support/Development
> > Boundless Spatial - http://boundlessgeo.com
> > lshaffer at boundlessgeo.com
> >
> > On Tue, Oct 31, 2017 at 5:54 AM, Mateusz Loskot <mateusz at loskot.net>
> wrote:
> >
> >> On 30 October 2017 at 23:14, Even Rouault <even.rouault at spatialys.com>
> >> wrote:
> >>> Trying to sum up my thoughts on this topic and answering to various
> >> points
> >>> raised in this discussion thread:
> >>>
> >>> - I believe a relevant question to ask to ourselves would be:
> >>> "imagine that GDAL would come without any build system at all, what is
> >> the one that we would add" ?
> >>
> >> Assuming multi-platform is the critical requirement, there are no real
> >> altenratives that beat CMake.
> >>
> >>> Ok, that's a bit silly to turn the question like that, a more
> >>> realistic one would be
> >>> "imagine you're going to create a software that will rule over the
> >> world, for the 20 next years
> >>> and beyond, and should run on all reasonable platforms, which build
> >> system would we use?
> >>
> >> IMHO, answer to that is:
> >> First, write portable code in build system agnostic way:
> >> - Stick to C/C++ standard libraries as much as possible
> >> - Stick to vanila preprocessor magic to handle C/C++ library
> >> differences and incompatibilities.
> >> - Eliminate or platform-specific generated headers (or keep it small,
> >> one for all)
> >> Then, choose multi-platfor build system and keep build configuration
> >> scripts *small*.
> >>
> >> That will make it easy to compile the code with any build system
> >> current user wants to use
> >> or any build system the project will switch over to in future, if
> >> necessary.
> >>
> >>> If the answer is clearly "cmake", then it is worth examining if there
> is
> >> not a path that
> >>> would lead us to that point.
> >> No, the anwer is not "clearly cmake".
> >> CMake is great because it's multi-platform, it is available virtually
> >> everywhere and easy to install.
> >> CMake is terrible because it did not develop any best practices or
> >> conventions regarding how
> >> a perfect build configuration should be developed with CMake. That
> >> historically led to home grown,
> >> overgrown, bloated piles of CMake scripts specific to projects that
> >> require maintenance on its own.
> >> Even worse, many of those badly written scripts made it into CMake
> >> distribution as FindXXX.cmake
> >> modules, and CMake team did not reject or unify them.
> >> The freedom led to ecosystem of numerous distinct build monsters
> >> governed by specific conventions
> >> based on CMake like ECM from KDE, BCM from Boost, and Borsch fall
> >> sinto similar category, I think.
> >> Naturally, GDAL would grown its own, beause CMake is rubbish :-)
> >>
> >> An ideal build configuration scripts should be small, even for a large
> >> project.
> >> Here is example of such setup proposed for Boost
> >> https://github.com/ned14/boost-bmv-cmake
> >> There is no pile of custom CMake macros or modules provided and that
> >> is an ideal :)
> >>
> >>> Similar question: is it an effort that will make GDAL development a bit
> >>> easier for new contributors?
> >> Perhaps, potentially, may be.
> >> Certainly, CMake makes development friendly for use with modern IDEs.
> >>
> >>> A nice side effect could also be the opportunity to drop some cruft
> that
> >> has
> >>> accumulated over years in the current build systems (supporting ancient
> >>> library versions that no longer make sense)
> >> Dropping existing cruft is good.
> >> Replacing cruft with new CMake cruft is bad idea.
> >> Unfortunately, in most cases I've seen that is what happens when a
> >> project migrates to CMake.
> >>
> >> Best regards,
> >> --
> >> Mateusz Loskot, http://mateusz.loskot.net
> >> _______________________________________________
> >> gdal-dev mailing list
> >> gdal-dev at lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> >>
> >
> >
> > _______________________________________________
> > gdal-dev mailing list
> > gdal-dev at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
--
http://schwehr.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171129/ab948ab4/attachment-0001.html>