[gdal-dev] About CMake build again

John Daniel jdaniel at etresoft.com
Mon Oct 30 12:20:52 PDT 2017 

Xcode builds, targeting both macOS and iOS, are my primary concern. I have to use CMake for MySQL and JPEG2000. MySQL is only for server-side work and testing because licensing issues keep it out of any distributed products. JPEG2000 works fine. I have tried and failed to get the current, CMake-based KML working. I can’t figure it out and neither can the KML developers. Instead, I am using the last autotools version. At some point I will port source code fixes from the new version back to my autotools branch.

I always assumed that CMake only works on Linux, and maybe Windows I guess. You can’t use macOS/iOS/Xcode as a rationale for using CMake. I can’t imagine the GDAL CMake maintainers having the Apple hardware, accounts, and expertise necessary to keep that working. I certainly don’t have the CMake expertise. My interest in all of this is working with GIS data and GDAL, not CMake. 

The problem is that CMake is such an opaque black box. If it doesn’t work the first time, you have to dive very deep into the internals to figure it out. Autotools, for all its faults, needs only decent UNIX skills to get working. My build scripts are very complicated, but they do work. The only part I ever completely failed at was a CMake library. 

If you want to sell CMake as a Windows solution, fine. I don’t do Windows and I don’t care. But CMake can’t handle Apple platforms. 

John Daniel
Etresoft, Inc.

> On Oct 29, 2017, at 11:15 AM, Dmitry Baryshnikov <bishop.dev at gmail.com> wrote:
> 
> Hi Kurt,
> 
> And how to deal with non Linux builds like Windows MSVC, mac OS XCode etc.?
> 
> Best regards,
>    Dmitry
> 
> 29.10.17 17:17, Kurt Schwehr пишет:
>> While autoconf isn't pretty, it has done me the least amount of harm of any
>> of the major build systems.  Without patches for the downstream packagers
>> and CI (travis/appveyor), I would be -1.  e.g. debian/ubuntu,
>> redhat/centos, macport/brew/fink, and others
>> 
>> With those patches, I'm -0.
>> 
>> BTW, It's not unreasonable to have separate build systems.  It's work, but
>> very doable.  I maintain a bazel build environment.  I have to notice add
>> or deletes of files in upstream, but it's been working for more than a
>> decade.
>> 
>> On Sat, Oct 28, 2017 at 11:11 PM, Alexander Bruy <alexander.bruy at gmail.com>
>> wrote:
>> 
>>> While I'm not GDAL developer or regular contributor (submitted only
>>> few patches),
>>> but still often build GDAL trunk on WIndows and Linux, and less often on
>>> Mac.
>>> 
>>> Personally I think that solution proposed in
>>> https://trac.osgeo.org/gdal/ticket/7080
>>> is more preferable. It does not require *all* dependencies to be build
>>> with cmake,
>>> it does not require maintaining forks with special directory structure for
>>> *all*
>>> dependencies, it plays well with conventions/packages provided on all
>>> systems
>>> out of the box. Maybe borsch is good for in-house use when all stack
>>> is build from
>>> scratch but it is not suitable for real-world use cases.
>>> 
>>> Of course, #7080 is not ideal, there are some issues, but as I understand
>>> the
>>> work is not over and most (if not all) issues can be solved.
>>> 
>>> Just my 2c.
>>> 
>>> 2017-10-28 0:06 GMT+03:00 Dmitry Baryshnikov <bishop.dev at gmail.com>:
>>>> Hi,
>>>> 
>>>> As Even said it make sense to move discussion from this ticket
>>>> (https://trac.osgeo.org/gdal/ticket/7080) to the list.
>>>> 
>>>> First of all I would like to make small introduction to Borsch project.
>>> Here
>>>> it is some useful links:
>>>> 
>>>> * Borsch repository: https://github.com/nextgis-borsch/borsch
>>>> 
>>>> * Presentation on FOSS4G 2016:
>>>> http://nextgis.ru/wp-content/uploads/2016/08/NextGIS-Borsh-
>>> presentation.pdf
>>>> * GDAL repository adapted for Borsch:
>>>> https://github.com/nextgis-borsch/lib_gdal
>>>> 
>>>> Shortly speaking Borsch is 3 CMake scripts which add ability to include
>>>> dependence library in one line of code.
>>>> 
>>>> It looks like:
>>>> 
>>>> find_anyproject(TIFF REQUIRED)
>>>> 
>>>> Certainly developer can add additional parameter to configure dependency
>>> in
>>>> find_anyproject function.
>>>> 
>>>> Inside  find_anyproject work 2 cases:
>>>> 
>>>> 1. First of all (by default, but can be overridden) CMake searches host
>>>> system for dependency library (headers and lib files). This is usual
>>> CMake
>>>> find_package (https://cmake.org/cmake/help/
>>> v3.0/command/find_package.html)
>>>> with some additional logic to try different options for hard cases (like
>>>> Qt).
>>>> 
>>>> 2. If library not found, find_anyproject can get the sources or prebuild
>>>> library from github.
>>>> 
>>>> So the GDAL or any other library can be build the normal way, but
>>> developer
>>>> have additional features to configure build all libraries with one
>>> compiler
>>>> and one set of compiler/linker settings (with some limits). Such way we
>>> can
>>>> have rather complicated scenarios to build GDAL and dependencies.
>>>> 
>>>> Here it is several examples of benefits of this approach:
>>>> 
>>>> 1. NextGIS Mobile SDK v3. SDK based on GDAL and need it in one library
>>> for
>>>> iOS as *.framework and for Android as *.so (arm7, arm64, i386, x86_64
>>>> architecture). I build all dependencies include GDAL statically and link
>>> in
>>>> one fat library. The all code that do it:
>>>> https://github.com/nextgis/nextgis_datastore/blob/master/
>>> cmake/extlib.cmake#L118-L236
>>>> By the way the library also builds on Linux and Mac OS (Windows under
>>>> development) and CMake try to use existed in host system libraries. If
>>> CMake
>>>> find GDAL in host system it will use it and all (-DENABLE_PLSCENES=OFF
>>> ... )
>>>> will be ignored as it already build with another parameters.
>>>> 
>>>> 2. Build GDAL Windows standalone installer and GDAL Ubuntu ppa:
>>>> https://github.com/nextgis/ppa/blob/master/gdal/master/debian/rules
>>>> 
>>>> 3. Build QGIS
>>>> (https://github.com/nextgis/nextgisqgis/blob/master/CMakeLists.txt#L149
>>> ),
>>>> PostGIS
>>>> (https://github.com/nextgis-borsch/postgis/blob/master/
>>> CMakeLists.txt#L165),
>>>> Formbuilder
>>>> (https://github.com/nextgis/formbuilder/blob/master/cmake/
>>> extlib.cmake#L53-L173)
>>>> This is main Borsch features.
>>>> 
>>>> 
>>>> There are some additional conventions like:
>>>> 
>>>>     * I modify all libraries included into Borsch repository to install
>>> on
>>>> unix-like paths. For Linux this is usual, for Windows and Mac OS this
>>> let us
>>>> to use Qt installer framework an install software mach similar like on
>>>> Linux. This is about target "install" which is vary on different
>>> libraries
>>>> (CMake has it own conventions about it). This is not mandatory for Borsch
>>>> itself but useful. CMake can register installed libraries in system
>>>> repository to simplify find them in find_package function.
>>>> 
>>>>     * CMake get library version from sources in all libraries included
>>> into
>>>> Borsch (if applicable, otherwise set it in CMake script). This is
>>> necessary
>>>> if exact version of library needed. This is not mandatory. One more
>>> benefit
>>>> during building process we can see dependency library version in console.
>>>> 
>>>>     * We modify all libraries included into Borsch repository to find
>>>> dependencies using find_anyproject. It is simple to use libraries from
>>> our
>>>> borsch repository, but developer can fork them or use any other sources
>>> and
>>>> build systems to have dependency library in it's host system.
>>>> 
>>>> One can see this is all very flexible.
>>>> 
>>>> 
>>>> What about GDAL.
>>>> 
>>>> 1. After unification GDALDataset and OGRDatasource current sources tree
>>> is
>>>> not fit for this new logic of GDAL classes. I rearranged sources more
>>> closer
>>>> to GDAL classes and CMake needs. Main changes are moving raster and
>>> vector
>>>> drivers inside drivers folder
>>>> (https://github.com/nextgis-borsch/lib_gdal/tree/master/drivers).This
>>>> simplify situation where different drivers need the same dependency
>>> library
>>>> (libpg, libsqlite, etc.). Also there are several raster/vector drivers
>>> which
>>>> need a separate directory but now presented in ogr or frmts directories.
>>>> There are some bad decisions I made - for example I moved unit tests into
>>>> separate repository - this was a mistake. We will return unit tests back
>>> to
>>>> GDAL repository.
>>>> 
>>>> An example of cmake friendly way see
>>>> https://github.com/nextgis-borsch/lib_gdal/blob/master/
>>> drivers/vector/CMakeLists.txt.
>>>> The driver developer must only create new folder and put CMakeLists.txt
>>> file
>>>> into it. The upper CMake script will find new driver and add it to GDAL
>>>> build. In common cases no need to modify upper CMake scripts.
>>>> 
>>>> 2. I remove third-party code from drivers folders (TIFF, GeoTIF, PNG,
>>> JPEG
>>>> etc). All this code are in separate repositories. I don't see the
>>> difference
>>>> to get this code from git pull from main GDAL repository or from the
>>>> separate repository via find_anyproject process. Current GDAL repository
>>>> looks like the https://github.com/nextgis-borsch packed in one
>>> repository.
>>>> 
>>>> In conclusion:
>>>> 
>>>> 1. Borsch added flexible and useful features and not remove existing
>>>> approach
>>>> 
>>>> 2. The current cmaked GDAL are in production in my company more than a
>>> year
>>>> on Windows, Linix, Mac OS, iOS, Android.
>>>> 
>>>> 3. I'm ready to discuss and improve current solution. Any help are
>>> welcome
>>>> 4. I also will be happy to contribute directly or via PR into GDAL trunk
>>>> instead of backporting in both directions improvements that we do in
>>> GDAL .
>>>> 
>>>> Finally:
>>>> 
>>>> Find the link to page with the CMake in GDAL discussion -
>>>> https://trac.osgeo.org/gdal/wiki/CMake
>>>> 
>>>> --
>>>> Best regards,
>>>>     Dmitry
>>>> 
>>>> 
>>>> _______________________________________________
>>>> gdal-dev mailing list
>>>> gdal-dev at lists.osgeo.org
>>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>> 
>>> 
>>> --
>>> Alexander Bruy
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>> 
>> 
>> 
> 
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

More information about the gdal-dev mailing list