[gdal-dev] About CMake build again

Sun Oct 29 06:39:49 PDT 2017

Hi Alexander,

Please read carefully that I wrote before. There are 3 options:

    1. Build gdal with all dependencies getting them from github
    (WITH_EXTERNAL). Preferable for Windows

    2. Build gdal using the system libraries. Preferable for Linux

    3. Build gdal using the dependency libraries build by user (out of
    source) and registered in CMake package registry. This is I use now.
    This script help me to clone all libraries, build them and register
    them in CMake package registry
    (https://github.com/nextgis-borsch/borsch/blob/master/opt/tools.py#L134).

Again, this is the options which one can choose. So if building 
everything from scratch not suits you - just select another option. I 
introduce presets (now there is only one for option 1 - 
https://github.com/nextgis-borsch/lib_gdal/issues/14) and this can be 
expanded for different scenarios. If you need full control on build just 
provide command line options to CMake and get what you need.

My fast code review solution of Hiroshi shows that it very far from 
working solution and need much work that already done in lib_gdal from 
Borsch.

My opinion is that different options is much flexible and suits for many 
scenarios as very limited solution from 
https://trac.osgeo.org/gdal/ticket/7080.

Best regards,
     Dmitry

29.10.17 9:11, Alexander Bruy пишет:
> While I'm not GDAL developer or regular contributor (submitted only
> few patches),
> but still often build GDAL trunk on WIndows and Linux, and less often on Mac.
>
> Personally I think that solution proposed in
> https://trac.osgeo.org/gdal/ticket/7080
> is more preferable. It does not require *all* dependencies to be build
> with cmake,
> it does not require maintaining forks with special directory structure for *all*
> dependencies, it plays well with conventions/packages provided on all systems
> out of the box. Maybe borsch is good for in-house use when all stack
> is build from
> scratch but it is not suitable for real-world use cases.
>
> Of course, #7080 is not ideal, there are some issues, but as I understand the
> work is not over and most (if not all) issues can be solved.
>
> Just my 2c.
>
> 2017-10-28 0:06 GMT+03:00 Dmitry Baryshnikov <bishop.dev at gmail.com>:
>> Hi,
>>
>> As Even said it make sense to move discussion from this ticket
>> (https://trac.osgeo.org/gdal/ticket/7080) to the list.
>>
>> First of all I would like to make small introduction to Borsch project. Here
>> it is some useful links:
>>
>> * Borsch repository: https://github.com/nextgis-borsch/borsch
>>
>> * Presentation on FOSS4G 2016:
>> http://nextgis.ru/wp-content/uploads/2016/08/NextGIS-Borsh-presentation.pdf
>>
>> * GDAL repository adapted for Borsch:
>> https://github.com/nextgis-borsch/lib_gdal
>>
>> Shortly speaking Borsch is 3 CMake scripts which add ability to include
>> dependence library in one line of code.
>>
>> It looks like:
>>
>> find_anyproject(TIFF REQUIRED)
>>
>> Certainly developer can add additional parameter to configure dependency in
>> find_anyproject function.
>>
>> Inside  find_anyproject work 2 cases:
>>
>> 1. First of all (by default, but can be overridden) CMake searches host
>> system for dependency library (headers and lib files). This is usual CMake
>> find_package (https://cmake.org/cmake/help/v3.0/command/find_package.html)
>> with some additional logic to try different options for hard cases (like
>> Qt).
>>
>> 2. If library not found, find_anyproject can get the sources or prebuild
>> library from github.
>>
>> So the GDAL or any other library can be build the normal way, but developer
>> have additional features to configure build all libraries with one compiler
>> and one set of compiler/linker settings (with some limits). Such way we can
>> have rather complicated scenarios to build GDAL and dependencies.
>>
>> Here it is several examples of benefits of this approach:
>>
>> 1. NextGIS Mobile SDK v3. SDK based on GDAL and need it in one library for
>> iOS as *.framework and for Android as *.so (arm7, arm64, i386, x86_64
>> architecture). I build all dependencies include GDAL statically and link in
>> one fat library. The all code that do it:
>> https://github.com/nextgis/nextgis_datastore/blob/master/cmake/extlib.cmake#L118-L236
>>
>> By the way the library also builds on Linux and Mac OS (Windows under
>> development) and CMake try to use existed in host system libraries. If CMake
>> find GDAL in host system it will use it and all (-DENABLE_PLSCENES=OFF ... )
>> will be ignored as it already build with another parameters.
>>
>> 2. Build GDAL Windows standalone installer and GDAL Ubuntu ppa:
>> https://github.com/nextgis/ppa/blob/master/gdal/master/debian/rules
>>
>> 3. Build QGIS
>> (https://github.com/nextgis/nextgisqgis/blob/master/CMakeLists.txt#L149),
>> PostGIS
>> (https://github.com/nextgis-borsch/postgis/blob/master/CMakeLists.txt#L165),
>> Formbuilder
>> (https://github.com/nextgis/formbuilder/blob/master/cmake/extlib.cmake#L53-L173)
>>
>> This is main Borsch features.
>>
>>
>> There are some additional conventions like:
>>
>>      * I modify all libraries included into Borsch repository to install on
>> unix-like paths. For Linux this is usual, for Windows and Mac OS this let us
>> to use Qt installer framework an install software mach similar like on
>> Linux. This is about target "install" which is vary on different libraries
>> (CMake has it own conventions about it). This is not mandatory for Borsch
>> itself but useful. CMake can register installed libraries in system
>> repository to simplify find them in find_package function.
>>
>>      * CMake get library version from sources in all libraries included into
>> Borsch (if applicable, otherwise set it in CMake script). This is necessary
>> if exact version of library needed. This is not mandatory. One more benefit
>> during building process we can see dependency library version in console.
>>
>>      * We modify all libraries included into Borsch repository to find
>> dependencies using find_anyproject. It is simple to use libraries from our
>> borsch repository, but developer can fork them or use any other sources and
>> build systems to have dependency library in it's host system.
>>
>> One can see this is all very flexible.
>>
>>
>> What about GDAL.
>>
>> 1. After unification GDALDataset and OGRDatasource current sources tree is
>> not fit for this new logic of GDAL classes. I rearranged sources more closer
>> to GDAL classes and CMake needs. Main changes are moving raster and vector
>> drivers inside drivers folder
>> (https://github.com/nextgis-borsch/lib_gdal/tree/master/drivers).This
>> simplify situation where different drivers need the same dependency library
>> (libpg, libsqlite, etc.). Also there are several raster/vector drivers which
>> need a separate directory but now presented in ogr or frmts directories.
>> There are some bad decisions I made - for example I moved unit tests into
>> separate repository - this was a mistake. We will return unit tests back to
>> GDAL repository.
>>
>> An example of cmake friendly way see
>> https://github.com/nextgis-borsch/lib_gdal/blob/master/drivers/vector/CMakeLists.txt.
>> The driver developer must only create new folder and put CMakeLists.txt file
>> into it. The upper CMake script will find new driver and add it to GDAL
>> build. In common cases no need to modify upper CMake scripts.
>>
>> 2. I remove third-party code from drivers folders (TIFF, GeoTIF, PNG, JPEG
>> etc). All this code are in separate repositories. I don't see the difference
>> to get this code from git pull from main GDAL repository or from the
>> separate repository via find_anyproject process. Current GDAL repository
>> looks like the https://github.com/nextgis-borsch packed in one repository.
>>
>>
>> In conclusion:
>>
>> 1. Borsch added flexible and useful features and not remove existing
>> approach
>>
>> 2. The current cmaked GDAL are in production in my company more than a year
>> on Windows, Linix, Mac OS, iOS, Android.
>>
>> 3. I'm ready to discuss and improve current solution. Any help are welcome
>>
>> 4. I also will be happy to contribute directly or via PR into GDAL trunk
>> instead of backporting in both directions improvements that we do in GDAL .
>>
>>
>> Finally:
>>
>> Find the link to page with the CMake in GDAL discussion -
>> https://trac.osgeo.org/gdal/wiki/CMake
>>
>> --
>> Best regards,
>>      Dmitry
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171029/c05bd86e/attachment-0001.html>