[gdal-dev] Code sprint Korea, reformat sources tree

[Dmitry Baryshnikov]

Hi Even,

05.09.2015 17:10, Even Rouault пишет:
> Dmitry,
>
>> During the code sprint in FOSS4G 2015 (Korea, Seoul) I plan to start
>> refactoring Cmake for GDAL (everybody are welcome
>> http://2015.foss4g.org/programme/code-sprint/). This is good starting
>> point to try release an idea to reformat source tree (combine drivers on
>> some principles - raster/vector/raster+vector). I digging the mailing
>> list, but didn't found discussion started by Even about this.
> Regarding unified drivers, it was a bit mentionned in
> https://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification . Basically the
> PCIDSK drivers have been merged in frmts/pcidsk, the PDF ones in frmts/pdf.
> And the raster side of GPKG has been added to the existing
> ogr/ogrsf_frmts/geopackage
> Potential changes on the tree structure were left out in the "Potential
> changes that are *NOT* included in this RFC" paragraph.
I plan to experiment with this and if I get good results, RFC will be 
written.
>> Also we have
>> new type of drivers - network. So, how it'll be best to organise sources?
>> This can be not only drivers, but the whole source tree. How should the
>> ideal GDAL source tree looks like?
>>
>> Also I plan:
>> 1. Move all internal libraries (zlib, libtiff, libjpeg, etc.) to
>> separate github repos to use CMake ExternalProject feature.
> Just to give some context: the point for the internal libraries was to have a
> no-brainer way of building GDAL without any prerequisite.
> - internal zlib is identical to its upstream v1.2.3 AFAIK
> - internal libtiff: most files are identical to libtiff CVS, but a few ones
> (tiffconf.h, tif_config.h) have been modified for integration with GDAL CPL, and
> tif_vsi.c is GDAL specific (I/O implementation) + a build time hack for TIFF
> JPEG 12 bit support
> - internal libjpeg is mostly upstream libjpeg v6b + one patch. There's also
> the build hack for libjpeg12
I only plan to move this internal libraries in separate repos, not to 
link official ones. So this is only give more structured sources tree.
>
>> 2. Remove any other building systems
> That sounds ambitious. Given the complexity and maturity of our current build
> systems, I guess this would take some time to have CMake catch up.
Yes, certainly. But anyhow current CMake branch not fully consistent for 
current build system. So this have to be done.
>
>> 3. Try CTest for testing
> What do you think it will bring w.r.t our current testing system ? Do we want
> to be dependant of a particular build system for our tests ?
> Regarding testing, I've somehow understood Kurt had mentionned plans for a
> "gdalautotest2"
This is only subject of experiments. Let's try CTest and see if it fits.
>
> Regarding all the above, I assume you mean in a fork of yours ?
Yes. All experiments will be on forked GDAL in separate branch.
>
>> As for me the ideal structure should looks like:
>> + apps
>> + autotests
>> + bindings
>> + core
>>     + port
>>     + ogr
>>     + gcore
> gnm core would go here too ?
Yes
>
>> + cmake
>> + data
>> + docs
>>     + doxygen
>>     + readme
>> + drivers
>>     + raster
>>     + vector
>>     + network
>>     + combined
>> + CMakeLists.txt
>> + LICENSE
>>
>> So, at the root of sources tree we will have only 8 folders and 2 files.
> Is the churn in the tree structure worth the effort ? Be aware that there are a
> number of interdependencies between drivers, so this might require fixing a
> number of source files. What advantages do you see in a new structure ?
1. More ease to understand sources tree for novice.
2. More useful for CMake macro. In current release there are a lot of 
hardcoded things. Macro give more flexibility.
3. More ease to add some new check needed by separate drivers.
4. More configurable (ease selected depended libraries installed in OS, 
or should be loaded via ExternalProject), more dependence checks.
5. May be CPack using in future to create distros.
>
> I'm afraid that if you want to change multiple things at a time (build system,
> testing mechanisms, tree structure), you will never manage to get a working
> result. Incremental approaches when feasible are less risky (but admitedly
> involve potentially a larger cumulated effort).
Yes, you may be right. But it seems to me that current Cmake version is 
too complicated than it can be. If Ican improve it it'll solve lot of 
problems, if not - ok this will be only an unsuccessful experiment.
>
> Even
>
I do not insist, maybe it's a crazy idea. But as was the discussion of 
unification, it seemed to me that this worth trying during improvements 
Cmake build system.

Best regards,
     Dmitry