[osgeo4w-dev] Alternative osgeo4w infrastructure

[Hugo Mercier]

Hi,

We've been involved recently in windows packaging based on osgeo4w. Some
of our customers needed binaries for windows for some of our projects
which require a recent C++ compiler.

We would like to share and discuss our current infrastructure and see if
there are some shared interests.

We've setup a compilation / CI infrastructure based on Gitlab CI that
triggers Virtual box virtual machines for the actual compilation and
packaging.
We use Windows 10 "development" VM
(https://developer.microsoft.com/en-us/windows/downloads/virtual-machines)
with Visual Studio 2015.
It allows us to have a common clean compilation environment.
Here is an example of a build log:
https://gitlab.com/Oslandia/tempus_core/builds/14314756

For now, one of our server "supersedes" the official osgeo4w http
repository so that we have our own osgeo4w repository for our packages
and still benefit from updated packages from osgeo4w.
You can have a look at the packages we have so far here:
http://osgeo4w.oslandia.net/osgeo4w/x86_64/release/ (some packages have
src packages with them, but not all of them yet)

Among these new packages, some of them may be of interest for other
projects, mainly PostgreSQL and PostGIS (only minimal version of PostGIS
for now - no raster, no sfcgal).

The goal is then for us to contribute back to the osgeo4w project.

So far, we noticed some issues with the current osgeo4w project and
infrastructure:
- the main repository seems heavily loaded and hard to reach sometimes
- packaging fixes or addition of new packages relies on developers to
install and manage their own instance of a complete Windows development
environment.
- the build of the entire distribution is hard to reproduce
- the points listed above also result in a situation where there is a
mix of compiler versions used for packages. Which should be avoided
(http://siomsystems.com/mixing-visual-studio-versions/)
We actually hit a problem probably related to that
(https://trac.osgeo.org/osgeo4w/ticket/529) and that could mean to
recompile gdal and all its dependencies with msvc 2015.

So here are some propositions:
- find a way to add a mirroring system for the osgeo4w distribution. The
idea is to register a list of mirrors that would receive a copy of files
when they are uploaded to the main repository.
Maybe by using something like https://github.com/axkibe/lsyncd
- share access to our gitlab-ci infrastructure and vbox VMs (and allow
it to be copied somewhere else if needed). The idea would be to have a
unique repository with all package "sources" and all
the machinery needed to trigger a build of the whole distribution or of
some selected packages on a known environment.

Regarding the issue of mixing compiler versions, we apparently still
need to compile python2.7 modules with msvc 2010 (which is the version
used to compile python27 and all wheel binaries).
Apart from that (which may imply to maintain two versions of the same
libraries), do you see problems if all packages get compiled with the
same (last) version of msvc (2015) ?

Are there any interests for that work ? Raise your hand if so, and raise
both hands if you could contribute (time or funds) :) we also still need
to find a way to fund this alternative osgeo4w / CI server.

Hope to read your comments.

Hugo