[Qgis-psc] Continuous Deployment for OSGeo4W

[Matthias Kuhn]

Hi Jürgen


On 06/11/2018 10:27 PM, Jürgen E. Fischer wrote:

>> The infrastructure in the background is to my knowledge not documented and we
>> are happy if it just works. And if it breaks if there is someone with
>> knowledge and access rights to fix things.
> The OSGeo4W infrastructure is download.osgeo.org.  It's documented in the trac
> wiki.  The only required access right is a osgeo id with shell access.
>
> There's no building infrastructure.  So maintainers have to build their
> packages on their own machine and upload them.  Some of the package (most of
> mine for instance) also contain (-src) packages with the build recipes for the
> package.
That's good to know, that will make it easier to go on if (and we don't
hope that of course) someone disappears.

> But there currently is no "standardized" way to invoke them, upload the package
> and run the packaging scripts.  The build recipes are diverse and use cmake,
> nmake, ninja, gnu make, batch files, shell scripts (cygwin/mingw) and use
> several different version of compilers (multiple versions of MSVC & mingw gcc).
> Some even require third party SDKs that cannot be redistributed.  There's no
> base build system or way to install build dependencies.
>
> AFAIK automated packages only exists for QGIS, GDAL and GRASS - those run on
> internal infrastructure (here inhouse for QGIS/GDAL and I think at the
> university Martin works for GRASS) and upload the results.
>
>
>> I just stumbled upon a repository from Oslandia [2] that streamlines the
>> packaging and integrates it with continuous deployment apparently.
> Odd that you had to stumbled upon it - this was also presented and discussed at
> the Madeira hackfest and it's one of the benifits of moving from github to
> gitlab.

Then this must be caused by my absence in Madeira ;)
If required a gitlab mirror of the current github repo to trigger
gitlab-ci builds should be the smallest part to setup within this
infrastructure ;)

> But it currently also depends on internal build servers (at oslandia).  It
> doesn't upload to the osgeo4w repository, but integrates the repository and has
> utility scripts to invoke the build, harvest the packages and add them to taht
> repository.  So the current packaging scripts would have to be adapted.
Reading this and the above paragraph, I think it's a quite good approach
here, proxying to the existing pre-built packages and building a
scripted repo on top that eventually can obsolete packages one-by-one.

>
>
>> Having such a repository under QGIS or OSGeo umbrella could help to take away
>> the burden of maintaining the Windows distribution from very few people,
>> having a pull request review scheme as we have for source code and distribute
>> the responsibility.
>> In short, having a repository where package scripts and build scripts are
>> publicly available, forkable, builds are reproducible and where it's possible
>> to make pull requests.
>> The main question in the first line is probably, is that something which is
>> considered a QGIS task, an OSGeo task or a task that individual companies
>> should be taking?
> This might introduce a long enough pole to make it bearable for more devs to
> touch windows and get more contributors for OSGeo4W. ;)
That's my hope :)

> If we wouldn't need (to spend funds on) windows licenses to get this to fly, we
> might already have public build machines.  And those seems to be the current
> missing bits.
I read between the lines that there have been previous (PSC) discussions
about this before. And I assume the bottom line was that donations
should not be used to feed MS. So I think it might be an opportunity of
user groups or organizations to jump in and fill the gap here. If that's
the case, I will see if I can do something to get that fixed.

Matthias


>
>
> Jürgen
>
>
>
>
> _______________________________________________
> Qgis-psc mailing list
> Qgis-psc at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/qgis-psc

-- 
Matthias Kuhn
matthias at opengis.ch <mailto:matthias at opengis.ch>
+41 (0)76 435 67 63 <tel:+41764356763>
OPENGIS.ch Logo <http://www.opengis.ch>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20180612/931ba100/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 6671 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20180612/931ba100/attachment.png>