[gdal-dev] Windows CI build times

Even Rouault even.rouault at spatialys.com
Wed Jun 3 15:41:01 PDT 2020

On lundi 20 avril 2020 20:16:41 CEST Howard Butler wrote:
> > On Apr 16, 2020, at 7:46 PM, Mateusz Loskot <mateusz at loskot.net> wrote:
> > 
> > On Fri, 17 Apr 2020 at 01:45, Mike Taves <mwtoews at gmail.com> wrote:
> >> It seems that many projects have shifted from AppVeyor to AZP for
> >> performance benefits.
> > 
> > Or GitHub Actions, which is an incarnation of AzP.
> Having done both GitHub Actions and AzP, I think Actions has more convenient
> expressiveness of job flow, a big win of integration with GitHub's API, and
> large directory of user-contributed actions.

https://github.com/OSGeo/gdal/pull/2648 has now a migration of the VS2019 build to 
GitHub actions, which will improve response time by cutting half of the load on AppVeyor

For anyone attempting similar migration, here are a few random points on which I stumbled 
- git on GitHub actions has the default behaviour of git on Windows, that is it must have 
core.autocrlf=true or whatever makes git turn LF into CRLF on some files. Fix was to set 
- powershell on GitHub actions exits as soon as a command in a script fails. Which is 
reasonable, but for some reason our GDAL appveyor powershell sequences don't like that, 
especially when used with the black magic exec() function we have in appveyor.yml that 
seems to be an emulation of the default Github action mode. Fix was to set 
$ErrorActionPreference = 'continue'
- anectodical: "Install-PackageProvider NuGet -Force", whatever that does, failed on github 
actions, but that turned out to be unnecessary.
(I now know at least I actively hate powershell)

So on AppVeyor remains the VS 2015 build, since github actions doesn't offer this compiler. 
At some point we will probably be able to remove VS 2015 builds. The main reason for 
keeping it is that OSGeo4W still uses VS 2015 to build GDAL, but this is going to be updated 
to 2019 soon according to J├╝rgen.

I've also migrated recently the Mac Travis-CI jobs, and all code quality checks into github 
actions. Was easier.

I will probably defer on migrating more things, that is the remaining Linux Travis-CI jobs. 
From a discussion with @pyusr on IRC, one solution to consider, and that would make us 
mostly CI host independent, would be to use Docker builds instead of relying on the VM 
provided by the CI host. I haven't considered that more, but on the paper it seems 
interesting. He was pointing to
https://github.com/tzickel/chunkedbuffer/blob/master/.github/workflows/test.yml +  
https://github.com/tzickel/chunkedbuffer/blob/master/Dockerfile.test as an example . One 
potential issue is that we have some Linux builds that start Docker containers for databases, 
so I'm not sure how that would interact with the build & tests being done in docker build. 
Would probably work by doing them as separate preliminary steps in a job.


Spatialys - Geospatial professional services
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200604/fb477cd5/attachment.html>

More information about the gdal-dev mailing list