[gdal-dev] Travis CI continuous integration service for GDAL

[Even Rouault]

Hi,

Following a recent similar move from MapServer, I've setup an instance of 
Travis CI, hosted continuous integration service. After each SVN commit in 
trunk, GDAL is built and the Java, Perl and Python tests are run.

Travis CI currently offers Ubuntu 12.04 i386 virtual machines. The GDAL build 
is configured with almost any possible dependencies to external libraries (see 
http://trac.osgeo.org/gdal/wiki/Buildbot for the list)

You can access to the latest builds here :
https://travis-ci.org/#!/rouault/gdal/builds

The result of builds is also notified on #gdal on irc.freenode.net and emailed 
to gdal-commits at lists.osgeo.org (only failures, or failure->success 
transitions are notified). It seems that a full build with tests takes about 10 
minutes (but I guess this may vary according to the load of the service).

When committing, you can prevent a build from being triggered by including the 
"[ci skip]" string in the commit message.

Technical details
------------------------

As Travis CI is closely linked with GIT, I've setup a cron job (with a 1 
minute frequency) on my computer (*) that refreshes a GIT mirror of the GDAL 
SVN trunk, and pushes it to https://github.com/rouault/gdal . Travis-CI is 
notified by GitHub each time a commit is done into that repository.

A http://svn.osgeo.org/gdal/trunk/.travis.yml file has been added at the root 
of the trunk. This file contains the instructions to download dependency 
libraries, setup databases (PostgreSQL, MySQL) that will be used for testing, 
build GDAL and its bindings, and run the tests.
 
For GDAL committers that wish to tweak .travis.yml, you can find the syntax of 
that file at http://about.travis-ci.org/docs/user/build-configuration/

One caveat of that system is that you don't have direct access to the Travis 
CI VM. So when tests fail on it and not on your own development machine, it 
might be hard to debug and fix them. Setting up a Ubuntu 12.04 i386 server 
edition VM and reproducing the instructions of .travis.yml will help in most 
cases. If a test fails specifically on Travis and cannot be fixed, you can skip 
it with

if gdaltest.skip_on_travis():
	return "skip"

in the Python autotest suite. One limitation I found is that the tests that 
create huge sparse files (several gigabytes) fail, probably due to the disk 
quota affected to the VM. There are also failures in a few tests involving 
gdal.VSIStatL() that I couldn't reproduce on my own VM, so those failures have 
been turned into skip to keep the build "green".

Note that to speed up the build, GDAL is compiled with clang 3.1 with --
enable-debug. Due to timeouts that would kill the build instance if the build 
and tests take too much time, I haven't turned on the GDAL_DOWNLOAD_TEST_DATA 
= YES environment variable that causes big test datasets, which are not in 
SVN, to be downloaded, so those tests will be skipped. For not too big 
datasets, manual downloading can still be done in .travis.yml (see the 
"script" section).

Best regards,

Even

(*)  The consequence is that when I turn off my computer for vacations, the 
refresh of the GIT repository will be paused.