[QGIS-Developer] PSA: Travis CI / Docker / Failing Tests

[Matthias Kuhn]

Dear all,

we have recently been working on bringing travis ci to the next level
and moving all the testing that is done into a docker image.

This has the advantage that

- we can use newer dependencies for testing (currently Ubuntu 16.04
  repos compared to 14.04 repos before)

- tests can be run locally the same way that they are run in CI (to
  debug)

- we can upload docker images (nightlies and released versions) that can
  be used for plugin testing (only partially implemented so far)


During this process, some issues have been discovered, mostly concerning
compatibility with newer gdal versions and most of them fixed (kudos to
Even for his reactive and valuable help with this).

At the moment, we are left with 3 tests that are still failing:

- qgis_rastersublayertest
- PyQgsShapefileProvider
- PyQgsWFSProvider

The reasons that these tests fail are either because the tests
themselves are unstable and rely on very particular behavior of the
environment or because they actually point out real issues related to
newer dependencies.

It would be very welcome if the original authors of these tests or the
underlying functionality could have a look and either

- fix the tests
- fix the underlying issues that caused the tests to fail
- or disable the tests if they are no longer meaningful.

A patch has been prepared that disables parts of the tests that trigger
the failures which can be used as reference to find where things go wrong:

https://github.com/qgis/QGIS/pull/5256/commits/988aa78feded78634d8600c7baeb701558b23546

If there are no fixes in sight, this patch will be merged as part of PR
5256.

Thanks
Matthias