[QGIS-Developer] compare failing/succeeding CI builds (how to?)

[Sandro Santilli]

On Thu, Mar 19, 2020 at 08:27:04AM +1000, Nyall Dawson wrote:
> On Wed, 18 Mar 2020 at 05:32, Sandro Santilli <strk at kbt.io> wrote:
> 
> > > This one not expected. What gdal/proj version do you use?
> >
> > I'm reporting what Travis says, so whatever GDAL/PROJ version is
> > running on Travis. This is the run of this PR:
> > https://github.com/qgis/QGIS/pull/31980
> 
> Hm, in that case something you've done has broken (or at least
> changed) the test...

Debugging this on Travis is an hell. Problem is some tests fail on
Travis but succeed on my system. The processing one is too big, but
this simple one succeeds locally:

  [strk at liz:/usr/src/qgis/build/do-not-override-source-files] ctest -R qgis_distanceareatest
  Test project /usr/src/qgis/build/do-not-override-source-files
      Start  1: logGitStatus
  1/3 Test  #1: logGitStatus .....................   Passed    0.02 sec
      Start 50: qgis_distanceareatest
  2/3 Test #50: qgis_distanceareatest ............   Passed    1.53 sec
      Start  2: checkGitStatus
  3/3 Test  #2: checkGitStatus ...................   Passed    0.02 sec

  100% tests passed, 0 tests failed out of 3

While it fails on Travis!
https://travis-ci.org/github/qgis/QGIS/jobs/659060340#L5117


 	50/601 Test  #51: qgis_distanceareatest .............................***Failed    0.58 sec
	QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
	********* Start testing of TestQgsDistanceArea *********
	Config: Using QtTest library 5.9.5, Qt 5.9.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 7.4.0)
	[...]
	FAIL!  : TestQgsDistanceArea::measureAreaAndUnits() Compared values are not the same
		 Actual   (units)                       : AreaUnknownUnit
		 Expected (QgsUnitTypes::AreaSquareFeet): AreaSquareFeet
		 Loc: [../tests/src/core/testqgsdistancearea.cpp(355)]
	[...]
	Totals: 12 passed, 1 failed, 0 skipped, 0 blacklisted, 316ms

How could that be explained ? Different PROJ version maybe ?

Locally I'm using

	[strk at liz:~] proj --version
	Rel. 7.0.0, March 1st, 2020

--strk;