[Qgis-psc] Handling the Travis CI situation

[Alessandro Pasotti]

On Fri, Nov 6, 2020 at 11:29 AM Sandro Santilli <strk at kbt.io> wrote:
>
> On Fri, Nov 06, 2020 at 05:07:40PM +1000, Nyall Dawson wrote:
>
> > - different Qt versions on Github vs Travis
> > - different underlying library versions (e.g. updated proj, gdal, and
> > others vs the older Travis build environment)
> > -  other differences in the build setup, e.g. different limits on the
> > environment under which the tests run
> >
> > Before we can make the switch and move away from Travis, we'll need to
> > update these tests and get as many of them passing as possible on
> > Github, and then handle the tricky job of backporting the test fixes
> > and Github action setup to the 3.16/3.10 branches.
>
> I'd recommend changing the tests to return success results with BOTH
> versions of Qt, proj, gdal etc. The goal would be making the testsuite
> pass on _developers_ machines, whatever libraries they are using.
>
> The CI-centric nature of QGIS testsuite has always been a problem for
> me as it makes it impossible to check the quality of changes locally.
>
> There are likely existing tickets addressing these issues.
> A 'testsuite' label exists. I'm not sure all the relevant tickets
> have the label applied, but it could be a good starting point:
>
>   https://github.com/qgis/QGIS/issues?q=is%3Aopen+is%3Aissue+label%3Atestsuite
>
> --strk;


Hi Sandro,

I don't think it is worth the effort, I'm pretty sure that "regular"
QGIS developers know how to run the tests locally (both python and
C++) directly into their IDEs, with debugger on if required, bypassing
ctest, and we need to focus on the library versions we support (see
the recent QEP:
https://github.com/qgis/QGIS-Enhancement-Proposals/issues/197)

The only thing that bugs me is that the control files for the
rendering tests  are often too different than the results I get
locally, I understand that there are difference sin DPIs, QT versions
and other machine dependent configurations and I know about the mask
files, but this is still a problem for me, and the only occasion in
which I fully rely on CI for the tests and I cannot trust my local
results.

Kind regards.

-- 
Alessandro Pasotti
QCooperative:  www.qcooperative.net
ItOpen:   www.itopen.it