[Qgis-developer] QGIS Server unit testing for labeling (was: September 1 branch for release)

[Larry Shaffer]

Hi Andreas,

On Mon, Sep 2, 2013 at 8:33 AM, Andreas Neumann <a.neumann at carto.net> wrote:

> Hi Larry,
>
> > I would love to know how that is possible. I've not seen a setup that
> > doesn't exhibit the issue yet. Maybe it slight enough that you just don't
> > 'see' it on manual visual inspection. Can you possibly run the current
> > limited labeling server test from the build dir on that machine (or one
> > with access to that web server)?
> >
> > It is run individually with: ctest -R PyQgsPalLabelingServer -V
>
> I was trying to run the tests but am missing the test-projects - where
> can I get them or where are they located?
>

Thank you for trying to setup the labeling server test.

When the tests for the server are run, the projects are generated
on-the-fly and written to somewhere the server can read (see below). Also
copied into that directory are the layer source data (small spatialite db)
and predefined styles [0].


> I changes $HOME/.qgis2/qgis_local_server.cfg but don't know where I
> should point the "projdir" variable to.
>

Right now it is a bit cumbersome because it relies upon configuring access
to an existing, compatible QGIS Server setup.

This is from the error message for an unconfigured setup [1]:
  projdir = path WRITEABLE by this user and READABLE by www server

Once that's set up, you can test compatibility by manually populating the
projdir with contents of [0] and editing under __main__ and running the
following script directly (see recent commit [2]):
  python qgis_local_server.py

A successful test of connection to configured test server should produce
the following PNG [3]. This particular manual connection test is not (will
not be) part of the QGIS ctest suite.

With this setup, you should ideally be testing against the
qgis_mapserv.fcgi binary produced as part of building the desktop, though
you can of course test against any existing fcgi process over a bound
TCP/IP address. While there are good reasons to test against a production
server, there are reasons definitely not to:

* Recent changes to the qgis_mapserv.fgi code will not be reflected.

* The test suite may move to spawning/restarting the fcgi binary often (to
keep server's cache from affecting test output).

* Some settings (labeling engine debug settings, e.g. candidate rectangles)
may persist on a long running fcgi process. Since restarting many servers
involves root access and affects up-time, such automated tests should be
relegated to a sandboxed, configured test server setup.

So, you can probably see why an automated, embedded and isolated spawn-fcgi
utility, running under the user id of the test runner, greatly facilitates
automated tests. (Should be committed in coming weeks.) I'll also look at
posting my successful configurations for setting up QGIS Server under
Nginx, which makes a nice sandbox setup (even alongside Apache), for
testing qgisweb-client and wsgi as well.

Lastly, your server tests against the control images I made on my Mac will
no doubt fail. Currently, multi-platform control images, to account for
font handing differences, are not supported. You will probably want to set
the env var PAL_CONTROL_IMAGE=1 and run:
  ctest -R PyQgsPalLabelingCanvas -V && unset PAL_CONTROL_IMAGE

This will overwrite the canvas test output control images, so that you have
a realistic comparison against the TestServerVsCanvas* classes' output,
similar to loading your test server as a WMS layer in the desktop app. Just
please do not commit your control images yet (unless as anomalies).

Alternatively, you can set the env var PAL_REPORT=1 and run `ctest -R
PyQgsPalLabelingCanvas -V`. This should open any failed tests in your
browser where you can choose to save your rendered images to the
appropriate canvas control images directories, naming them as anomalies.

If you have even read this far  :^)  and are going through this setup,
thank you very much. While it will be good to use an automated spawn-fcgi
setup, there are definite benefits for testing against real-world web
server setups, especially those managed by experienced users.


IMHO, until a test framework and a suite of many tests are finished,
complex changes to labeling code should be ill-advised, and probably be
postponed.

While it would be good to get funding from the community to complete this
type of work, I understand how unglamorous this is compared with new
features.

[0] https://github.com/qgis/Quantum-GIS/tree/master/tests/testdata/labeling
[1]
https://github.com/qgis/Quantum-GIS/blob/master/tests/src/python/qgis_local_server.py#L192
[2] https://github.com/qgis/Quantum-GIS/commit/ff05ee06
[3] http://drive.dakotacarto.com/qgis/test-connection.png


Regards,

Larry

Thanks,
> Andreas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20130902/81c3de9b/attachment.html>