[Qgis-developer] New test for python binding coverage

[Nyall Dawson]

Hi all,

Recently I pushed a new unit test to master which is designed to test
coverage of the python sip bindings. I thought it's probably a good
idea to give a quick run-down of how this test works, and what it
means for you.

First, the motivation for this new test is to prevent new classes or
methods being added to the public API which aren't available in the
python bindings. It's not perfect, as various limitations mean that
I've only be able to test whether a particular member exists within
the bindings, and can't test for matching signatures/overloads (seems
this isn't possible in python).

The test uses the doxygen API docs to determine what public api should
be available in the python bindings. So if it's not in the docs, it
won't be tested. Just like the doxygen coverage test, there's a
hardcoded limit for the number of members which are missing bindings.
If your commit causes this threshold to be exceeded, the test will
fail.

To avoid this test tripping, you can:
1. Make sure any public/protected methods or public members you add
are also added to the python bindings.

2. If this isn't possible, the test will skip any classes which have a :
"@note not available in Python bindings"
doxygen note for the class. So if you add a class to core/gui/...
which ISN'T intended to be public API, either exclude it from the docs
with the doxygen
/// @cond
...
/// @endcond
directives, or tag the class with the @note

3. For individual members/methods, the test will skip any with a
doxygen note containing the word "python". So, notes of the form:
"@note not available in python bindings"
"@note available as getStuffV2 in python api"
will be skipped. (The thought here is that it indicates that some
consideration has been given to the python bindings and it's not just
an intentional omission).

If the test fails, a full list of the classes/members missing from the
bindings will be available in the test report on cdash
(http://dash.orfeo-toolbox.org/index.php?project=QGIS). This will help
identify which added members have tripped the test.

Like I said at the start, the test is far from perfect/infallible, but
at least it's a quick indicator/reminder to add stuff to the sip
bindings before submitting commits/Pull Requests, and a good
indication to reviewers if these haven't been added yet.

Nyall