[gdal-dev] Call for discussion on RFC 72: Update autotest suite to use pytest

[Craig de Stigter]

Thanks Sean & Even

> I'm still concerned about using the SWIG bindings to test the C API

Fair enough, though as you noted, this is a bit tangential; RFC72 doesn't
make any changes to this situation, it's merely a refactor of the Python
tests.

> It uses a lot of magic, which some Python developers don't like, and so
it isn't necessarily *the way* to test in Python

That's true. Some people don't like Django for the same reason; a lot of
its internals are full of metaclasses and descriptors. SQLAlchemy is in the
same boat. Magic can be difficult to grok, and difficult to maintain.
However I think in all three cases they've acquired a significant user base
for the simple reason that the internal magic enables a really clean,
simple and effective interface.

I don't know if I'd want to *develop* pytest; I've seen some of the code
and I think it would take me a long time to understand it well enough to
work on it. But I do think the interface it presents is really excellent.

> The example shown in the RFC, for example: is making a fixture for every
gdal-bin program or adding checker fixtures that make multiple asserts
patterns that will help or hurt long term

Note that the example given in the RFC is just one possible future. I'd
hope those questions would be answered in pull requests as they happen. For
instance, the `require_files` fixture in the example doesn't exist yet. I'm
not committed to making those particular changes if there's opposition to
them.

That said, the advantages of a `require_files` fixture at least seem
reasonably clear to me: it would make the requirements of each test more
explicit and reduce boilerplate. I think this would also allow us to run
tests in parallel without them interfering with each other. The fixture
would put just the required files in a temp directory for each test, and
remove it afterwards. Being able to parallelise the tests would be a huge
win.

For what's actually implemented in the phase 1 pull request, see the
conftest.py
<https://github.com/craigds/gdal/blob/pytestify/autotest/conftest.py> file.
There's not much there beyond what's required for compatibility with the
old test runner. Notably, there's a
`@pytest.mark.require_driver(drivername)` marker, which I've applied to
some of the tests I had to change manually.

Cheers
Craig

On Wed, 5 Dec 2018 at 06:52 Sean Gillies <sean at mapbox.com> wrote:

> Craig,
>
> Thanks for doing this huge lift. Not all heroes wear capes!
>
> I'm still concerned about using the SWIG bindings to test the C API
> because subtle defects in the wrappers could hide deeper bugs and wish
> there were more direct tests, but RFC 72 is a big improvement.
>
> I use pytest in most of my projects and like how it reduces the friction
> for writing tests. It uses a lot of magic, which some Python developers
> don't like, and so it isn't necessarily *the way* to test in Python. I
> think there is a small risk that the hard to understand patterns of the
> home-grown approach could be replaced by hard to understand fixtures in a
> pytest approach. The example shown in the RFC, for example: is making a
> fixture for every gdal-bin program or adding checker fixtures that make
> multiple asserts patterns that will help or hurt long term? I'm not super
> concerned, just want to point out that pytest isn't a panacea and doesn't
> default to the golden path of testing.
>
> Thanks again!
>
> On Mon, Dec 3, 2018 at 4:49 PM Craig de Stigter <
> craig.destigter at koordinates.com> wrote:
>
>> Hi,
>>
>> This is a call for discussion on RFC 72: Update autotest suite to use
>> pytest
>>
>> https://trac.osgeo.org/gdal/wiki/rfc72_pytest
>>
>> Summary:
>>
>> The document proposes and describes conversion of the existing Python
>> autotest suite to use the ​pytest framework.
>>
>> Using pytest provides significant productivity gains for writing, reading
>> and debugging python tests, compared with the current home-grown approach.
>>
>> --
>>
>> Please submit feedback soon, as we'd like to have this merged in time for
>> the imminent GDAL 2.4.0 RC if possible.
>>
>> Thanks
>> Craig de Stigter
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> --
> Sean Gillies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20181205/266a2ee3/attachment.html>