[gdal-dev] autotest questions/issues
robert.coup at koordinates.com
Wed Sep 1 00:29:42 PDT 2021
On Wed, 1 Sept 2021 at 01:33, Greg Troxel <gdt at lexort.com> wrote:
> A few comments from the perspective of a gdal test newbie, which I hope
> are helpful:
> * autotest is not in the distribution tarball
Yeah, it's never been in the release tarballs. And the release tarballs
start one folder level down (gdal/) as well.
TBH I think the release tarballs should just match/be an archive of the git
tag. But that's above my pay grade :-)
@Debian and other package maintainers — are you just getting the Git tags
these days or do you actually use the release tarballs?
> + Does it test the gdal that is installed in one's PATH (and
> PYTHONPATH), or is is testing the gdal that is in the source tree?
CONTRIBUTING.md has a guide to run `scripts/setdevenv.sh` which sets up
in-tree execution for utilities/libraries and the tests.
> + Is one supposed to check out the corresponding branch (or really
> tag), or is it ok to test 3.3.2rc2 with master?
The tests should match the code, so 3.3.2rc2 tests run against 3.3.2rc2
> * no clear path to test a not-yet-installed gdal
> When building a new gdal, it seems a good idea to run tests before
> installing it. autoconf has done this for year with make check. I see
> no obvious way to do this.
setdevenv.sh? Adding some sort of `make test` might be a cleaner approach
> * puzzling test dependencies
> It's not really clear if a missing dependency (vs requirements.txt)
> will cause problems (assuming pytest and lxml are there), in terms of
> just skipping some things, bad output, or perhaps some missing pretty
a mixture. Is there some particular reason installing everything in
requirements.txt from PyPi is a problem?
> I think pytest-env is
> but that appears unmaintained, with a last release in 2017. Perhaps
> there are no outstanding bugs and no changes and it is actually
> maintained but there is no reason to change -- but that is highly
> implausible. It seems boutique, based on not being in pkgsrc, which
> means no one else has needed it, even though we have 20K packages.
It's about 45 lines of code. Pytest plugins are often fairly minimal, and
just do One Thing.
> [Now I see that osr tests fail with a warning about env if this is
> missing, but really the test should be skipped or hard fail and not
> run without the env.]
Yes, I think it should hard-fail
> * I'm guessing the expectation is that all the tests pass. With
> autotest from master and 3.3.2rc2 installed (base lib and python
> bindings), I get:
In general all the tests pass. There's skips/xfails for particular known
issues, but maybe you've found a bug? Maybe BSD-specific, BSD isn't part of
> * skipping tests after a failure.
> It would be nice to have hints in README.md about skipping tests, but it
> seems easy enough to run osr and then the following ones.
you mean, stopping on the first failure? `pytest -x ...`
autotest/README.md has some docs on running specific tests / modules / etc
GDAL has some odd test layouts with particular inter-test dependencies from
when the test suite was bulk-ported via automation to work under pytest —
this made it a lot saner, but some of the "test 18 depends on test 17
passing" issues remain.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the gdal-dev