[GRASS-git] [OSGeo/grass-addons] 5a189a: CI(tests): Surface gunittest failures instead of t...

[Edouard Choinière]

  Branch: refs/heads/grass8
  Home:   https://github.com/OSGeo/grass-addons
  Commit: 5a189aa7e61e459174e3c122c305ca0b7883c462
      https://github.com/OSGeo/grass-addons/commit/5a189aa7e61e459174e3c122c305ca0b7883c462
  Author: Edouard Choinière <27212526+echoix at users.noreply.github.com>
  Date:   2026-09-17 (Thu, 17 Sep 2026)

  Changed paths:
    M .github/workflows/apt.txt
    M .github/workflows/test.sh
    M .gunittest.cfg
    M AGENTS.md
    M pyproject.toml
    M src/general/g.citation/testsuite/test_g_citation.py
    M src/raster/r.cell.area/testsuite/test_r_cell_area.py
    M src/raster/r.forestfrag/testsuite/r_forestfrag_trivial.py
    M src/raster/r.green/r.green.gshp/libgshp/testsuite/test_gpot.py
    M src/raster3d/r3.forestfrag/testsuite/r3_forestfrag_trivial.py
    M src/temporal/t.rast.what.aggr/testsuite/test_whataggr.py
    M src/vector/v.flexure/testsuite/test_v_flexure.py
    M src/vector/v.stream.order/testsuite/test_stream_order.py
    M src/vector/v.to.rast.multi/testsuite/test_rasterize_multi.py

  Log Message:
  -----------
  CI(tests): Surface gunittest failures instead of tolerating them (#1851)

* r.green.gshp: Fix gpot import and data path in testsuite

test_gpot.py imports gpot directly, but running it as a standalone
script only adds its own directory (testsuite/) to sys.path, not the
parent libgshp/ directory where gpot.py lives, so the import always
failed with ModuleNotFoundError.

TestRasterGPot also read its input data from testsuite/data/, but
gunittest already copies that directory to the test's own working
directory as data/, so the path needs to be relative to that, not to
testsuite/.

* v.stream.order: Install imagemagick so generate_manpage_images.sh works in CI

The testsuite script calls convert to render the manual page images
and was failing every CI run because imagemagick was not installed.

* CI: Mark currently failing gunittest tests as expected failures

The gunittest run was tolerated up to 19% file failures via
--min-success, which hid real regressions in the addon that was
actually changed by a PR. Before that tolerance can be removed, every
test that is currently and consistently failing on main needs to be
accounted for.

Mark each with @unittest.expectedFailure and a short comment on the
observed cause, so a currently red CI run turns green here, and any
newly introduced failure (or an unexpected pass, reported separately)
stands out once --min-success is dropped.

See https://github.com/OSGeo/grass-addons/issues/1850.

* CI: Require all gunittest tests to pass

Bump the gunittest --min-success threshold to the default of 100% (by
dropping the flag entirely) now that every currently failing test is
either fixed, marked as an expected failure, or excluded below. This
turns CI red the next time an addon's tests actually regress, instead
of silently tolerating up to 19% of test files failing.

Three tests cannot usefully be marked as expected failures with
unittest.expectedFailure, so exclude them from the run instead:

- v.in.pygbif's test queries the live GBIF API, so its outcome depends
on network access and on data that changes over time.
- v.surf.rst.cv's cross-validation test consistently exceeds the
360 second per-file timeout, which gunittest reports as the whole
file failing rather than as a test failure or error that
expectedFailure could catch.
- t.rast.import.netcdf's Sentinel-2 test downloads a NetCDF file over
the network, which can also run past the per-file timeout.

See https://github.com/OSGeo/grass-addons/issues/1850.

* CI: Fail pytest tests whose xfail unexpectedly passes

xfail_strict was unset, so pytest's default lenient xfail let a test
marked xfail pass silently instead of failing the run, unlike
unittest.expectedFailure (used for the gunittest xfails added in the
previous commits), which always fails on an unexpected pass. Set
xfail_strict globally so any future pytest.mark.xfail behaves the same
way: an unexpected pass in any CI matrix job turns the check red
instead of being tolerated.

No existing test currently uses pytest.mark.xfail, so this changes
no test outcomes today.



To unsubscribe from these emails, change your notification settings at https://github.com/OSGeo/grass-addons/settings/notifications