[gdal-dev] The Not Rocket Science Rule Of Software Engineering

[Greg Troxel]

Andrew C Aitchison <andrew at aitchison.me.uk> writes:

> I have been reading The Not Rocket Science Rule Of Software Engineering
> https://graydon2.dreamwidth.org/1597.html
> which is:
>        automatically maintain a repository of code
>        that always passes all the tests
>
> Does https://github.com/OSGeo/gdal have a branch with this property ?
>
> If we do not yet, it should in principal be easy:
> whenever HEAD of master passes all test, pull that to a current-working branch
> (in practice we have 16 top-level tests, so would need a wrapper which says
> whether they all passed or not).

In principle it's easy.  In the real world, "passes all tests" is
woefully underspecified because there are many of:

 operating systems
 operating system versions
 CPU types

 versions of each dependency (which themselves may or may not "pass all tests")

 whether or not grids are installed for proj
 
 probably something else I haven't thought of.

In the general case, CI schemes cover a small fraction of the above.

> A comment on the above blog post is:
>   Aegis has a delightful wrinkle on this: each changeset has to not just
>   pass all existing tests, but also come with a new test, independently
>   tracked and associated with the change, that must fail before the
>   change, and pass after it.
>
> That would be a useful addition too.

It would lead to bugs not getting fixed because it becomes too much
effort.  Or maybe you're volunteering to write the tests for all the
one-line bugfixes?

And, it runs into the same issue: I recently submitted a PR -- which was
merged -- that improved portability of mkgdaldist.sh.  I don't think
there were tests for that, and if there were CI doesn't happen on NetBSD
anyway.  Surely you don't mean it is reasonable to block such PRs on
lack of testing.

I do think that for changes that fix a real bug, it is nicer to have a
new test case than not, and that changes shouldn't be merged if they
cause test regressions on the set of systems CI runs on, plus reports.

And if there is a bug, it's nice to add a new xfail test case to the
tests, even before the fix.


From watching gdal (and many other things as a packager), gdal is in
very good shape this way.  Tests seem to essentially always pass on
GNU/Linux, and I have found some failures on NetBSD, now almost entirely
resolved (a blend of test environment documentation and fixes to gdal).
So I think things are actually in quite good shape, and the biggest
issues are the proj grid-present issue and lack of official CI on
minority platforms.  Minority platform CI is important to everyone,
because it tends to shake out bugs that remain latent on the platform
most development happens, because of course bugs that manifest there are
already fixed.



I really don't mean to argue against "tests are a great thing and should
exist" at all.  Just that I think it's harder than it seems, and that
gdal is already doing a great job.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220329/4fb1159d/attachment.sig>