[Proj] Testing framework

[Mateusz Loskot]

On 29 May 2018 at 22:02, Kristian Evers <kreve at sdfe.dk> wrote:
> On 29 May 2018, at 21:14, Mateusz Loskot <mateusz at loskot.net> wrote:
>> On Tue, 29 May 2018, 20:37 Kristian Evers, <kreve at sdfe.dk> wrote:
>>>
>>> So in summation my proposed TODO list goes something like:
>>>
>>>     1. Move tests from src/ to the Catch2 framework
>>>     2. Move selftest remnants in gie.c to the Catch2 framework
>>>     3. Move shell-script tests in nad/ to test/
>>>     4. Add a test framework for the command line applications, also in
>>> test/
>>>
>>> Thoughts? What have I missed?
>>
>>
>> Although might be to early, but perhaps an outline of tests organization
>> into suites and cases.
>>
>> For example:
>>
>> - test suite per API function - a sort of unit kind of tests focused on
>> exercising single function (similar tests also could verify definitions of
>> data structures). Eg. Call foo(a, b) with valid, invalid, boundary, random,
>> etc. values for parameters.
>>
>> - test suites of more complex test cases, more complex Arrangements
>> preparing for actual test as well as more elaborate assertions following a
>> testing act - I like to think of those as functional tests (function is a
>> black box) or inter-function integration tests :) Eg.  Verifying conversion
>> of X from CRS A to B gives expected results, depending on input valid,
>> invalid, boundary…
>
> Your first point sounds like something we should do at least. We definitely
> need tests for each function in the library. The second point, if I understand
> correctly, is more or less covered by gie tests.

Yes, I think so.

Although, there are some cases where such Catch-based functional testing
may be a good idea. For example, verification that pj_init and
pj_init_plus for the
same definition result in equialent instance of projPJ; any kind of verification
of library/object states following interleaved/sequenced API calls;
any kind of round-trips verification with assertions for intermediate results
Such functional tests typically have more assertions per case than unit tests.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net