[PROJ] PROJ 6 testntv2 fails

[Greg Troxel]

Markus Metz <markus.metz.giswork at gmail.com> writes:

> On Wed, Mar 6, 2019 at 6:57 PM Greg Troxel <gdt at lexort.com> wrote:
>>
>> Given that the grid files are intended to get better results, it would
>> be really nice if the tests passed with them, either via fuzz (which
>> makes sense if both answers are correct) or via having two recorded
>> acceptable values.
>
> I guess the tests are designed to test the numerical stability of the proj
> code, for given input data. Therefore it does not matter how accurate the
> input data are, what matters is that for the expected input data the
> expected result must be provided. If the input data are not as expected,
> the test fails, obviously.

yes, but

> The output line
> -->
> Try with NTv2 and NTv1 together ... falls back to NTv1
> <--
> looks suspicious, also causing the test failure.
>
> README:
> "The test suite requires that the proj-datumgrid package is installed
> in `PROJ_LIB`."
>
> This should be expanded to note which datumgrid packages should *not* be
> installed for the tests, or that only the package proj-datumgrid-<X>.<Y>
> should be installed, but no other datumgrid packages.

This is what's awkward.  Various packaging systems will have different
approaches.  One is to include all the grids (space is cheap, simple),
another is to make them separate packages, and another is to not package
them at all.  Telling people that you can't run proj tests if the other
package is installed, when really you should have it installed, just
makes it hard and will make the tests be ignored.

Another approach would be to cause proj not to load the extra grids when
running tests via some control flag or environment variable.

I think the test issue is actually showing a real issue, which is that
with grids installed or not, proj on a particular input produces
different results.  But we discussed that, and the answer was that both
results are right, just that one is more accurate, and the fact that
people running the same conversion on the same input will get different
results is not concerning.