[gdal-dev] GDAL 3, PROJ 6.2 exportToProj4() drops Datum silently

[Nyall Dawson]

On Thu, 7 Nov 2019 at 20:05, Roger Bivand <Roger.Bivand at nhh.no> wrote:

> Thanks for contributing! In research reproducibility, being able now to
> get the same results as when the research was conducted using the same
> software is basic.

(again, apologies if I'm ignorant here -- I'm coming from a totally
different discipline and background).

If studies were made using a software version with a bug that was
later fixed, in that case you definitely wouldn't want to reproduce
earlier results, right? E.g. the recent Willoughby-Hoye script bug
which potentially invalidated ~150 studies
(https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/).
Admittedly the situation with proj is much more complex than just a
"bug in the software"!

My gut instinct is that if I was relying on any results generated
using proj 4's logic, I'd much rather have someone re-run the data
using proj 6's refined handling (including requirin any requisite
modifications to the scripts to shift from proj strings to auth/id
codes or wkt definitions), and compare these new results with the
earlier results in order to **justifiably** verify that different
handling of datum shifts and reprojection did NOT have any significant
impact on the results.

Nyall


 However, when the "same software" - including calls to
> metadata like EPSG descriptions - is no longer the "same", we need to be
> careful. I agree that wanting to flag and accept situations in which
> subsequent versions of apparently the same software yield different and
> more "precise" outcomes is reasonable.
>
> A specific issue with GDAL 3 using PROJ 6 is that exportToProj4() drops
> keys like +datum=. In R packages, Proj4 strings have been used since the
> beginning (early 2000s) to attach CRS information to spatial objects. When
> (justifiably) +datum= is deprecated, users suddenly and without warning go
> from legacy precision to ballpark only, so worse by two orders of
> magnitude on transformation (because the source object CRS description is
> degraded).
>
> We are now scrambling to add WKT2_2018 as a string representation on read
> and for write, because unlike exportToProj4(), exportToWkt() does preserve
> datum specifications. For those interested, the rgdal package on R-Forge
> (development, unstable) now piggy-backs WKT2 as a comment on sp CRS
> objects for reading or writing vector and raster files, using WKT2 rather
> than Proj4 strings if available. Next steps are to adapt coordinate
> operations to use WKT2 CRS descriptions rather than Proj4 strings for
> coordinate operations, including adding choices of pipelines and
> on-the-fly downloading of grids further ahead.
>
> >
> > If a particular study isn't affected by relatively small shifts such as
> > the 1.5m shift in the GDA2020 situation, then the results should remain
> > reproducible even if this shift is corrected. But if a particular study
> > IS affected by distances of this magnitude then I'd argue that it would
> > be vital that the older proj behavior can't be reproduced exactly, and
> > accordingly the previous study/results are invalidated as a direct
> > result..!
>
> The negative outcomes are rather because say +datum= did a reasonable job
> before GDAL 3, but with GDAL 3 and the deletion of tags (+datum= is gone,
> +towgs84= and +init= are vulnerable and should be avoided), outcomes in
> GDAL3/PROJ6 are much worse (justifiably so) than GDAL2/PROJ5.
>
> I think adaptation of R packages to modern PROJ/GDAL is feasible, and
> enough progress has been made to make me a bit more comfortable. See also
> coments by Markus Metz for GRASS and Mike Sumner (R, special interest in
> the Antarctic); I've benefitted also from reading the Cython code in
> pyproj (Alan Snow has posted here too - thanks!).
>
> Roger
>
> >
> > Nyall
> >
> >
> >>
> >> --
> >> Spatialys - Geospatial professional services
> >> http://www.spatialys.com
> >> _______________________________________________
> >> gdal-dev mailing list
> >> gdal-dev at lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en