[QGIS-Developer] QGIS and Proj 6

[Nyall Dawson]

Hi all,

Following the merge of https://github.com/qgis/QGIS/pull/30040 , I now
consider QGIS' port to the Proj 6 API complete. Here's a bit of a
summary of what's been done here (on builds based on proj 6.1 and
later only, more on that later[1])

- The Proj CRS database is now used to populate QGIS' internal CRS
database. We no longer use the combination of GDAL CSV files and
manual QGIS overrides we've used till now. This means that ALL
responsibility for CRS definitions and updating these sit were they
belong, upstream in the proj library. It also means we'll be an exact
match wrt projections as other tools which have completed the port,
such as GDAL 3.0. (All this goodness is thanks to the GDAL barn
raising effort). We still HAVE an internal projection database, in
order not to break existing 3.x API, so we'll need to revisit that at
4.0 and see if it can be safely removed then.

- We rely entirely on Proj 6's wonderful logic for generating the best
coordinate operation to transform between CRS pairs now. This means
(amongst other stuff), we correctly support complex things like
operations which require a "pivot datum", e.g. transformation to and
from GDA2020 coordinate systems. Again, this is thanks to the GDAL
barn raising effort... if it wasn't for that, we'd be... YEARS away
from even dreaming about supporting these correctly. Seriously, hats
off to the proj/GDAL teams here, they've done a brilliant effort, and
have been tirelessly answering tons of questions on their mailing
lists.

- Instead of the older approach QGIS used for datum transformations
(carrying around our own table of when grid shift files can be used),
we now use proj to determine these. This considerably changes the user
interface shown when a user has opted into selecting manually a
transform to use when multiple transforms exist, and we now show a
simplified list of available (and non-available) pathways.

- We also use proj's database to populate lists of available
ellipsoids for use in distance and area calculations. This has cleaned
up the ellipsoid choices considerably, and added many additional
ellipsoid definitions as a result. (woohoo... no more duplicate
"WGS84"/"WGS 84" entries!)

- The UX for notifying users about issues in coordinate transforms is
greatly improved, e.g. users are now alerted when a more accurate
transform is possible, but not usable on their system (e.g. due to
missing .gsb grid shift files). Wherever possible, we present users
with direct download links to obtain these required/desired grid shift
files. The intention here is to ensure users are informed when
transforms can be improved, instead of silently falling back to less
accurate options.

- Users also now have the option of placing grid shift files in "proj"
folder off their user profile. This means users can install grid shift
files and make them available in QGIS without requiring administrative
rights (possibly in future we could make QGIS fire up a background
task which automatically downloads and installs grids on demand...)

- We've also completed a project which began back in the lead-up to
3.0, which ensures that project-specific transformation pathway
settings are correctly respected EVERYWHERE a coordinate transform is
performed. This also means we're ready for the next stage in handling
4d coordinate transforms (when these start to land in 2020 and beyond)

As noted above, a lot of this is only possible thanks to improvements
in the proj and gdal libraries, which landed as a result of the GDAL
barn raising effort. On the QGIS side, it was ONLY possible thanks to
funding from the Australian ICSM. It's been a HUGE undertaking, so I'm
extremely grateful that the ICSM has had the foresight to invest into
QGIS and open source software development in this way.

Nyall

[1] Note that building 3.8 using the Proj 6 library requires a minimum
of Proj 6.1 -- it's not possible to build using Proj 6.0. There was an
API call added in 6.1 which we require, and I couldn't work around.
Sorry for any inconveniences this causes!