[PROJ] "Fallback" support for transformations

Tue Feb 4 02:32:29 PST 2020

Hi Nyall,

For the Netherlands we use a better and (in my opinion) more geodetic sound approach, that gives "correct" results outside the grid. It also prevents getting completely different results between two nearby points if one point is just within and the other point just outside the grid.

What we did:

- We estimated 7 parameters for the "fallback" transformation between the Dutch CRS and ETRS89 as precise as possible.

- We created an official grid shift file for the residuals between the results of the fallback and real Dutch CRS, where we faded out the values to zero in a transition zone outside the Netherlands.

- We redefined the Dutch CRS as the result of the transformation from ETRS89 with these 7 parameters *and* grid shift.

- We created another grid shift file that is the sum of the 7 parameter transformation and the official grid shift file for a second variant of the transformation. 

As a result, the official transformation works for the entire world. The second variant of the transformation, which is easier to implement, gives the same results inside the grid as the official transformation. Outside the grid of the second variant, the fallback can be used, where it gives the same results as the official transformation. I would recommend this approach to all producers of grid shift files. Grid shift files that are not faded out to zero values at the bounds of the grid should not be used, in my opinion. 

So, I would recommend you to choose option 1 and only give results outside a grid for grid shift files that are faded out to zero values at the bounds of the grid. You can tell everyone that complains that a transformation doesn’t give results outside the grid that the cause is a flaw in the grid shift file and they should ask the grid shift files producer to use the Dutch approach instead.

Kind regards, Jochem

PS: The official transformation and the second variant of the transformation without 7 parameters transformation, can only give exactly the same results at a fixed height. Therefore, we decided to use a fixed hight (0 m in Dutch CRS) in the 7 parameter transformation. This isn't very elegant and doesn't comply with the ISO standard on CRS. We just do that as long as many software packages can't handle a two-step transformation with a 7 parameter transformation and a grid shift.

-----Original Message-----
From: PROJ <proj-bounces at lists.osgeo.org> On Behalf Of Nyall Dawson
Sent: dinsdag 4 februari 2020 08:12
To: PROJ <proj at lists.osgeo.org>
Subject: [PROJ] "Fallback" support for transformations

Hi list,

Looking for advice on an issue we are seeing reported a lot downstream at QGIS. Because proj 6 + the other associated changes in the stack have made use of grid shift files much more transparent and easier for users, we're seeing an associate uptake in users who are using coordinate operations which require a grid in their work (which is great!).

But... one downside of this is that operations based on a grid are much stricter with respect to area of use -- if you try to transform coordinates outside of the grid bounds, you (rightly) get null results. This is in contrast to purely mathematical transforms, where you still get... some... result outside of the area of use (albeit possibly meaningless). This manifests in issues like
https://github.com/qgis/QGIS/issues/33929 -- where a user is trying to use a grid based operation to transform features which fall outside the grid bounds.

I'm seeking advice on what the best approach is to take with this. I can see a few options:

1. Stand on the geodesic high-ground, and give null results -- the selected operation can't be used for those coordinates, so it's a user error. (the current approach in QGIS)

2. If a transform fails with the specified operation, fallback to using proj_create_crs_to_crs_from_pj to create a "backup" transform and let proj decide what to do. Benefits: users always get **some** result. Downside: users always get **some** result, which may or may not be using the operation they specified and could potentially be misleading or inaccurate.

3. As above, but show a user-visible warning that a "fallback"
transformation was used and the results are indicative only. Benefits:
users always get **some** result. Downside: user confusion due to a complex warning message, we run the risk of desensitising users to important transformation related warnings and advise.

4. As per 2, but make use of a fallback transform opt-in. When the selected operation fails, show a user-visible message giving them the option of using a non-preferred operation. Benefits: no potentially inaccurate results by default, user-friendly path to overcome the probem. Downside: as per 3

The other issue is that options 2-4 are not straightforward changes to make. In order to have the chance to transform using a "backup"
operation, we'd first need to make a temporary of all the original coordinates, so that if the preferred transform fails we have the original coordinates to use with the backup transform. We'd have to do that **always** when transforming, so there's an (unavoidable?) associated performance cost.

I'm at a bit of a loss which approach would be "best" to take. I'm personally leaning toward 3, but am not convinced. What's upstream's advice here?

Nyall
_______________________________________________
PROJ mailing list
PROJ at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/proj

Disclaimer:
De inhoud van dit bericht is uitsluitend bestemd voor geadresseerde.
Gebruik van de inhoud van dit bericht door anderen zonder toestemming van het Kadaster 
is onrechtmatig. Mocht dit bericht ten onrechte bij u terecht komen, dan verzoeken wij u 
dit direct te melden aan de verzender en het bericht te vernietigen. 
Aan de inhoud van dit bericht kunnen geen rechten worden ontleend.

Disclaimer:
The content of this message is meant to be received by the addressee only.
Use of the content of this message by anyone other than the addressee without the consent 
of the Kadaster is unlawful. If you have received this message, but are not the addressee, 
please contact the sender immediately and destroy the message.
No rights can be derived from the content of this message