[PROJ] "Fallback" support for transformations

Wed Feb 5 02:01:57 PST 2020

Hi Joel,

> (1) have the grid define its own recommended fall-back transformation, 

I very much like your idea of having a grid define its own fallback method. A zero correction could be used as default. 

> + what distance would be appropriate / conventional for the fade to the fallback / null Tf?

If interpolation between grid and fallback is used to create a smooth transition, then the grid itself should also define the width of this transition zone. As default, the width of the transition zone could be based on de maximum grid shift at the outer grid bounds, e.g. if the largest grid shift at the outer grid bounds is 0.25 m, then a transition zone could be 25 km in width. This example is based on the idea that the grid gradient should not exceed 1 cm/km, which is the accuracy of the Dutch CRS itself. This gradient could also be a reasonable value for other grids with a maximum difference with the fallback of a few metres. I believe that in in general a fallback transformation should not differ more than a few metres. When the differences are larger, the fallback should probably be improved. 

> + what extent is appropriate for the fallback portion of the grid. (Did you say global?)

Whatever bound to the extent of the transformation you create, there will always be users that for some strange reason want to use the transformation outside that extent. When you use a 7 parameter transformation as fallback (like we did for the Netherlands), the extend of the fallback is global. Otherwise you would have to find a reasonable area, like the extend of the tectonic plate, or the extend of the EU for a European national CRS, or something much smaller like the extent of the national border plus a buffer of about 0.5 degree to the nearest integer degree. Note that often a map projection is used too for the CRS with the grid shift file and that the projection might not be able to map the entire world.

> + How to best define the transformation 'extent'?

Maybe a distinction should be made between the official extent for the use of the transformation, which should be the official extent of the CRS (e.g. within the national borders) and probably will be smaller than the area covered by the grid shift file, and the practical extent of the transformation discussed above.

> + I believe that a similar effect could be produced for any grid shift file by employing a sufficiently sparse 'parent grid' (within the same NTv2 gsb file) equivalent to the fallback. The grid interpolator would then correctly fade to the fallback between the original grid and the new.  That said, I'm not sure how many people actually use parent and child grids in their GSB.

Yes and no. Indeed, it is better to define the transition to the fallback exactly in the grid shift file itself. To model the effect of a 7 parameter transformation with a grid shift file, a sparse resolution is enough (at least locally). However, by using the sparse grid to do the transition too, the width of the transition zone is the same as the resolution of the sparse grid, which might be too small. For the Netherlands we use a sparse grid with a resolution of about 0.1 degree (corresponding to about 10 km) to model the 7 parameter transformation as accurate as we require the transformation to be (1 mm: one order of magnitude better that the relative precision of the national CRS), but to keep the gradient of the grid below 1 cm in the transition zone, a transition zone of 25 km is needed. Therefore, our transition zone is in the dense subgrid. Another constraint for gsb files is that the resolution of the sparse parent grid should be a multiple of the resolution of the dense subgrid and that a subgrid should cover the area of an integer number of parent grid cells.

Kind regards, Jochem

-----Original Message-----
From: PROJ <proj-bounces at lists.osgeo.org> On Behalf Of Joel Haasdyk
Sent: woensdag 5 februari 2020 05:01
To: Even Rouault <even.rouault at spatialys.com>; Nyall Dawson <nyall.dawson at gmail.com>
Cc: PROJ <proj at lists.osgeo.org>
Subject: Re: [PROJ] "Fallback" support for transformations

Option 4: 'make use of a fallback transform opt-in' 
is what I have heard expressed most often as the  desired functionality when discussing this issue in the past, and certainly gets my vote.

As already mentioned, Option 4:
    + offers an easy solution to users for the problem encountered,
    + replicates existing options/functionality in other software (such as FME) [which makes it familiar, not necessarily correct].
    + is not likely to return significantly different transformations between adjacent points,
       (if a reasonable backup is suggested and employed)...
       That said, it will still return a (small?) step at the grid extent...  [not many complaints received to date]
        A few ways around this could be to:
        (1) have the grid define its own recommended fall-back transformation, 
              and then have the grid extents equivalent to that fallback. 
        (2) have the software provide an interpolation for a defined distance away from the grid.

The Netherlands approach discussed is interesting indeed. A few questions remain before common adoption:
    + what distance would be appropriate / conventional for the fade to the fallback / null Tf?
    + what extent is appropriate for the fallback portion of the grid. (Did you say global?)
    + How to best define the transformation 'extent'? 
    + I believe that a similar effect could be produced for any grid shift file by employing a sufficiently sparse 'parent grid' (within the same NTv2 gsb file) equivalent to the fallback. The grid interpolator would then correctly fade to the fallback between the original grid and the new.  That said, I'm not sure how many people actually use parent and child grids in their GSB.

Joel Haasdyk | GDA2020 Implementation Program Manager (NSW)
--------------------------------------------------------------------------------------------------
Spatial Services | Department of Customer Service p (02) 87376322 | m 0427 229 589 e Joel.Haasdyk at customerservice.nsw.gov.au
w www.spatial.nsw.gov.au
Level 14 West, McKell Building, 2-24 Rawson Place, Sydney NSW 2000

https://www.icsm.gov.au/gda2020 (FAQs & Forum)
--------------------------------------------------------------------------------------------------

-----Original Message-----
From: PROJ <proj-bounces at lists.osgeo.org> On Behalf Of Even Rouault
Sent: Wednesday, 5 February 2020 10:09 AM
To: Nyall Dawson <nyall.dawson at gmail.com>
Cc: PROJ <proj at lists.osgeo.org>
Subject: Re: [PROJ] "Fallback" support for transformations

> While that approach could work for map rendering, it's not without 
> issues. E.g. a point layer dataset with some points outside the extent 
> of the grid -- when viewed at small scales you'll see all points, then 
> as you zoom in past a certain threshold you'll suddenly lose the 
> outside points when the grid shift kicks in. That's definitely going 
> to lead to user confusion. It also won't help with other 
> transformation tasks like layer exports, because we don't know the 
> intended scale for the output we'd always have to use the 
> user-specified grid based operation (and result in more bug reports:
> "i can see the points in the canvas, but when I save the layer I only 
> get some from the centre of my map in the saved file").

Hard/impossible to make everybody happy on that subject. I would probably suggest going to your 2. behaviour (always return something, which was pretty much the behaviour with PROJ < 6), being the default. With an advanced setting for advanced users that want maximum geodetic pedantism to go to mode 1.
The cases where the grid based transformation are - generally (see below note)
- in situations where points are outside of the area of use of the CRS (or are of use of the transformation), so you don't care a lot about accuracy of the transformation for those, because it is essentially unknown.

Side note:
There are a few cases also where the precise selection of a given transform / grid is not the ideal choice. For example (can't really think about another
one) for the NAD83 -> NAD83(HARN) transformation, the NADCON4 grids used by PROJ are per US state. So, with selection of a single transformation, if you want to transform a dataset covering more than one US state, you're stuck. 
That's a annoyance that the data producer realized, and NADCON5 grids are for the whole conterminous US (but those are not integrated in PROJ for now)

Even

--
Spatialys - Geospatial professional services https://clicktime.symantec.com/37K99JRAW4KNFbsYyyR4m177Vc?u=http%3A%2F%2Fwww.spatialys.com
_______________________________________________
PROJ mailing list
PROJ at lists.osgeo.org
https://clicktime.symantec.com/32H699CiEohhsWYWncgFZtW7Vc?u=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fproj

**********************************************************************************
This email message and any attached files is confidential and intended solely for the use of the individual or entity to whom it is addressed and may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you have received this email in error, delete all copies and notify the sender.

This email is subject to copyright. No part of it should be reproduced, published, communicated or adapted without the copyright owner's written consent. No employee or agent is authorised to conclude any binding agreement on behalf of the Department of Customer Service (DCS) by email without express written confirmation.

The views or opinions presented in this email are solely those of the author and do not necessarily represent those of the DCS. DCS accepts no liability for any loss or damage arising from the use of this email and the recipient should check this email and any attached files for the presence of viruses.

**********************************************************************************
_______________________________________________
PROJ mailing list
PROJ at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/proj

Disclaimer:
De inhoud van dit bericht is uitsluitend bestemd voor geadresseerde.
Gebruik van de inhoud van dit bericht door anderen zonder toestemming van het Kadaster 
is onrechtmatig. Mocht dit bericht ten onrechte bij u terecht komen, dan verzoeken wij u 
dit direct te melden aan de verzender en het bericht te vernietigen. 
Aan de inhoud van dit bericht kunnen geen rechten worden ontleend.

Disclaimer:
The content of this message is meant to be received by the addressee only.
Use of the content of this message by anyone other than the addressee without the consent 
of the Kadaster is unlawful. If you have received this message, but are not the addressee, 
please contact the sender immediately and destroy the message.
No rights can be derived from the content of this message