[PROJ] "Trustworthiness" of vertical transformations

Wed Nov 27 09:25:36 PST 2019

This is a similar situation to that in New Zealand but on a much larger and more complex scale.  The recently established New Zealand Vertical Datum 2016 provides is based on an aerial gravity survey of the country providing a consistent datum and consistent level of accuracy across the entire country.  Prior to that there were 13 principal levelling datums each derived from levelling from a tide gauges.  Once the new datum was established we built a set of correction grids to convert between them - essentially these represent the local offset to the geopotential level surface representing mean sea level at each tide gauge, plus the systematic errors inherent in the levelling network.

The accuracy is much more complex to define - in many applications the local relative accuracy is much more important than the absolute accuracy.  The levelling datums were very accurate between adjacent benchmarks, but much less accurate in terms of absolute height above the geopotential surface.

Similarly when we transform a data set we may introduce an error to the data set as a whole, but the relative accuracy of data within the data set is very little changed.  It can become horribly comlicated very quickly!

Cheers
Chris
________________________________________
From: PROJ [proj-bounces at lists.osgeo.org] on behalf of Brian Shaw [brian.shaw at noaa.gov]
Sent: Thursday, 28 November 2019 5:54 a.m.
To: proj at lists.osgeo.org
Subject: Re: [PROJ] "Trustworthiness" of vertical transformations

Just a few things to note from my perspective but certainly not "official" from the US National Geodetic Survey (NGS).  Probably more information than you want to know but I believe it can be helpful to have a general skepticism in transformations and vertical transformations in particular.  I also apologize since this email got quite lengthy.

Vertical datums in the US have inherent errors that cause uncertainty in places due to systematic errors or incorrect assumptions.  Here is a little information about the historical US vertical datums and Ill note the future datum should be far more accurate since it will be derived primarily through terrestrial, aerial and satellite gravity measurements.

NGVD29 used 26 tide gauges across North America as constraints and made the assumption that sea level was the same in all places, which we now know is not the case.  This caused inherent errors in the middle of the country where data was made to "fit" through least squares adjustments causing inaccurate heights due to the assumption that all tide gauges were a height of 0.

NAVD88 tried to rectify this by only using one tide gauge as a constraint.  This caused the systematic errors in the leveling to propagate across the country.  Today people get NAVD88 heights by tying into bench marks that have NAVD88 heights on them which allows for them to get consistent heights in NAVD88 and is precise and provides repeatable heights, but precision is not accuracy as we are seeing while planning for the new datum in 2022.

Here at NGS we are working on a new datum and you can see the systematic errors from NAVD88 in this map<ftp://ftp.ngs.noaa.gov/dist/bshaw/Approximate_Vertical_Change_2022_sm.png> that shows the estimated change that will occur from NAVD88 to the new Geopotential Datum (Vertical Datum).  Its called "Geopotential" since it will be developed primarily through gravity and not hundreds of thousands of kilometers of leveling like past vertical datums.

A note on transformations in general is you should understand that almost all transformations are often termed mapping grade which is fine for GIS but not for geodetic coordinates.  You should also understand the uncertainties of the datums being used.  For instance if you are using WGS84 coordinates you should understand that just by using that datum you are introducing 3-5 meters of uncertainty in the horizontal coordinates, I'm not sure about vertically. Geodetically the best rule of thumb for having the most accurate coordinates is to preserve your original observations and process them in the datum of choice.  That is not always possible since you may not be the person/group collecting the data but it is something to keep in mind.

A closing note on accuracies as mentioned below by Kristian is that not all people report accuracies in the same way.  Even here at NGS there are different products that report in different measures. Some report at the one sigma level (standard deviation, square root of variance) also described as having 68% confidence in the value. Others report at the two sigma level (variance, std dev squared) which is often termed having 95% confidence in the value.  So I would encourage you to know what sigma is used to report the accuracy and also understand this may just be the accuracy estimated in the datum which doesnt necessarily account for the uncertainty in the datum itself.

Cheers
Brian

On 11/27/2019 3:18 AM, Kristian Evers wrote:

The algorithm currently is rather
simple: just add the (in)accuracies of each step. I do have some vague
remembering from univ that this was not always the "right" thing to do from a
math point of view. I guess sometimes perhaps taking the max() would be more
appropriate. But given that we can potentially mix uncomparable things,
probably not that a big deal...

Yeah, this can definitely be improved but it's a simple and effective solution that
doesn't promise too much which is always better that saying that accuracy is
better than it really is. I think this could be a fun topic to have a workshop on
at the OSGeo code sprint in the spring.

/Kristian

-----Original Message-----
From: Even Rouault <even.rouault at spatialys.com><mailto:even.rouault at spatialys.com>
Sent: 27. november 2019 01:06
To: proj at lists.osgeo.org<mailto:proj at lists.osgeo.org>
Cc: Nyall Dawson <nyall.dawson at gmail.com><mailto:nyall.dawson at gmail.com>; Kristian Evers <kreve at sdfe.dk><mailto:kreve at sdfe.dk>
Subject: Re: [PROJ] "Trustworthiness" of vertical transformations

On mercredi 27 novembre 2019 09:34:33 CET Nyall Dawson wrote:

On Tue, 26 Nov 2019 at 20:52, Kristian Evers <kreve at sdfe.dk><mailto:kreve at sdfe.dk> wrote:

You can get the transformation accuracy using the API function
proj_coordoperation_get_accuracy(). I think it would be cool if
information about transformation accuracy where readily available in
software like QGIS (nudge, nudge :-))

I should mention that the information about accuracy should be taken with a
grain of salt for several reasons:
- the EPSG guidance note 7-1 [1] underlines that the exact definition of
accuracy varies according to geodetic agencies
- in a number of situations, PROJ will have to synthetize the resulting
accuracy when chaining several steps. The algorithm currently is rather
simple: just add the (in)accuracies of each step. I do have some vague
remembering from univ that this was not always the "right" thing to do from a
math point of view. I guess sometimes perhaps taking the max() would be more
appropriate. But given that we can potentially mix uncomparable things,
probably not that a big deal...

That's where I'm coming from now... I'm just wondering if and how we
should expose vertical transformation functionality. Do you have any
ideas on what functionality you would expect an end-user application
to expose for vertical transformations?

Depends on the use cases. If you want to aggregate point cloud datasets that
are referenced against several vertical CRS into a single dataset, you will
probably want to convert them into a single Geographic CRS (ITRF2014, WGS84
G1762) or a single CompoundCRS appropriate for the area of study so that they
can be stashed together.

Even

[1] http://www.epsg.org/Portals/0/373-07-1.pdf , §6.5.4.1 Coordinate Operation
accuracy

--
*************************************
Brian Shaw
Geodesist
NOAA/NOS/National Geodetic Survey
Phone # 240-533-9522

________________________________

This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.