[QGIS-Developer] [Qgis-user] (how to) register gsb transformation in proj.db?

[Thomas Schüttenberg]

Hi all!

Because I better should not roll out QGIS with the afore mentioned premanent ballpark-warning in place, I investigatetd further and stumbled across an "allowFallback"-setting:

By placing a default datum transformation in the settings/options.../CRS you get new entries in your QGIS3.ini [find it on win at %APPDATA%\QGIS\QGIS3\], which are the operation parameters themselve and a setting to allow the approximate “ballpark” transformation [...] between a source and destination CRS pair, in the case that the preferred coordinate operation fails".

[Projections]
EPSG%3A31467\EPSG%3A25832_allowFallback=true
EPSG%3A31467\EPSG%3A25832_coordinateOp="+proj=pipeline +step +inv +proj=tmerc +lat_0=0 +lon_0=9 +k=1 +x_0=3500000 +y_0=0 +ellps=bessel +step +proj=hgridshift +grids=Minden.gsb +step +proj=utm +zone=32 +ellps=GRS80"

This is also writen to any project file containing layers with the crs', covered by the default datum transformation. It appears within in the tag <transformContext><srcDest [...] allowFallback="1"> and causes the warning to show up every time this project will be loaded, although the datum transformation is already working an no fallback seems necessary. 

If the _allowFallback=-setting in QGIS3.ini is (manually) edited to 'false', newly saved project files also contain <srcDest [...] allowFallback="0"> and the warning will not be displayed again on project load.

Reference to my quite hacky findings can be found in the QGIS Python API Doc, search for 'allowFallback': https://qgis.org/pyqgis/3.10/search.html?q=allowFallback
Some of the allowFallback stuff is labeled "New in version 3.12", however I currently investigate on 3.10.5-LTR. Might this indicate a backporting issue? (Otherwise, the implementation of my datum transformation very well might still be incomplete, don't know.)

Just wanted to inform you about my (current) workaround, any further hints are certainly appreciated.
Greetings!
Thomas

> Thomas Schüttenberg <thomas at qgis.de> hat am 04.06.2020 11:28 geschrieben:
> 
>  
> Hi Even,
> thank you! After the
> 
> INSERT INTO "grid_transformation" VALUES(
> 'PROJ','Minden','DHDN to ETRS89 (Minden)','DHDN to ETRS89 for City of Minden.','For applications requiring an accuracy of cm.','EPSG','9615','NTv2','EPSG','4313','EPSG','4258','EPSG','1625',NULL,'EPSG','8656','Latitude and longitude difference file','Minden.gsb',NULL,NULL,NULL,NULL,NULL,NULL,NULL,0);
> 
> (adapting the BWTA2017 record) and restarting QGIS the transformation appears in the Select Datum Transformations-dialog. It seems to work and the result is visibly more accurate than the BETA2007.
> 
> But I get a waring everytime the transformation is applied or loaded:
> 
> An alternative, ballpark-only transform was used when transforming coordinates between EPSG:31467 - DHDN / 3-degree Gauss-Kruger zone 3 and EPSG:25832 - ETRS89 / UTM zone 32N. The results may not match those obtained by using the preferred operation:
> +proj=pipeline +step +inv +proj=tmerc +lat_0=0 +lon_0=9 +k=1 +x_0=3500000 +y_0=0 +ellps=bessel +step +proj=hgridshift +grids=Minden.gsb +step +proj=utm +zone=32 +ellps=GRS80
> Possibly an incorrect choice of operation was made for transformations between these reference systems. Check the Project Properties and ensure that the selected transform operations are applicable over the whole extent of the current project.
> 
> (in German:
> Eine alternative ungenaue Transformaton wurde zur Koordinatentransformation zwischen EPSG:31467 - DHDN / 3-degree Gauss-Kruger zone 3 und EPSG:25832 - ETRS89 / UTM zone 32N. Die Ergebnisse könnten nicht mit denen der bevorzugten Operation übereinstimmen. 
> Möglicherweise wurde eine falsche Operation zwischen den Referenzsystemen gewählt. Bitte die Projekteigenschaften prüfen und sicherstellen das die gewählten Transformationsoperationen auf das ganze Projektausmaß anwendbar sind.)
> 
> Don't know what exactly this tries to tell me, but I think something is still missing and so far I was not able to find a correct "operation" to point to in proj.db.
> Or is it because of the area_of_use, which is far bigger than the grid extend? Or the missing accuracy? - Sarching for ballpark I found this issue #34983[1] but can't really say that I understand the comments.
> 
> [1] https://github.com/qgis/QGIS/issues/34983
> 
> Btw, certainly you are absolutely right regarding the publication with PROJ and registration of our gsb-file and therefore I contacted the authors for the nessesary informations...
> 
> Greetings
> Thomas
> 
> 
> > Even Rouault <even.rouault at spatialys.com> hat am 3. Juni 2020 15:37 geschrieben:
> > 
> > 
> > > Is it correct, that the tranformations in this dialog are read from
> > > \share\proj\proj.db and therefore this is the place to add my
> > > configuration?
> > Yes
> > > I found that there is a record for the (not shipped)
> > > BWTA2017.gsb. in table grid_transformation of proj.db. So, do I just have
> > > to add a record to grid_transformation for my gridfile in the same way?
> > Yes
> > > What would be the correct parameters? Is there another table that has to be
> > > adapted to make it work?
> > The critical parameters to set are the EPSG code of the source and target CRS. You must use the geographic CRS ones that correspond to the projected one you mentions. So EPSG:4314 (DHDN) as the source and EPSG:4258 (ETRS89) as the target. Starting from the record for BWTA2017 should be a good idea.
> > For the area of use, you may want to look at the "area" table for a code that fits the bounding box of the grid, or create a new one if there is no match
> > For the auth_name of the transformation use 'PROJ', and for the code give some name like 'EPSG_4314_TO_EPSG_4258_FOR_AREA_XXXX'
> > Utimately, if this grid is open data matching the criteria mentioned at https://github.com/OSGeo/PROJ-data, you may want to submit for inclusion here *and* also make sure the grid is registered in the EPSG database by contacting IOGP:
> > http://www.epsg.org/EPSGDataset/Makechangerequest.aspx
> > Once donce, that would avoid you in the future to have to hack the PROJ database.
> > Note: there are a number of other grids for Germany listed at
> > https://github.com/OSGeo/proj-datumgrid/issues that would be candidates for such a process.
> > Even
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com