[PROJ] Question about performances of proj_trans or proj_trans_generic when using a grid shift file

[Stephan Hügel]

Depending on your use case, it might be worthwhile to convert the grid shift file in question into a key-value structure such as a hashmap (you can use a perfect hash function for this, since all the values are known ahead of time), and serialise that to disk. Your conversion routine then has two steps: convert using PROJ without the grid shift step to obtain the uncorrected new coordinates, then look up the shift values for those coordinates in the hash map, and perform the bilinear interpolation manually to obtain the corrected coordinates.

There are drawbacks: the serialised hashmap will be quite large on disk (less than 100 MB, I would imagine), and you’re adding complexity to your conversion pipeline, but it may be worth it for you if you have a need to convert coordinates at scale on a regular basis, such that a 30x slowdown is too large. I’ve found it to be a worthwhile approach for very fast conversion from WGS84 to OSGB36, using the OSTN15 transform: https://github.com/urschrei/lonlat_bng, but in retrospect it’s difficult to judge whether it was worthwhile from a cost / benefit perspective.

-- 
steph

> On 2 Jul 2021, at 16:13, Even Rouault <even.rouault at spatialys.com> wrote:
> 
> Kilian,
> 
> looking at the NTv2 file reader, I see it will do a seek + read for each points to transform (actually 4 seek + read, given the bilinear interpolation). I suspect that might be costly.
> 
> The GTiff grid file reader keeps in memory the last used strip or tile, so if transformating locations that are sufficiently close from each other, the amont of file read is reduced. You could try to use the https://github.com/OSGeo/PROJ-data/blob/master/fr_ign/fr_ign_ntf_r93.tif GeoTIFF grid (also available at http://cdn.proj.org/fr_ign_ntf_r93.tif), which is a conversion from the NTv2 ntf_r93.gsb (and in that instance, it is a single block grid, so it will be just read once)
> 
> And your analysis is write: proj_trans_generic() will not give you speed-up (at least with current implementation) compared to repeated calls to proj_trans()
> 
> Even
> 
> Le 02/07/2021 à 15:44, Kilian Demeulemeester a écrit :
>> Hi, 
>> 
>> I've added the support of grid shift in the software suite we are developing here at YellowScan. 
>> 
>> Now that the grid shift files are found, the coordinates are projected correctly and our customer is happy. 
>> 
>> However, the projection time completly exploded: for instance, on our customer dataset, our LAS creation algorithm exports our data in approx 5s without using a grid shift, and it now takes up to 2min20s when using it. 
>> 
>> The projection pipeline used is: 
>> 
>> +proj=pipeline
>> +step +proj=push +v_3
>> 
>> +step +proj=cart +ellps=WGS84
>> 
>> +step +proj=helmert +x=168 +y=60 +z=-320
>> 
>> +step +inv +proj=cart +ellps=clrk80ign
>> 
>> +step +proj=pop +v_3
>> 
>> +step +proj=lcc +lat_1=44.1 +lat_0=44.1 +lon_0=0 +k_0=0.999877499 +x_0=600000 +y_0=3200000 +ellps=clrk80ign +pm=paris
>> 
>> +step +inv +proj=lcc +lat_1=44.1 +lat_0=44.1 +lon_0=0 +k_0=0.999877499 +x_0=600000 +y_0=3200000 +ellps=clrk80ign +pm=paris
>> 
>> +step +proj=hgridshift +grids=ntf_r93.gsb
>> 
>> +step +proj=lcc +lat_0=46.5 +lon_0=3 +lat_1=49 +lat_2=44 +x_0=700000 +y_0=6600000 +ellps=GRS80
>> 
>> 
>> 
>> The grid shift file used is ntf_r93.gsb
>> 
>> 
>> 
>> Is this time normal? Is there a way to speed things up? I've tried to use proj_trans_generic with no success, and exploring the source code on github, I realized that it is using a for-loop approach, using proj_trans (so no optimization to expect - but I might be wrong).
>> 
>> 
>> Any help is appreciated! 
>> 
>> Cheers ! 
>> 
>> Kilian Demeulemeester
>> Software Project Lead - R&D
>> 
>> <x-disclaimer-534644925-0.png>
>> 525 Avenue Saint Sauveur du Pin
>> 34980 Saint-Clément-De-Rivière - France
>> Tel: +33 (0)4 11 93 14 06
>> yellowscan-lidar.com
>> 
>> 
>> 
>> 
>> _______________________________________________
>> PROJ mailing list
>> PROJ at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/proj
> -- 
> http://www.spatialys.com
> My software is free, but my time generally not.
> _______________________________________________
> PROJ mailing list
> PROJ at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/proj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/proj/attachments/20210702/23c596d1/attachment-0001.html>