[PROJ] RFC4: last chance to comment before motion

[Nyall Dawson]

On Fri, 3 Jan 2020 at 22:24, Even Rouault <even.rouault at spatialys.com> wrote:
>
> Hi Nyall,
>
> > My understanding of the current API leads me to believe that this
> > wouldn't be possible in the current RFC4 API. The missing bits would
> > be:
> >
> > 1. Public api to query if the preferred operation between two crses
> > requires a grid download for a certain area
> > 2. (Thread safe) API to manually download a grid for a certain area
> >
> > What's your thoughts on this?
>
> I appreciate your suggestions but I can see some difficulties:
>
> - "requires a grid download for a certain area". Imagine that the case is
>   to reproject the 4 corners of Continental US. You just need (part of) the
>   grids at those points, not for the whole continent. More generally, this
>   depends on the 'density' of points on a given area with respect to the
>   resolution of grid(s).

(Actually, when QGIS reprojects a bounding box (such as a layer
bounds), it will automatically densify the edges first -- so we'd need
at least the grids covering the whole of the perimeter as a start. But
that's nitpicking!)

>
> - there is also the fact that the local cache is of limited size, so if for
>   the whole set of points to coordinates you'd need more than the cache
>   maximum size, eviction will occur. For that point, that could be potentially
>   avoided by making the cache size such that it never happens. I arbitrarily
>   decided for a default of 100 MB. The total size of GeoTIFF grids is close
>   to 500 MB currently. So if we set the default to a bit more than the total
>   size of grids, eviction would never occur. That said, this point is probably
>   only theoretical. The 2 largest files currently are egm08_25.tif (77 MB) and
>   BWTA2017.tif (70 MB), so even if you reprojected entire Baden-Wurttemberg
>   and referenced EGM08 height that would still below 100 MB.
>
> So perhaps an easier & robust solution for your use case could be:
>
> - for the transformation of interest, use the PROJ_GRID_AVAILABILITY_IGNORED
> flag to have the operations returned in the ideal case. Then for each point to
> transform, you check the first candidate operation whose area of use
> intersects with the point. That's the one that would be selected in an ideal
> world. If it references a grid that proj_coordoperation_get_grid_used()
> indicate as not available, take note of it, and use the next operation that
> doesn't need a grid or for which grids are available, to perform the
> approximate coordinate transformation.
>
> - at the end of this stage, collect the set of missing grids and download them
> (entirely), and restart the transformation.

That's exactly the current approach - minus the auto download.
Instead, we just warn users and advise them on the steps required for
downloading the grid themselves and provide an option to "install" the
downloaded grid (copy it to a proj lib folder).

Based on this, I'm thinking that the best approach to take with QGIS
in a post rfc-4 world would be to leave the built-in proj networking
disabled, and then extend the current QGIS approach to provide an
option to download the whole grid from the CDN instead of just showing
the manual steps to download them. (And possibly, just do a
post-install, first run background task which bulk downloads the
complete set of grids from the CDN so that they're available
already...)

> Which can be done with existing (non-RFC4) API.

On a different note -- I'm finding it very difficult to find the RFC4
proposal text. This appears to live only in the proj website contents
in the rfc4 branch on your proj fork, so aren't actually available on
the proj website (and googling proj rfc4 gives no useful results as a
consequence). This seems a step back from the GDAL/trac RFC approach,
where RFC text is available for widespread review on the GDAL page
**prior** to the RFC PR merge. Is the approach used for RFC4 the plan
for RFCs going forward?

Nyall


>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com