[PROJ] RFC4: last chance to comment before motion

[Nyall Dawson]

On Thu, 2 Jan 2020 at 23:47, Even Rouault <even.rouault at spatialys.com> wrote:
>
> Hi,
>
> PROJ 6.3 being out, it is time to consider the next steps :-)
>
> RFC4
> https://github.com/rouault/PROJ/blob/rfc4_remote_and_geotiff_grid/docs/source/community/rfc/rfc-4.rst
> is hopefully now in its final state. Number of changes have been added to it since
> the initial version to synchronize it with the implementation, but nothing dramatically different.
> You can the diff at https://github.com/rouault/PROJ/compare/5bbfc07..2b16de3

Hi Even,

Over the Christmas season I've been mulling over this in the back of
my mind, specifically trying to brainstorm on how the new CDN based
delivery approach could fit into a GUI type application like QGIS.

While the current RFC4 API will allow us to reuse the existing QGIS
network layer (making it seamlessly work with existing proxy settings,
etc), I'm doubtful that the current API will allow for a good user
experience in QGIS. My reservation is that given that the network
fetching will be (understandably) a blocking process, it means that
QGIS will suffer application/GUI hangs whenever proj determines that
it needs to fetch a new area of the grid and the network download
occurs. (why a GUI hang? well, by necessity all the render job
preparation has to occur on the main thread. So any blocking operation
during this preparation, such as reprojecting layer extents to the
target map crs, will block the main thread without any way for us to
notify users as to why this is occurring or how the download is
progressing).

I've been trying to mull over what would be an ideal UX here. I think
the best we could possibly do would be:

1. If a grid download is required, we don't do this upfront. Instead,
we use the best available non-download-grid operation so that the
reprojection can occur without blocking.
2. We notify the users that a non-preferred projection was used for the render.
3. In a background thread, we fire up a download request for the
required grid portion.
4. When the download finishes, the map refreshes automatically using
the newly available coordinate operation

and then

5. Proactively fetch nearby grid regions based on the current map view
extent (i.e. download the neighbouring tiles in background threads)
6. Offer users a way to manually download the grids for a selected region

My understanding of the current API leads me to believe that this
wouldn't be possible in the current RFC4 API. The missing bits would
be:

1. Public api to query if the preferred operation between two crses
requires a grid download for a certain area
2. (Thread safe) API to manually download a grid for a certain area

What's your thoughts on this?

Nyall





>
> It is now backed by a working implementation available at
> https://github.com/OSGeo/PROJ/pull/1817
>
> The CDN, storing grids converted to GeoTIFF, is now available at https://cdn.proj.org/
>
> As an example, the following will now work (using the OSTN15_NTv2_OSGBtoETRS.tif grid):
>
> $ echo 60 2 | PROJ_NETWORK=ON cs2cs -d 8 OSGB36 ETRS89
> 59.99957003     1.99763761 0.00000000
>
> On next invocations, the parts of the grid that are needed and have already been
> queried will be retrieved from the cache (at ~/.local/share/proj/cache.db on Linux)
>
> For now, the RFC4 implementation works with a grid_alternatives database table still
> pointing at the NTv2 & GTX files of https://github.com/osgeo/proj-datumgrid and it will
> automagically try to fetch the correspond .tif files from the CDN when they are not locally
> available.
>
> In a next step, we'll also need to decide how/when we switch to
> https://github.com/osgeo/proj-datumgrid-geotiff (which is the master of cdn.proj.org) and
> if we provide a single or multiple packages of it (current total size of files is 486 MB),
> and/or a utility to retrieve as a post installation stage grids of interest from a given
> bounding box / producer / country from the CDN.
> I was initially thinking to a python utility, but there could have
> issues with Python dependencies, so perhaps a binary using the new curl dependency would be
> easier to deploy & use. This can be for follow-up discussions.
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> PROJ mailing list
> PROJ at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/proj