[QGIS-Developer] Proj 7 auto grid download

[Nyall Dawson]

On Mon, 23 Mar 2020 at 17:49, Kristian Evers <kreve at sdfe.dk> wrote:
>
>
>
> > On 23 Mar 2020, at 01:28, Nyall Dawson <nyall.dawson at gmail.com> wrote:
> >
> > Not just QGIS. The auto-download behavior is an opt-in switch for
> > applications, it's not the default behavior.
> >
> > I can't find any application which use the CDN by default
> > -- searching
> > github for proj_context_set_enable_network only gets hits in proj
> > itself.
>
> PROJ 7 has been out for 23 days. Give everyone a chance to catch up, please.

Sorry, wasn't intending that to be criticism. It's just a reflection
of where things are right now.

> And honestly, I think many developers look at what you guys are doing, so
> expect to be the front-runners on this.

Honestly, I think QGIS needs to step back for a couple of releases and
let other projects lead the way on relying on newer proj features.
Through no fault of the proj library, we've had a rocky road over the
last couple of releases wrt transformations, and I'd very much like to
get some nice stable releases out before we push things forward again
on this front. My belief is that we should have resolved all the
nasties in 3.10.3 and 3.12.1 and should be finally back providing
stable transformation handling.

On another note, which I raised in either discussion on the proj list
or with Even directly (I can't recall now), but I just don't think
it's possible for us to deliver a good user experience if we use the
proj 7 inbuilt auto download feature. My reasoning being:
- transformations are all blocking operations in qgis -- we can't
process them async
- many transformations occur on the main thread. E.g. during the
preparation steps for a background map render, transformations are
done on the main thread in a blocking way. And when moving the mouse
over the canvas, many transformations are occurring in order to update
the status bar, process the current map tool correctly, etc. These
**can't** be delegated to a background thread, it's just not possible.
- understandably, transformations in the proj library are also
blocking. So when a transform is requested which requires a grid
section download, the calling application (qgis) will be blocked while
this download occurs
- the consequences are that transformations will be happening on the
main thread which may take many seconds (or more, on a bad connection)
to return. This will be unpredictable and will result in very poor UX
-- QGIS will just appear to be hanging and stuttering randomly during
normal operations.
- We can't even give users feedback about what's happening during
these hangs -- the main thread will be blocked, so we can't show
anything in the GUI to let users know that things are downloading in
the background and that QGIS will unfreeze when this is done!

It's a **great** feature for command line applications and scripting
tools, but I just can't see anyway to nicely take advantage of it from
a GUI application like QGIS. (ideas welcome!.. e.g. maybe if proj
offered a background "pre-fetch" grid API then we could trigger that
when a grid is first requested and immediately return a non-grid based
result, giving users feedback via a task report that more accurate
results will be shortly available. That's a lot of complexity on both
proj/qgis side though, vs just forcing every install to have every
grid ever, by default!!)

We've also got to consider users with no internet connection, and no
knowledge in advance of the need for grids... ouch!

> Would the choice have been obvious if auto-downloading was turned on
> by default?

Nope - see above. I'd have hardcoded a call in to automatically switch
it back off until we can deliver a good user experience!

Nyall

>
> /Kristian
>
> [0] https://github.com/conda-forge/proj.4-feedstock/blob/master/recipe/scripts/activate.sh
>
>
>
>