[QGIS-Developer] [Qgis-user] Python Implementation of QgsNetworkContentFetcher

Wed Nov 27 00:20:44 PST 2019

Hi Nyall,

Many thanks!
I posted your answer on stackexchange to make the info more easily
accessible.
https://gis.stackexchange.com/questions/343126/how-to-perform-sync-or-async-network-request-in-pyqgis/343127#343127

The nam does offer support for authcfg and blocking request apparently (not
tested yet though).

Cheers,
Denis

Le mer. 27 nov. 2019 à 06:35, Nyall Dawson <nyall.dawson at gmail.com> a
écrit :

> On Wed, 27 Nov 2019 at 15:26, Denis Rouzaud <denis.rouzaud at gmail.com>
> wrote:
> >
> > Thanks Nyall for the precisions.
> > I wasn't aware of QgsBlockingNetworkRequest.
> >
> > So basically, if I don't require authcf, then starting from 3.6 there is
> no reason to use my python nam, I should use both QgsNetworkContentFetcher
> and QgsBlockingNetworkRequest. And from 3.10 if require authcfg in
> QgsNetworkContentFetcher.
> > Is this correct?
>
> QgsBlockingNetworkRequest has always supported auth, so you should use
> that in 3.6 and above in all situations WHEN you need a blocking
> request.
>
> QgsNetworkContentFetcher is async, so definitely prefer to use that
> whenever you don't need a blocking request.
>
> I'd say the matrix is something like
>
> QGIS 3.4:
>
>  Blocking:
>    With auth: python nam (but be aware that it CAN and DOES crash
> crashes if used on the main thread)
>    Without auth: python nam (but be aware that it CAN and DOES crash
> crashes if used on the main thread)
>
>  Non-blocking:
>    With auth:      nothing available off-the-shelf -- roll your own
> using raw the qt and QGIS auth api?
>    Without auth: QgsNetworkContentFetcher
>
> QGIS 3.6 - 3.8:
>
>  Blocking:
>    With auth: QgsBlockingNetworkRequest
>    Without auth: QgsBlockingNetworkRequest
>
>  Non-blocking:
>    With auth:      nothing available off-the-shelf -- roll your own
> using raw the qt and QGIS auth api?
>    Without auth: QgsNetworkContentFetcher
>
> QGIS 3.10:
>
>  Blocking:
>    With auth: QgsBlockingNetworkRequest
>    Without auth: QgsBlockingNetworkRequest
>
>  Non-blocking:
>    With auth:      QgsNetworkContentFetcher
>    Without auth: QgsNetworkContentFetcher
>
>
> Nyall
>
>
>
> >
> > Cheers,
> > Denis
> >
> > Le mer. 27 nov. 2019 à 00:27, Nyall Dawson <nyall.dawson at gmail.com> a
> écrit :
> >>
> >> On Wed, 27 Nov 2019 at 00:44, Denis Rouzaud <denis.rouzaud at gmail.com>
> wrote:
> >> >
> >> > Hi,
> >> >
> >> > Jumping late but we use the work from NAM Alessandro did in a couple
> of locator filters.
> >> >
> https://github.com/opengisch/qgis-swiss-locator/blob/master/swiss_locator/core/network_access_manager.py
> >> >
> >> > I believe I have done very very few adaptations/fixes to the original
> work and it works like a charm.
> >>
> >> Not quite. It works in some circumstances, but it uses an event loop.
> >> So if you run it on the main thread, you're potentially doing an
> >> effective call to QApplication.processEvents, which can lead to all
> >> sorts of crashes. That said, it's generally a safe approach to use for
> >> QGIS <= 3.4... just don't use it in any code which relates to
> >> rendering!
> >>
> >> >
> >> > I had in mind to bring this as is within QGIS core for some time. The
> main point being for us the blocking request rather than authcfg.
> >>
> >> That's EXACTLY what QgsBlockingNetworkRequest is -- please don't
> >> re-re-implement this!
> >>
> >> Nyall
> >>
> >>
> >> >
> >> > Denis
> >> >
> >> > Le mar. 26 nov. 2019 à 15:36, C Hamilton <adenaculture at gmail.com> a
> écrit :
> >> >>
> >> >> Thanks everyone. This gives me some good ideas. These are the
> possible solutions:
> >> >>
> >> >> 1. Write a python implementation of QgsNetworkContentFetcher or
> QgsBlockingNetworkRequest. With this I could look at the QGIS version and
> if it is 3.10 or greater import the core method; otherwise, import the
> python implementation.
> >> >> 2. Use the old Boundless implementation if it still works in QGIS 3.
> >> >> 3. Use QgsFileDownloader and save it as a temporary file before
> processing.
> >> >>
> >> >> Calvin
> >> >>
> >> >> On Tue, Nov 26, 2019 at 2:29 AM Alessandro Pasotti <
> apasotti at gmail.com> wrote:
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Tue, Nov 26, 2019 at 8:22 AM Richard Duivenvoorde <
> rdmailings at duif.net> wrote:
> >> >>>>
> >> >>>> On 26/11/2019 00.06, Nyall Dawson wrote:
> >> >>>>
> >> >>>> > There's not. But there's the ex-boundless "networkaccessmanager"
> >> >>>> > implementation that supports authcfg, which you can find at
> >> >>>> >
> https://github.com/north-road/qgis-redistricting-plugin/blob/master/redistrict/linz/networkaccessmanager.py
> >> >>>> > (and other projects -- as far as I'm aware there's no real
> upstream
> >> >>>> > for this... Alessandro?).
> >> >>>>
> >> >>>> I used this Boundless thingie for several plugins now (original
> source
> >> >>>> is now pointing/redirecting to planetlabs [0]).
> >> >>>
> >> >>>
> >> >>> Yes, that's the "upstream" but I don't know if that one is
> maintained anymore.
> >> >>>
> >> >>> Btw, there is also a specialized class for sync/async downloads
> that supports authcfg:
> https://qgis.org/api/classQgsFileDownloader.html#details
> >> >>>
> >> >>>
> >> >>>>
> >> >>>>
> >> >>>> I think we already discussed several times that it would be good to
> >> >>>> incorporate this into a commons lib for QGIS. But if I'm correct
> the
> >> >>>> actual interface was not exactly what some wanted, and if I recall
> >> >>>> correct Nyall tried to pull this into c++ but hit some
> multithreading
> >> >>>> issues.
> >> >>>>
> >> >>>> But as Nyall says: I think this is a great lib to use in a plugin
> (also
> >> >>>> because it used the QgisNetwork-stuff, so you can receive the
> signals
> >> >>>> from it or log the precise urls, like we do in the
> QGISnetworklogger
> >> >>>> plugin [1]
> >> >>>>
> >> >>>> Regards,
> >> >>>>
> >> >>>> Richard Duivenvoorde
> >> >>>>
> >> >>>> [0]
> >> >>>>
> https://github.com/planetfederal/lib-qgis-commons/blob/master/qgiscommons2/network/networkaccessmanager.py
> >> >>>> [1] https://github.com/rduivenvoorde/qgisnetworklogger
> >> >>>> _______________________________________________
> >> >>>> Qgis-user mailing list
> >> >>>> Qgis-user at lists.osgeo.org
> >> >>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> >>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Alessandro Pasotti
> >> >>> w3:   www.itopen.it
> >> >>
> >> >> _______________________________________________
> >> >> QGIS-Developer mailing list
> >> >> QGIS-Developer at lists.osgeo.org
> >> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> >
> >> > _______________________________________________
> >> > QGIS-Developer mailing list
> >> > QGIS-Developer at lists.osgeo.org
> >> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20191127/5aa720cc/attachment-0001.html>