[Qgis-user] official plugins repository unavailable?

[Alessandro Pasotti]

On Sat, Nov 5, 2016 at 10:27 AM, Richard Duivenvoorde <rdmailings at duif.net>
wrote:

> On 05-11-16 10:11, Alessandro Pasotti wrote:
> > Hi
> >
> > I've been monitoring the server and I think that with the caching in
> > place the timeouts are gone, but of course we need to add the new
> > versions as they appear.
> >
> > Richard: what do you think about adding a couple of future versions too,
> > so we don't forget?
>
> Yep, done: 3.0 3.1 3.2 3.3 added
>
> Currently the nightly requests the ?qgis=3.0 version (see [0])
>
> by the way, running the get_plugins.xml script takes  2,5 minute
> $ time ./get_plugins_xml
> real    2m27.058s
> user    0m0.040s
> sys     0m0.096s
> And runs every 10 minutes. Maybe we should bring that up in future, or
> make the script faster?
>
>

I've already spent quite a lot of time trying to make it faster and it's
now indeed much faster: a single run (for a single QGIS version) is ~3
seconds depending on the server load (it was sometimes over 10 seconds
before my latest changes!).
What seemed a simple addition (the trusted flag) came out to be a
performance killer, because the user "trusted" information is a couple of
tables away:
plugin_version -> plugin -> author -> group -> permission

We are now using raw queries:
https://github.com/qgis/QGIS-Django/commit/3ffd9c3d989ce144a2cf9d7f02ea2b0dc8f5b5ad

The problem is also that on that server there are a lot of other processes
going on (doc builds for example) that eat resources.

I don't think we can achieve much better results without some deep
refactoring of the plugins application (adding triggers to set the trusted
flag on the plugin version row could be a good starting point)


BTW, I'd lower the frequency of the caching script: I don't think that
waiting 30 minutes or more to see a plugin appear in the XML will kill
anyone.


-- 
Alessandro Pasotti
w3:   www.itopen.it
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20161105/21ff0505/attachment.html>