[Qgis-developer] hub.qgis.org dead slow?

[Alex Mandel]

On 12/13/2012 07:29 AM, Jürgen E. Fischer wrote:
> Hi Andreas,
> 
> On Thu, 13. Dec 2012 at 10:48:07 +0100, Andreas Neumann wrote:
>> I think we could easily pay from the existing QGIS funds for a dedicated 
>> build/test server. At  
>> http://www.hetzner.de/en/hosting/produktmatrix/rootserver-produktmatrix-ex 
>> (where I also have my private server and the GIS server of the City of  
>> Uster) you can get dedicated root servers with lots of memory and good  
>> CPUs for about 70-90 Euros per month. It is a very reliable company and  
>> runs on renewable energy. Adds up to 840 to 1080 Euros per year - in my  
>> opinion a good investment if we can get better performance for our core  
>> services.
> 
> The builds work fine and don't cause the issues (and nicing them didn't help a
> bit) - the only problem with that, is that the diskspace runs out - something
> was added lately that occupies space.
> 
> I think it's the documentation site. But Alex is about to organize more space.
> So that's should be shortly solved.
> 
> But I'm just monitoring the stuff and try to find out what's wrong, although I
> didn't setup any of the webservices.  So it's hard to tell what's necessary and
> whats not and sometimes if something behaves normal or goes bezerk.
> 
> There's now a script in place that restarts apache, whenever is stops
> responding - with a report of what was going on at the time.
> 
> It's currently about cron.daily time (or was when I started this) - there were
> four planets jobs running feedjack_update.py and apparently stressing
> postgresql with concurrent queries (SELECTs, INSERTs DELETEs & UPDATEs).   I
> supposed these should be guarded to not run in parallel.  And there was also an
> unniced backup of redmine running.
> 
> There are probably more cron jobs that have that problem.  Some cron jobs and
> webservices also don't/didn't rotate their logs (and produces/d huge ever
> growing logs - which apparently nobody looks at anyway).
> 
> BTW The nightly builds are niced and should only run if there's nothing else
> wanting the CPU.  See the red stuff in
> http://webextra.osgeo.osuosl.org/munin/osgeo.org/qgis.osgeo.org-cpu.html.
> 
> I for one don't want to move the build stuff to another server - I would even
> have one (also from Hetzner ;), if I wanted to do that.  Please move everything
> else ;)
> 
> BTW can we alternatively get more CPU on our VM?
> 
> 
> Jürgen
> 

Yes, I'm going to request +10GB and +1 Cores, hopefully this week.

As for the plugin repo, I think we should look at mirror/caching options
for the plugin repo files that are served up rather than a whole
separate server. Some sort of cdn service would make plugin install
quite quick for many people around the world and free up some resources
on our server. Of course we want to find a service that refreshes or is
easy for us to push updates quickly. Alternate to that, since several of
us have fast servers around with space, I could setup a mirrorbrain
which auto refers people to geographic closest copies of plugins (I ran
this for osgeo live for a while).

Thanks,
Alex

PS: I'm happy to host mirrors of content at my university too (West
Coast US)