[SAC] Backup and migration ideas

[Howard Butler]

On Aug 20, 2009, at 2:15 PM, Tyler Mitchell (OSGeo) wrote:

> Hi all,
> There are a couple systems related ideas on my mind for a while and  
> I wanted to put them past you for your thoughts, ideas, critique.   
> In general, I've been digging into our options for hosting at http://www.osuosl.org 
>  and a few ways we could capitalise on the opportunity without  
> making too much work in the short term.
>
> 1 - Migrate services from osgeo2 to OSL.
> This is likely the simplest server to move over and has the added  
> benefit of reducing our costs by around 50%.  In some ways we've  
> maxed out osgeo2 and upgrading is not cheap.  I suggest we focus on  
> building on a new secondary server
> * Systems affected: wiki, planet, rsync backups from other servers,  
> development Drupal sites, others?
> * Challenges:  LDAP auth back to osgeo1 needed?, others?

I fully support this as a way to dip our toes into OSL.  I'm in no  
position to lead such an effort, but I'm willing to help bikeshed.   
Having each service virtualized with whomever does the work's choice  
of virtualization environment would be fantastic.  osgeo2 and osgeo1  
don't get upgraded at all because the rigor of interdependencies makes  
the prospect quite scary.  If we can have more independent chunks, we  
can break smaller things when we do stuff.

>
> 2 - Mirror project websites onto OSL.
> If successful, then potentially switch over to the new server where  
> appropriate.  My particular interest here is to have all main OSGeo  
> project websites served in a managed server environment.  Hosting  
> these onto osgeo1 typically opens up security questions.  We may be  
> able to get VMs for each project or use LDAP shell authentication  
> (like on some xblades) to handle the various projects in the same  
> VM.  Could also open this up to include projects not hosted on OSGeo  
> infrastructure, even just static mirrors.
> * Sites affected: gdal, grass, qgis, geotools, mapserver (?),  
> geonetwork, mapbender, ?
> * Challenges: several different platforms, shell authentication, ?

Project sites are on the blades because this is where we feel most  
comfortable giving them the keys and letting them drive off their own  
cliffs.  A virtualization effort, similar to above (on the same box  
even?) that allows each project to have their own keys would be  
enticing enough for people to go through the pain of a server move.   
Some of projects, like MapServer, GDAL, and GRASS were able to mirror  
off to other places in the time it took for DNS to switch over during  
the past outage.  Other projects weren't quite as ready as that, but  
that's always been the deal with running on the blades.

>
> 3- Build upgraded osgeo.org site on OSL machine.
> Wolf and I have discussed a Drupal upgrade over the next few months  
> that should fix a few things.  Building it on a new system has some  
> advantages when upgrading all our modules, etc.  I already have an  
> account and Drupal set up on an OSL machine for testing.
> * Sites affected: www.osgeo.org, mapguide, fdo, mapbender

... subversion, trac, postgresql for trac, mysql, ldap.  There's a ton  
of critical stuff on osgeo1 right now.  svn/trac + its db should  
probably be separated onto its own instance somewhere (it also  
probably best-positioned and most self-contained for a move off of  
osgeo1 as well).