[SAC] Backup and migration ideas

[Tyler Mitchell (OSGeo)]

Thanks for the feedback guys, I'm glad it's not totally off the rocker. 
One more important point to add...

I've been pinging OSL on and off on similar ideas the past few weeks. 
Using the existing VM they gave us would be simple for the Drupal upgrade,
but beyond that they JUST let me know they are hurting for hardware space
at the moment.  So, I'm not sure what they were thinking when discussing
hosting ideas with us at OSCON, but either way...

I will therefore propose that we get some nice new hardware and host it
there.  Anyone on this list especially good at spec'ing out machines?   I
assume we can get some good hardware for the equivalent cost of a couple
months (~$3k) of our current hosting and shouldn't make a major impact on
our overall finances, especially if we can end up retiring osgeo2.

I can see that using the blades has already helped us get into the mindset
of using VMs for compartmentalising our sets of services into groups.  Now
this is where I'm beyond my knowledge to recommend anything, but setting
up virtual server instances is something that OSL staff seem to do quite
well.  I'll start up some docs to help brainstorm this general migration
discussion and then we'll have something to take up with them when we are
ready to dig deeper.

Thanks again for the comments, but especially for your continued help with
OSGeo systems.  It's nice to see that outages are so rare that we really
notice them ;-)

Tyler

>
> On Aug 20, 2009, at 2:15 PM, Tyler Mitchell (OSGeo) wrote:
>
>> Hi all,
>> There are a couple systems related ideas on my mind for a while and
>> I wanted to put them past you for your thoughts, ideas, critique.
>> In general, I've been digging into our options for hosting at
>> http://www.osuosl.org
>>  and a few ways we could capitalise on the opportunity without
>> making too much work in the short term.
>>
>> 1 - Migrate services from osgeo2 to OSL.
>> This is likely the simplest server to move over and has the added
>> benefit of reducing our costs by around 50%.  In some ways we've
>> maxed out osgeo2 and upgrading is not cheap.  I suggest we focus on
>> building on a new secondary server
>> * Systems affected: wiki, planet, rsync backups from other servers,
>> development Drupal sites, others?
>> * Challenges:  LDAP auth back to osgeo1 needed?, others?
>
> I fully support this as a way to dip our toes into OSL.  I'm in no
> position to lead such an effort, but I'm willing to help bikeshed.
> Having each service virtualized with whomever does the work's choice
> of virtualization environment would be fantastic.  osgeo2 and osgeo1
> don't get upgraded at all because the rigor of interdependencies makes
> the prospect quite scary.  If we can have more independent chunks, we
> can break smaller things when we do stuff.
>
>>
>> 2 - Mirror project websites onto OSL.
>> If successful, then potentially switch over to the new server where
>> appropriate.  My particular interest here is to have all main OSGeo
>> project websites served in a managed server environment.  Hosting
>> these onto osgeo1 typically opens up security questions.  We may be
>> able to get VMs for each project or use LDAP shell authentication
>> (like on some xblades) to handle the various projects in the same
>> VM.  Could also open this up to include projects not hosted on OSGeo
>> infrastructure, even just static mirrors.
>> * Sites affected: gdal, grass, qgis, geotools, mapserver (?),
>> geonetwork, mapbender, ?
>> * Challenges: several different platforms, shell authentication, ?
>
> Project sites are on the blades because this is where we feel most
> comfortable giving them the keys and letting them drive off their own
> cliffs.  A virtualization effort, similar to above (on the same box
> even?) that allows each project to have their own keys would be
> enticing enough for people to go through the pain of a server move.
> Some of projects, like MapServer, GDAL, and GRASS were able to mirror
> off to other places in the time it took for DNS to switch over during
> the past outage.  Other projects weren't quite as ready as that, but
> that's always been the deal with running on the blades.
>
>>
>> 3- Build upgraded osgeo.org site on OSL machine.
>> Wolf and I have discussed a Drupal upgrade over the next few months
>> that should fix a few things.  Building it on a new system has some
>> advantages when upgrading all our modules, etc.  I already have an
>> account and Drupal set up on an OSL machine for testing.
>> * Sites affected: www.osgeo.org, mapguide, fdo, mapbender
>
> ... subversion, trac, postgresql for trac, mysql, ldap.  There's a ton
> of critical stuff on osgeo1 right now.  svn/trac + its db should
> probably be separated onto its own instance somewhere (it also
> probably best-positioned and most self-contained for a move off of
> osgeo1 as well).
>
> _______________________________________________
> Sac mailing list
> Sac at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/sac
>