[SAC] Subversion backups

[Howard Butler]

All,

I have reworked our subversion backup strategy a bit in hopes of  
eliminating our locking problems.

First, I have changed things so that we do an incremental backup  
every three hours.  The incremental backup, which stores its output  
in /home/back/svn_backup/project/incremental, only does an 'svnadmin  
dump' for revisions that it hasn't already dumped.  The script, which  
lives in /var/www/svn/tools/incremental-backup.py, spits out a file  
called 'last_backed_up' and uses that to do its work.  The script  
runs three times per day, and if a project hasn't committed anything,  
there is almost no overhead to running the script.  Hopefully this  
approach will limit our locking and repository-falling-on-the-floor  
problem we were seeing.

Second, I have implemented a full backup using the Subversion  
project's hot-backup.py script, which also lives in /var/www/svn/ 
tools/.  This script is setup to run a full hotcopy of the entire  
repository once per day at 0600 UTC and puts its output in /home/back/ 
svn_backup/project/full.  14 copies of the repository are kept, which  
is similar to before.

Third, these backups are now done on a per-project basis rather than  
treating /var/www/svn/repos as one giant repository.  When adding a  
new project, we have to make sure to create a few backup directories  
in /home/back/svn_backup/ and edit the two scripts in /root/scripts/ 
backup_svn_{full,incremental}.sh.  If someone wishes to come up with  
something less crude that doesn't have that overhead ... as they say  
in #openlayers... patches graciously accepted ;)

This split approach gives us multiple chances to recover if things  
really go haywire... we have a sequential dump that can be cat'ted  
together in ./incremental, and a full dump of everything (14 of  
them!) with the bdb files and everything else in the repository.

I would request that another admin (Tyler, Frank, Shawn) come along  
behind me and check that everything is working correctly and behaving  
as it should.  <http://wiki.osgeo.org/index.php/ 
Migration_Documentation#current_cron_jobs_for_backups> updated to  
reflect current situation.

/me crosses fingers.

Howard