[SAC] rsync backups for download.osgeo.org

[Alex Mandel]

On 11/27/2010 12:56 AM, Markus Neteler wrote:
> On Sat, Nov 27, 2010 at 9:05 AM, Alex Mandel <tech_dev at wildintellect.com> wrote:
>> On 11/26/2010 11:35 PM, Markus Neteler wrote:
>>> On Fri, Nov 26, 2010 at 6:40 PM, Frank Warmerdam <warmerdam at pobox.com> wrote:
>>>> Folks,
>>>>
>>>> Martin pointed out that the download.osgeo.org rsync backups have not been
>>>> running (for some time!).  I have fixed download.osgeo.org so that rsyncd
>>>> is started from inetd and reenabled rsyncing from download.osgeo.org to
>>>> backup.osgeo.org again.
>>>
>>> Thanks for your efforts.
>>>
>>>> I'll try to check in that things are working smoothly in a few days.
>>>>
>>>> Martin and I also briefly discussed what else is, or is not, getting
>>>> backed up on the new VMs and I think there are likely some holes.  I think
>>>> he may follow up on some of those points but folks responsible for services
>>>> on the new VM might want to confirm that everything they need is getting
>>>> backed up in a way that we could recover fairly quickly if necessary.
>>>
>>> The backup of the "project" VM is quite important. Is there anything
>>> known about that (and/or an updated Wiki page describing the state
>>> of the art)?
>>>
>>> thanks
>>> Markus
>>
>> Well there are 2 distinct steps.
>> 1. a project needs to dump it's files/databases/configurations somewhere
>> easy to find and in an efficient format.
> 
> yes (I do this for GRASS occasionally).
> 
>> 2. we need to configure a way to pull those to the Backup machine, most
>> likely using Bacula, but possibly just relying on rysnc
> 
> So you mean that the project VM is not backup'ed at all?
> oh.... since 9 servers are running on it... As of today
> it contains:
> 
> docs.geotools.org.conf
> geos.osgeo.org.conf
> geotools.org.conf
> mapserver.org.conf
> projects.mapbender.osgeo.conf
> sr.org.conf
> tilecache.org.conf
> www.gdal.org.conf
> www.remotesensing.org.conf
> 
> oh.... since 9 servers are running on it...
> 
>> For step one I think a convention of /backup/projectname should make it
>> easy for us to sort through and grab backups with bacula.
>>
>> Does that work well for others?
> 
> Please set up a bacula backup soon.
> 
> thanks,
> Markus

Bacula is setup and currently backs up /etc on all of the new servers.
We have have not setup a backup for a /backup folder yet as far as I
know because we never agreed on the plan. If this idea works then we'll
have Martin setup the jobs on Bacula. It will be up to each of the above
mentioned projects though to create a script appropriate for dumping
their data appropriately.

We also need to decide on the frequency, full vs incremental and the
time schedule:
I would say every night(or 2), incremental, full every 2 weeks, maybe
right after the /etc backups happen.
An alternate if there is a lot of database dumping might be every week,
full every 4 weeks, sometime on the weekend.

Thoughts?

Thanks,
Alex