[SAC] download server: No space left on device

[Alex M]

On 10/19/2017 02:14 AM, Sandro Santilli wrote:
> On Thu, Oct 19, 2017 at 10:51:53AM +0200, Jürgen E. Fischer wrote:
>> Hi,
>>
>> On Fri, 24. Mar 2017 at 23:15:12 -0700, Alex Mandel wrote:
>>> More space on the current server is not a financial issue. Technical,
>>> only Martin and OSU have access to modify the ganeti disks. This is a
>>> short term solution to expand or add another disk slice. The easiest way
>>> is to mount a 2nd ganeti disk using some of the unallocated space from
>>> the host. I can write up the commands if that helps, but don't have
>>> access to do it myself. Growing the existing disk is also possible but
>>> requires more than a reboot, and has a slight risk of data loss (as all
>>> ext3 expansions do).
>>
>> Could we please finally do something about this?
>>
>> This is still an issue.   I think it happened again yesterday/today.  I have a
>> bunch of changed timestamps in osgeo4w now of unknown origin.  I guess somebody
>> went through it and removed some (obsolete?) stuff.
> 
> From http://wiki.osgeo.org/wiki/SAC_Service_Status I understand that
> "download.osgeo.org" is a VM on a so-called "osgeo3" machine hosted
> by OSUOSL, but nothing is found on that page about how to access it.
> Neither osgeo3.osgeo.osuosl.org nor osgeo3.osuosl.org have a DNS
> entry, so I don't even know the IP of that machine.
> 
> The same wiki page mentions that "Primary administrators" have ssh
> keys to connect to physical machines. This set shuld include Alex and
> Markus (and others) in addition to Martin, see:
> https://wiki.osgeo.org/wiki/SAC:Primary_Administrators
> 
> Markus, Alex: do you not have acces to that machine ?
> Martin: any reason not to share that Ganeti access with other SAC
> members ? If we cannot control that machine maybe we should move
> it elsewhere ?
> 
> Besides, the access file on "secure" contains some ganeti
> related access credentials, but they don't seem to work.
> Should we contact OSUOSL support ?
> 
> --strk;
> 

Martin is the only one who ever managed to get access to that host.
Partly because it required some hops via an osuosl backend service,
which is for security.

Adding a 2nd ganeti disk is probably the easiest option, resizing
existing disks is a hassle with real downtime (more than a restart).
Yes if you write up what you want we can put a ticket in to OSUOSL to
add the 2nd disk.

Here's the current disk allocation best I know
http://wiki.osgeo.org/wiki/Infrastructure_Transition_Plan_2010#osgeo3_.26_osgeo4

An NFS mount from osgeo6 is also an option. Could be done all by us.

-- Backstory --
OSUOSL offered to move us to their parallel FTP server system for
downloads, but something in the coordination never made that happen.
If someone wants to re-inquire about that, speak up.

We originally didn't have direct access because OSUOSL was managing a
custom gentoo install for the hosts. We've all learned years later that
we should have done it a different way. At this point osgeo3 should be
working towards retirement.

osgeo7, yet to be purchased is the planned replacement/upgrade of osgeo3
where we would configure it ourselves, like osgeo6.

However osgeo7 can't happen without retiring OSGeo4 (we need the rack
space and power(PDU space)), which means moving the QGIS and Adhoc VM
components to osgeo6. As I've noted QGIS materials could be moved at any
time. No idea what to do with Adhoc, containers would be nice, but maybe
a straight VM copy is easiest.

Thanks,
Alex