[Live-demo] System behaviour in low disk space conditions

[Hamish]

Jeremy wrote:
> > Firstly, it's pretty common when I'm working with students that they're
> > not careful with disk space but are used to having lots of it. Even if you
> > give them another partition on the USB stick, or even tell them about
> > accessing the hard drive on the machine from the Live system, inevitably
> > one or more will just download big ZIP files to their home directory,
> > unpack the contents there, never clear it out, etc. So eventually the home
> > directory & thus the system runs out of space.

eventually is actually quite quickly, as every changed file vs. the original
compressed image is saved to the overlay-filesystem on the USB drive. As Angelos
mentioned a package upgrade can quickly fill it, and over time if you are
moving packages and programs around it adds up, maybe just after a few weeks.
A user who isn't reinstalling packages and just writes to the home dir should
get a bit longer out of it. But the persistent USB certainly fills up faster
than you expect.

> > My observation is that the system degrades very poorly in these circumstances,
> > becoming unpredictably unreliable and sometimes unbootable. Often it's
> > PostgreSQL that seems to be the poorest to cope but this is all anecdotal
> > and hasn't been performed in test conditions. Is there any way that this
> > behaviour can be modified, or at least some system monitor be included to
> > warn of the impending situation?

It's a good observation, I too noticed that the persistent USB was much slower
than the same USB stick with the non-persistent setting. Mounting the overlay-
filesystem (it's just ext2-in-a-big-file) showed that indeed Postgresql was
taking up lots of the space. I've recently added 'VACUUM ANALYZE' after all
the database loading, I think that was quietly auto-running on first boot and
slowing things down + filling the drive up. So hopefully that aspect will be
better for the next release.

Angelos:
> I am sharing the same problem too, since I use the disk to
> teach classes. In my case I avoid the use of persistent usb
> sticks and prefer to use the VM version which has more space
> available in the virtual drive. Of course this is taking
> longer to setup, but students get to learn how to use
> virtual machines too :)

they are not as portable, but probably better for a month long+
course. For a 2 day short course I'd argue to hit the ground
running with a persistent (+disposable/reformattable) usb drive.
Learning that you can do magic like having a full setup on a
usb and run a VM are both empowering things to be introduced to.

> Since the mini iso is large enough not to leave much space
> to a 4GB usb stick for persistence,

Indeed, for the 6.5 release I had to make a stripped down micro
ISO locally in order to do it, as we were a couple hundred MB
over budget. The 6.0 would have been possible but was missing
one little tweak.

> I am thinking that given a minimum of a 8GB usb stick, we
> could apply some quota to the default user (something around
> 2-3GB?) so that the system is protected from running out of
> space. This does not solve the problem of system updates though,
> that can easily occupy much of the free space...

and is just as likely the cause of the trouble. I'm not a big fan
of placing artificial restrictions on folks, it's something that
drove me to use FOSS in the first place. I know stock Ubuntu 10.04
pops up a message when drive space is low (saw it the other day),
but I'm not sure about modern xubuntus.


> > Secondly, the university runs a proxy to the Internet.
...
> Hmm, this is tricky. We should check if there is a better
> settings tool to include in Xubuntu.
> I don't see switching to plain Ubuntu as a possible solution.

Each program gets to decide how to do it on its own. If lots
of the programs come from Gnome, and Gnome respects whatever
setting xubuntu has made, then it's good. The main one is to
get firefox to respect it I guess. But there's no way for all
of our Geospatial programs to automatically fall into line,
probably the best is to encourage member projects to respect
http_proxy and ftp_proxy environment variables if they are set.
(then set them by hand in /etc/bash.bashrc or /etc/profile.d/)


> > A final issue is of ways to back-up from the system. I realise
> > that a Live system is not exactly intended for long-term use,
> > and particularly not from a USB drive.

still, it's nice if you work doesn't blow away in the winds.
 
> Indeed, it is more for testing purposes.

I use one for quick access to a secure + familiar setup while,
on the road. Many people could use it for many reasons..

> For long term use, we propose either installing the system
> or using the VM version.

You'd probably want to backup your VMs too, in case the master
VM file get corrupted.

> > However I'd like to be able to use this with students
> > across a period of ~15 weeks. This year I found that
> > the batch of USB drives (Kingston DataTraveler 100 G2
> > 16GB drives) did not perform well, and 3-4 of them have
> > failed (at a low level in the hardware, it seems, so
> > both partitions are lost & the devices are not
> > recognised on the USB bus as mass storage devices (or at
> > all)).

it happens.

> > In some cases of partial failure (e.g. when the booted
> > system has filled and is not bootable) I can mount
> > the drive on another Linux system, mount the squash and
> > casper files and fish around to extract some of the
> > contents, e.g. to copy to another drive before re-imaging
> > with a fresh Live installation. I'd just support the
> > requests to present an easy way to access the Live file
> > system when the stick is plugged into another system, or
> > at least documentation on how to mount the systems.

fwiw, some is already on the wiki, albeit maybe not in the best place:
  http://wiki.osgeo.org/wiki/Live_GIS_Disc_Quick_Start_for_USB#OSGeo_Live_6.5

(somewhere were more notes about it in the wiki from years ago too, not
sure if just burried in a wierd place or deleted in a purge)


the missing piece there might be to backup the /media/liveusb/casper-rw file
(the filesystem overlay ext2 filesystem) and then copy it back onto another
new usb stick which has the live iso already installed to it. Probably
it matters to tell the new install that you want to use persistence, I
suspect you might get away with it if the persistence sizes do not exactly
match.(?)   [untested]

> > I have, for example, yet to find where the PostgreSQL
> > contents are when accessing from a separate system.

the original ones remain compressed and untouched in the base squashfs,
any files that have changed, and any new files are in the ext2 overlay
filesystem, and get substituted for the old ones on the fly.
So for pgsql you'll get a mix, with only the changed/new files in the
overlay fs file (aka '/casper-rw').


> > I hope this helps. I realise the third topic is rather nebulous!

shared experience is helpful.


regards,
Hamish