[Live-demo] OSGeo-Live 7.0 - big data version?

[Hamish]

Hi all,

So here's how the big data version is going to have to work ...

(sorry, there's really not much room for leeway here since we are
constrained on a number of sides, so it mostly has to be like this for
technical reasons whether we like it or not.)

Project quickstarts should reference the /usr/local/share/data/extra
directory. this will either be a symlink to the real data dir, or a real
directory containing a readme or index.html page telling the user where
to find the data online. Since /usr/local/share/data is symlinked into
$HOME, ~/data/extra/ is another less stable way of referencing it.

At boot time /etc/rc.local checks /cdrom/ to see if a directory called
extra_data/ exists, and set things up accordingly. (the Windows and Mac
installers are dealt with in the same way)
The code for this is now in place.

The entire USB drive will be formatted as one big FAT32 partition.*

The main file is /cdrom/casper/filesystem.squashfs (~3.3gb) That
contains all of the live environment linux files.

The installers and extra_data will live outside the main squashfs,
so take care with uncompressed data (e.g. large TIFFs with no internal
compression) stored there.

[*] ext3 and 4 could only be on the USB if journals are disabled, and/or
with the noatime filesystem mount flag. otherwise you wear a hole in your
flash wear-leveler. So in general if ext FS is to be used on flash media,
for performance ext2 is used. In practice our data dir would be mostly a
read-only affair, so the journal isn't very important, only the access
time stat is. But in any case we are stuck booting from fat32, so fat32
is what we have to use, and the choice of ext version + mount options is
moot. MS Windows does not deal well with with multi-partioned usb sticks,
but 8gb fat32 works well enough for the purpose.


Also, worth repeating/due notice: we are *extremely* space limited on the
core system. Any non-common dataset > ~25mb (compressed) is on thin ice
and is a strong candidate to go into the extra/ data dir.


The files may need to be copied on to the USB master by hand prior to
duplication. I expect we'll dedicate some space at download.o.o or
live.o.o for the collected data files, and allow the users without
the on-disc data to download the datasets they want a la carte from
links in the extra/readme file.


regards,
Hamish