[Live-demo] please avoid storing huge files in SVN

[Hamish]

Cameron wrote:
> I'm open to moving large files out of svn, but we should
> set up a consistent policy on how we do this, which we
> describe in our wiki somewhere.

sure. added a bit to to the build page about this.

> I would like to be able to have versions of big images like
> cd-covers or desktop backgrounds.
> We all will be updating these files, and I'm sure there
> will be times when we want to roll back to prior versions,
> or find out who changed a version, so that we can talk with
> them and understand their motives, or similar.

no argument with that, but 68mb photoshop files do sting a bit.
especially because the local svn checkout keeps two copies of
it. (one for diffs is held in .svn/)
They take up precious dvd space and at least for me with my
might-as-well-be dial-up bandwidth limitations makes a full
`svn up` unusable.

fwiw, I'd prefer if we could keep with oodraw, inkscape, gimp
formats for the graphics sources.


> I'm also uncomfortable throwing out alpha versions of our
> ISO builds. I think that will be a problem for us some time
> in the future. Someone will say "But it worked in alpha3,
> what changed which caused it not to work in the final
> release?"

being able to test against known deltas is a good thing, yes.
'til now we've had to share resources though, so it wasn't
always practical.

> It is possible we should be considering moving to some
> project hosting infrastructure, like google code or
> sourceforge in order to get the disk space we require.

I'd really rather avoid fragmentation if we can. I don't know
about google code but I do know that SF frowns on massive file
dumps. Releasing files there is also a 1 way door, you can't
remove or clean them up them later, only issue an update.

anyway, Alex is doing a great job sorting out the new download
server and things should be well clear once July/August rolls
around. In a pinch Frank has offered us whatever space we need
on his server space as well, but I don't think we'll need it.

> > Also, large files are a one way door in growing the
> > backend database on the svn server. (ungood, but less-
> > worse than the effect was with CVS)
>   
> You mentioned similar concerns about moving documentation
> from html to odt,

right, although after reading more, svn maintains binary diffs,
so it isn't as wasteful as CVS and I thought it was. compared to
the large photoshop images a megabyte tutorial doc is nothing.

> which is an issue that I think we should reopen some time
> soon in preparation for a 4.0 release.

I agree. I'm still very much in favour of a few pages of HTML
and wget'ing in full PDF tutorials & cookbooks, as it's the
only efficient way to efficiently bulk manage, review, and
update so many different docs. If it's not automated and
efficient, we'll drown in the work.

For anything more substantial, we've got to somehow convince
member projects and FOSS4G workshop/tutorial folks to get their
content written and in to us well early of the conference.
Which means getting onto it ASAP..


> > ps- double check that svn:mime-types are set correctly.

(I mentioned that because svn diff went nuts on a pdf it thought
was a text file)

> I have to confess that I haven't learned up on mime types
> (or maybe I did once, but have forgotten). I know you have
> mentioned this a few times. It would help me if there were a
> few words on this in the wiki somewhere which I could check
> back on when I commit.

ok:   http://wiki.osgeo.org/wiki/Live_GIS_Build#See_also
which points to
https://trac.osgeo.org/grass/wiki/HowToSVN#Fileproperties

As this is a total PITA I wrote a script called
module_svn_propset.sh which takes care of some of the common
chore work semi-automatically.

see /etc/mime.types for common options and some advice.


thanks and have a nice weekend,
Hamish