[GRASS-dev] Re: what if: a new GRASS directory layout?

[Ivan Shmakov]

>>>>> Glynn Clements <glynn at gclements.plus.com> writes:

 >> It seems that the reasonable behaviour would be to make a
 >> back-reference if possible, and issue a warning if not.

 > A warning isn't sufficent. If the GC relies upon back-references, but
 > you can't create one, the data could disappear while a process is
 > trying to use it.

	Still it's an improvement, since currently the data could just
	as well be overwritten.  The transition to tmp/ + rename ()
	alone doesn't help much, as a support file may be renamed while
	the other is being read.

	OTOH, I expect the mapsets being accessible read-only to some
	parties to be used primarily for SELECT, less INSERT, and for a
	negligible amount of UPDATE or DELETE.  At least, this may
	explain why the issue didn't arise before.

 > Essentially, if you can't ensure that the back-references can be
 > created, you can't design a GC which relies upon them.

	Well, yes.  That's why I've suggested some solutions in my
	previous post.  To simplify, they are:

	* hardlink the entire maps into a place where you can create
	  back-references, e. g. the current mapset;

	  Pro's: could be done with `g.copy' (changed); becomes possible
	    as soon as both the ``in place'' and ``atomic replace''
	    issues have been dealt with, no need for an inventory
	    scheme;

	  Cons.: you have to keep your copy in sync; it should be
	    possible to hardlink;

	* hardlink just the objects/; the place may be the current
	  mapset's objects/ directory, as there's nothing wrong with
	  mixing objects related to different mapsets or even different
	  locations within a single directory;

	  Pro's: no need to keep the copy in sync, as there's no copy;
	    the whole thing is transparent to the user;

	  Cons.: requires transition to the inventory scheme; it should
	    be possible to hardlink;

	* disable GC for the mapset in question altogether;

	  Pro's: hard links aren't necessary; it could be even made
	    possible to fetch the objects over, e. g., HTTP;

	  Cons.: the disk space reclamation policy should be figured
	    out, unless the mapset is intended to be completely static.

	The first two solutions effectively split the GC domain in two,
	thus solving the problem.

 >> * no such mirroring is currently possible precisely due to that some
 >> files may be updated in place; thus, I believe the ``in place''
 >> issue has to be resolved irrespective to whether the proposed scheme
 >> will be accepted as a whole or not.

 > I agree with this point at least.

 > Regardless of the layout, the API will need to change. In particular,
 > there would need to be a distinction between creating a support file
 > for a new map, and replacing the support file for an existing map.

 > In all cases, any functions which open elements for writing will need
 > a matching close function to atomically replace any existing file
 > with the new version.

	But note that there's still no way to atomically replace any
	existing /raster/.

 > Fortunately, there isn't a lot of direct file output outside of the
 > libraries:

[...]

	Good.  I'll try to take a glance at them.