[GRASS-dev] Re: what if: a new GRASS directory layout?

[Ivan Shmakov]

>>>>> Glynn Clements <glynn at gclements.plus.com> writes:

 >>>>> It seems that the reasonable behaviour would be to make a
 >>>>> back-reference if possible, and issue a warning if not.

 >>>> A warning isn't sufficent. If the GC relies upon back-references,
 >>>> but you can't create one, the data could disappear while a process
 >>>> is trying to use it.

 >>> Still it's an improvement, since currently the data could just as
 >>> well be overwritten.  The transition to tmp/ + rename () alone
 >>> doesn't help much, as a support file may be renamed while the other
 >>> is being read.

 >> Thinking on it further, the change to the
 >> LOCATION/MAPSET/raster/RASTER/ scheme doesn't actually improve
 >> the situation, as the directory may be renamed while the other
 >> party reads the support files

 > That can be handled by locking, and it can probably be done without
 > significant changes to the API.

	Oh, I see that you mean fcntl () locks here.  But weren't there
	issues with them on certain OS' (W32) and FS' (NFS, in
	particular with -o nolock)?

[...]

 > The module would write any support files after the last call to
 > G_put_raster_row() but before the G_close_cell() call. When opening
 > for write any element of a map which is open for write, the call would
 > open a new file in the output map's temporary directory. The
 > G_close_cell() call would lock the destination; upon acquiring the
 > lock, the entire directory would be replace with the new version, and
 > the lock released.

 > Also, locking doesn't have any permission issues; if you can read a
 > file, then you can acquire a lock on it.

	So, a read lock could be used to create a ``reference'' for the
	matters of GC.

 >>>> Essentially, if you can't ensure that the back-references can be
 >>>> created, you can't design a GC which relies upon them.

 >>> Well, yes.  That's why I've suggested some solutions in my previous
 >>> post.  To simplify, they are:

 >>> * hardlink the entire maps into a place where you can create
 >>> back-references, e. g. the current mapset;

 >>> Pro's: could be done with `g.copy' (changed); becomes possible as
 >>> soon as both the ``in place'' and ``atomic replace'' issues have
 >>> been dealt with, no need for an inventory scheme;

 >> Actually no, as long as the raster might be replaced while `g.copy'
 >> is being run.  And since the other mapset is read-only, there's no
 >> way to prevent it.

 >> It thus seems that among the RASTER/ and the inventory scheme, only
 >> the latter provides good support for concurrency, and not the
 >> former.

 > Even with no changes to the directory structure, you can eliminate
 > the race conditions with enough locking. A single directory just
 > makes locking simpler.

 > The inventory system can prevent inconsistencies without any locking.
 > However, there's still the potential for updates to be discarded if
 > you don't have locking. I.e. if two process attempt to update a
 > module concurrently, the update which completes first will be lost.

	Agreed.  Though I'm not sure that the only reasonable way to
	prevent this is to make any concurrent write access to fail.

	FWIW, the appropriate locking to a writable mapset could be
	implemented without fcntl ().