[GRASS5] Raster directory structure, comments

[Glynn Clements]

Raymond Burns wrote:

> I beleive there is an argument against the directory structure
> pondered in the "features needed for 6.2" thread.
> 
> <mapset>/raster/<mapname>/filetypes
> 
> VS
> 
> <mapset>/raster/<filetypes>/<mapname>
> 
> There does appear to be a limitation on the number of subdirectories
> a directory supports, whereas, there appears to be no realistic limit
> on the number of files a directory can contain.
> 
> No I haven't dug into the code to see where it is but, currently grass
> is limited (by the operating system) to 32000 raster maps per mapset
> due to some hard limit on my systems

Unix OSes impose a limit on the number of hard links to an inode. Each
subdirectory has a ".." entry which adds an extra hard link to the
parent.

You can determine the limit for a given filesystem using:

	pathconf(path, _PC_LINK_MAX)

where "path" is any directory within that filesystem.

> which precludes making a
> subdirectory in cell_misc for map 32001. At the same time I can add
> (touch) additional files in fcell, cellhd, cell, cats ...
> 
> I realize 32000 is not a significant limit for most uses of grass
> today. As we work with more 3D, volume, animation and modeling it will
> probably become more of an issue.

I don't consider that we need to support tens of thousands of maps in
a single mapset.

Apart from the hard limit on the directory's link count, having that
many entries in a directory will adversely affect performance on
filesystems where directories are stored as unsorted lists (requiring
a linear search to look up a filename). It will have an even worse
effect upon the performance of anything which wants a sorted list of
maps.

Consequently, most people dealing with that many maps will probably
want to split the data into multiple mapsets.

-- 
Glynn Clements <glynn at gclements.plus.com>