[GRASS-user] error: "Too many links" with r.in.gdal

[Markus Neteler]

Hi Stefan,

On Tue, May 8, 2012 at 7:05 PM, Stefan Luedtke <sluedtke at gfz-potsdam.de> wrote:
> Dear friends,
>
> I am running r.in.gdal on a RED HAT system, ext3 file system, grass version
> is 6.4.1. Am not the admin of the box, so I donno much about it.
>
> After importing 4 geotiffs with r.in.gdal, each holds ~7300 layers, I got
> the error message "Too many links".
>
> I checked the thread
> http://www.mail-archive.com/grass-dev@lists.osgeo.org/msg08547.html and that
> says something about splitting the mapset. I dont understand all the rest in
> there. By importing 4 of 5 tiffs I almost got the number of 32000 that was
> mentioned in that link too,

The problem is this:

> 7300 * 4
[1] 29200

If you start to add one more (big) multilayer GeoTIFF, you reach the
magic number
of 32000 for the ext3 filesystem. You have to know that a GRASS raster
map is stored in a series of subdirectories. Once you exceed ~ 32000 maps in a
mapset (which is also a directory effectively), you are out of range for ext3:

http://en.wikipedia.org/wiki/Ext3#Functionality
-> "A directory can have at most 31998 subdirectories, because an inode
     can have at most 32000 links."

> the following command echos:
>
> GRASS 6.4.1
> (mekong_latlong):~/grassdata/mekong_latlong/era_interim/cell_misc > ls -l|wc
> -l
> 31999

Exactly (it was also my problem in the past as you have seen in the
previous posting).

> Any suggestions.

Sure, several possibilities;
- convince, if the kernel permits, your admin to update to ext4
filesystem (I did that
   already, no big deal)
- convince, if the kernel permits, your admin to offer a disk
partition with xfs filesystem
  ( we are using this here in FEM for our Terabytes of GRASS data)
- if not possible, simply make 2 (or more) separate mapsets. Thanks to g.mapsets
   you can still have all maps visible (i.e., in the current map
search path), so zero
   efforts when it comes to analysis. In my pre-xfs years, I used to
split time series
   by years.

Nowadays the ext3 mapset is a bit limited when it comes to massive geodata
processing...

Hope this helps,
Markus

-- 
http://gis.cri.fmach.it/neteler/