[GRASS-user] parallel grass operations

[Glynn Clements]

Dmitriy Tyugaev wrote:

> When I run my script with parallel grass operations (about 12 processes at
> the same time), sometimes I get this error from the g.region module:
> 
> *** glibc detected *** g.region: double free or corruption (out):
> 0x0000000015dcfdd0 ***
> ======= Backtrace: =========
> /lib64/libc.so.6[0x360427245f]
> /lib64/libc.so.6(cfree+0x4b)[0x36042728bb]
> /usr/lib64/libgrass_gis.so.6.4(G__read_Cell_head+0xe5)[0x303ee39eb5]
> /usr/lib64/libgrass_gis.so.6.4(G__get_window+0xbd)[0x303ee23fed]
> /usr/lib64/libgrass_gis.so.6.4(G_get_window+0x148)[0x303ee241d8]
> g.region(main+0x15fb)[0x403a3b]
> /lib64/libc.so.6(__libc_start_main+0xf4)[0x360421d994]
> g.region[0x4021c9]
> ======= Memory map: ========
> 00400000-00409000 r-xp 00000000 08:11 916425443
> /usr/lib64/grass-6.4.1/bin/g.region
> 00608000-0060d000 rw-p 00008000 08:11 916425443
> /usr/lib64/grass-6.4.1/bin/g.region
> 15dca000-15deb000 rw-p 15dca000 00:00 0 [heap]
> 303ee00000-303ee51000 r-xp 00000000 08:11 915563294
> /usr/lib64/libgrass_gis.so.6.4.1
> 
> Tell me please what's the problem?

The error being reported by glibc is due to the code which reads a
region from a file (e.g. the WIND file) being unable to cope with the
file changing while it's being read. First it scans the file to
determine how many lines it contains, then it reads those lines into
memory. If the file grows between the line-counting and the reading,
it ends up using more memory than it has allocated.

Normally, this shouldn't happen.

The real problem is that you're trying to run multiple GRASS commands
concurrently in the same mapset. In general, that won't work.

GRASS doesn't perform any locking on the files within a GRASS
database, so you may end up with one process reading a file while
another process is in the middle of writing it. The most problematic
case is the WIND file, which contains the current region, although
there are others.

If you want to run multiple commands concurrently, you need to take
steps to ensure that this type of conflict doesn't happen. For the
current region, you can use the WIND_OVERRIDE environment variable to
specify a named region which should be used instead of the WIND file. 

Or you can use the GRASS_REGION environment variable to specify the
region parameters (the syntax is the same as the WIND file, but with
newlines replaced with semicolons). With this approach, the region can
only be read, not modified.

Problems can also arise if you read files from another mapset while
another session is modifying those files. The WIND file isn't an issue
here, nor are the files containing raster data (which are updated
atomically), but the various support files may be.

-- 
Glynn Clements <glynn at gclements.plus.com>