[GRASS-dev] r.viewshed fails on large raster: temp file already exists

[Markus Neteler]

Hi Benedikt,

thanks for your detailed problem report:

On Mon, Dec 30, 2013 at 9:27 AM, Benedikt Notter
<benedikt.notter at infras.ch> wrote:
> Hi folks
>
> I was running r.viewshed in GRASS 7.0 under Win7 on a large raster with
> 11000 rows by 11000 columns. The algorithm ran in external memory mode with
> temporary files being written to a local directory (on C:). On starting
> sweeping, two temporary files were created, “STREAM_a06524” and
> “STREAM_b06524”, of which the first never increased in size beyond 0 KB, and
> the second grew to about 11 GB during sweeping. When sorting events,
> additional temp files were created, each of about 255 MB size, which were
> all named in the same logic, i.e. “STREAM_c06524”, “STREAM_d06524” etc, with
> only the letter after the underscore in the file name changing to the next
> letter in the alphabet. This continued until  the file “STREAM_z06524” had
> been created, then the algorithm crashed with “File exists” (see sh output
> below). Obviously it had tried to create a file that already existed,
> probably named “STREAM_a06524”. It seems that either the programmer never
> counted on more files being necessary than the alphabet has letters, or
> maybe something gets mixed up in the loop and the code assumes that
> “STREAM_a06524” does not exist because it never wrote any data to it.

I suspect moreover that the tmp file name creation under Windows has
some issue. The names generated on Linux are random as they should be:

To check, I have tried the "same" command using the new EU DEM (just
as a test, I used the entire tile):

# set big region:
GRASS 7.0.svn (eu_laea):~ > g.region rast=eu_dem_25m_northern_italy -p
projection: 99 (Lambert Azimuthal Equal Area)
zone:       0
datum:      etrs89
ellipsoid:  grs80
north:      3000000
south:      2000000
west:       4000000
east:       5000000
nsres:      25
ewres:      25
rows:       40000
cols:       40000
cells:      1600000000

# get some coordinates for viewer:
g.region -c
center easting:  4500000.000000
center northing: 2500000.000000

# check available space:
[neteler at blade16 ~]$ df -h /storage/local/rviewshed_tmp/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       228G   27G  201G  12% /storage/local

# create temp dir:
GRASS 7.0.svn (eu_laea):~ > mkdir /storage/local/rviewshed_tmp

# run the job, measure job duration while we are at it:
GRASS 7.0.svn (eu_laea):~ > time -p r.viewshed
input=eu_dem_25m_northern_italy output=test_js1_2m_all
coordinate=4500000.0,2500000.0 obs_elev=4 tgt_elev=1.5
stream_dir=/storage/local/rviewshed_tmp/ --verbose
Nodata value set to -nan
rows=40000, cols=40000, total = 1600000000
In-memory memory usage is 160000320000 B (152588 MB), max mem
allowed=524288000 B(500MB)
***** EXTERNAL_MEMORY MODE *****
Intermediate files will not be deleted in case of abnormal termination.
Intermediate location: /storage/local/rviewshed_tmp/
To save space delete these files manually!
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
Estimated size active structure:
(key=64, ptr=8, total node=96 B)
Total= 3840000 B
Start sweeping.
Computing events...
ERROR 1: libgrass_I.so: cannot open shared object file: No such file
or directory
ERROR 1: libgrass_I.so: cannot open shared object file: No such file
or directory
   3%
[detached]

# ... using screen to be able to detach/reconnect to the remote terminal

# See what happens in the tmp dir (after 12min, 78% of "Computing events"):
[neteler at blade16 ~]$ l /storage/local/rviewshed_tmp/
total 117439744
-rw------- 1 neteler gis            0 Dec 31 15:40 STREAM_ERIIEd
-rw------- 1 neteler gis 115229065216 Dec 31 15:52 STREAM_K3zGLE

When reaching "Sorting events..."

[neteler at blade16 ~]$ l /storage/local/rviewshed_tmp/
total 157911308
-rw------- 1 neteler gis    429391872 Dec 31 15:56 STREAM_ERIIEd
-rw------- 1 neteler gis 148445779584 Dec 31 15:56 STREAM_K3zGLE
-rw------- 1 neteler gis    261861248 Dec 31 15:56 STREAM_GeVOJ9
-rw------- 1 neteler gis    261861248 Dec 31 15:56 STREAM_OOwDs9
-rw------- 1 neteler gis    261861248 Dec 31 15:56 STREAM_61D4FA
-rw------- 1 neteler gis    261861248 Dec 31 15:56 STREAM_Dmlzau
-rw------- 1 neteler gis    261861248 Dec 31 15:56 STREAM_S9f90O
-rw------- 1 neteler gis    261861248 Dec 31 15:57 STREAM_CRITSv
[...]

So the names on Linux are completely randomized unlike on your Windows box.

The problem appears to be in mktemp() on Windows, i.e. in the file
iostream/ami_stream.cpp" line 85

or two lines above where you get STREAM_a06524, STREAM_b06524,
STREAM_c06524, STREAM_d06524 etc. (process ID used?).

Searching around, I found a similar report:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47439

--> http://msdn.microsoft.com/en-us/library/34wc6k1f%28v=VS.80%29.aspx
"_mktemp can create a maximum of 26 unique file names for any given
combination of base and template values. Therefore, FNZ12345 is the
last unique file name _mktemp can create for the base and template
values used in this example."

Ops.

Potential patch to be re-used:
http://gcc.gnu.org/ml/fortran/2011-03/msg00102/mktemp.diff

Anyone?
Markus