[GRASS-dev] r.viewshed fails on large raster: temp file already exists

[Benedikt Notter]

Hi folks

I was running r.viewshed in GRASS 7.0 under Win7 on a large raster with 11000 rows by 11000 columns. The algorithm ran in external memory mode with temporary files being written to a local directory (on C:). On starting sweeping, two temporary files were created, "STREAM_a06524" and "STREAM_b06524", of which the first never increased in size beyond 0 KB, and the second grew to about 11 GB during sweeping. When sorting events, additional temp files were created, each of about 255 MB size, which were all named in the same logic, i.e. "STREAM_c06524", "STREAM_d06524" etc, with only the letter after the underscore in the file name changing to the next letter in the alphabet. This continued until  the file "STREAM_z06524" had been created, then the algorithm crashed with "File exists" (see sh output below). Obviously it had tried to create a file that already existed, probably named "STREAM_a06524". It seems that either the programmer never counted on more files being necessary than the alphabet has letters, or maybe something gets mixed up in the loop and the code assumes that "STREAM_a06524" does not exist because it never wrote any data to it.

Heres' the sh output:

GRASS 7.0.svn> r.viewshed input=dom_js1_2m at bn output=test_js1_2m_all coordinate
=639965,245032 obs_elev=4 tgt_elev=1.5 stream_dir=C:/tmp/grass_temp_visibility_
files --verbose
Nodata value set to nan
rows=11000, cols=11000, total = 121000000
In-memory memory usage is 12100088000 B (11539 MB), max mem
allowed=524288000 B(500MB)
***** EXTERNAL_MEMORY MODE *****
Intermediate files will not be deleted in case of abnormal termination.
Intermediate location: C:/tmp/grass_temp_visibility_files
To save space delete these files manually!
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
Estimated size active structure:
(key=64, ptr=4, total node=80 B)
Total= 880000 B
Start sweeping.
Computing events...
100%
Sorting events...
ami_single_temp_name: mktemp failed: : File exists
Assertion failed: 0, file ami_stream.cpp, line 97

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.


A screenshot of the temp directory with the created files is attached.

Best regards
Benedikt



________________________________

Dr. Benedikt Notter

INFRAS
Forschung und Beratung
Mühlemattstrasse 45
3007 Bern
Schweiz
________________________________

Tel +41 31 370 19 14
Fax +41 31 370 19 10
benedikt.notter at infras.ch<mailto:benedikt.notter at infras.ch>
http://www.infras.ch<http://www.infras.ch/>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20131230/e7e9a81f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: r.viewshed_error_02.PNG
Type: image/png
Size: 52460 bytes
Desc: r.viewshed_error_02.PNG
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20131230/e7e9a81f/attachment-0001.png>