[GRASS5] error encountered in r.terraflow

[Shaun Walbridge]

Do you have an large amount of disk space available on your /var 
partition? The error:
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
indicates you'd run out of space. As mentioned in the help for 
r.terraflow, it requires 80N where N is rows x columns of space to 
compute the output. This can be upwards of 40G for large rasters.

cheers,
Shaun

abhay menon wrote:

> hello all,
>  
> am still facing this problem.... even after updating the patch file..
>
> r.terraflow December 2003
> region size is 13593 x 15215
> cell 960MBdata header compatible with region header
> elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type 
> CELL_TYPE --you should use r.terraflow.short
> STREAM temporary files in /var/tmp  (THESE INTERMEDIATE STREAMS WILL 
> NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE 
> SPACE PLEASE DELETE THESE FILES MANUALLY!)
> SFD (D8) flow direction
> D8CUT=999999986991104.000000
> memory size: 400.00M (419430400) bytes
> Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
> reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
> total elements=206817495, nodata elements=50042031
> largest temporary files:
>    FILL: 10.79G (11581779720) [206817495 elements, 56B each]
>    FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
> Will need at least 23.36G (25084074240) space available in /var/tmp
> ------------------------------
> COMPUTING FLOW DIRECTIONS
> classifying nodata (inner & boundary)
> finding nodata
> relabeling nodata
> MM warning: limit=419430400B. allocating 209608296B.  limit exceeded 
> by 23602B.
> MM warning: limit=419430400B. allocating 209608296B.  limit exceeded 
> by 23633B.
> merging relabeled grid
> MM warning: limit=419430400B. allocating 209567256B.  limit exceeded 
> by 23558B.
> MM warning: limit=419430400B. allocating 209567256B.  limit exceeded 
> by 23589B.
> ----------
> assigning preliminary directions
> ----------
> finding flat areas (plateaus and depressions)
> plateau.cc:starting memscan
> plateau.cc:memscan done
> removing duplicate plateaus
> MM warning: limit=419430400B. allocating 209567260B.  limit exceeded 
> by 23574B.
> :
> MM warning: limit=419430400B. allocating 209567260B.  limit exceeded 
> by 23636B.
> relabeling plateaus
> MM warning: limit=419430400B. allocating 209567260B.  limit exceeded 
> by 23570B.
> MM warning: limit=419430400B. allocating 209567260B.  limit exceeded 
> by 23601B.
> generating plateau statistics
> MM warning: limit=419430400B. allocating 209403100B.  limit exceeded 
> by 23570B.
> MM warning: limit=419430400B. allocating 209403100B.  limit exceeded 
> by 23601B.
> ----------
> assigning directions on plateaus
> MM warning: limit=419430400B. allocating 209649312B.  limit exceeded 
> by 19306B.
> :
> MM warning: limit=419430400B. allocating 209649312B.  limit exceeded 
> by 19368B.
> --------------
> generating watersheds and watershed graph
> creating windows
> warning: using slower scan
> generateWatersheds
> MM warning: limit=419430400B. allocating 209628776B.  limit exceeded 
> by 10438B.
> :
> :
> :
> :
> :
> MM warning: limit=419430400B. allocating 209628776B.  limit exceeded 
> by 11182B.
> file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void 
> makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T = 
> waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err 
> == AMI_ERROR_NO_ERROR' failed.
>  
> -- 
> Best Regrads
>
> Abhay Menon
>
> On 4/28/06, *Shaun Walbridge* <walbridge at nceas.ucsb.edu 
> <mailto:walbridge at nceas.ucsb.edu> > wrote:
>
>     Apologies, I forgot to re-attach the patch to this list.  The only
>     changes I've found necessary are those couple spots in ami_stream.h.
>     We've successfully run Terraflow on 4GB+ datasets after the swap to
>     fseeko/ftello.
>
>     Cheers,
>     Shaun
>
>     --- ami_stream.h        2004-11-09 05:29:58.000000000 -0800
>     +++ ami_stream.h.edit   2006-04-27 10:44: 48.000000000 -0700
>     @@ -394,8 +394,8 @@
>         seek_offset = offset * sizeof(T);
>       }
>
>     -  if (fseek(fp, seek_offset, SEEK_SET) == -1) {
>     -    cerr << "AMI_STREAM::seek offset=" << seek_offset <<
>     "failed.\n";
>     +  if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
>     +    cerr << "AMI_STREAM::seek offset=" << seek_offset << "
>     failed.\n";
>         assert(0);
>         exit(1);
>       }
>     @@ -463,7 +463,7 @@
>
>       assert(fp);
>       //if we go past substream range
>     -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
>     +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>         return AMI_ERROR_END_OF_STREAM;
>
>       } else {
>     @@ -489,7 +489,7 @@
>       assert(fp);
>
>       //if we go past substream range
>     -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
>     +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>         return AMI_ERROR_END_OF_STREAM;
>
>       } else {
>     @@ -512,7 +512,7 @@
>
>       assert(fp);
>       //if we go past substream range
>     -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
>     +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>         return AMI_ERROR_END_OF_STREAM;
>
>       } else {
>     @@ -532,7 +532,7 @@
>
>       assert(fp);
>       //if we go past substream range
>     -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
>     +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>         return AMI_ERROR_END_OF_STREAM;
>
>       } else {
>