[GRASS5] error encountered in r.terraflow
Andrew Danner
adanner at cs.duke.edu
Fri Apr 28 11:19:26 EDT 2006
Also, even if you have a ton of space, if Terraflow has been crashing a
lot, it may leave a bunch of files in /var/tmp which you need to
manually delete. They should probably all look like STREAM_xxxxxx
If you do not have enough space in /var/tmp, but perhaps some other
directory you can use the STREAM_DIR option in terraflow to specify
another location for temp files. Make sure this directory supports
large files. For example, FAT32 file systems can be mounted under
linux/Unix, but cannot support files larger than 2 or 4 GB.
-Andy
On Fri, 2006-04-28 at 08:01 -0700, Shaun Walbridge wrote:
> Do you have an large amount of disk space available on your /var
> partition? The error:
> file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> indicates you'd run out of space. As mentioned in the help for
> r.terraflow, it requires 80N where N is rows x columns of space to
> compute the output. This can be upwards of 40G for large rasters.
>
> cheers,
> Shaun
>
> abhay menon wrote:
>
> > hello all,
> >
> > am still facing this problem.... even after updating the patch file..
> >
> > r.terraflow December 2003
> > region size is 13593 x 15215
> > cell 960MBdata header compatible with region header
> > elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type
> > CELL_TYPE --you should use r.terraflow.short
> > STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL
> > NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE
> > SPACE PLEASE DELETE THESE FILES MANUALLY!)
> > SFD (D8) flow direction
> > D8CUT=999999986991104.000000
> > memory size: 400.00M (419430400) bytes
> > Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
> > reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
> > total elements=206817495, nodata elements=50042031
> > largest temporary files:
> > FILL: 10.79G (11581779720) [206817495 elements, 56B each]
> > FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
> > Will need at least 23.36G (25084074240) space available in /var/tmp
> > ------------------------------
> > COMPUTING FLOW DIRECTIONS
> > classifying nodata (inner & boundary)
> > finding nodata
> > relabeling nodata
> > MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> > by 23602B.
> > MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> > by 23633B.
> > merging relabeled grid
> > MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> > by 23558B.
> > MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> > by 23589B.
> > ----------
> > assigning preliminary directions
> > ----------
> > finding flat areas (plateaus and depressions)
> > plateau.cc:starting memscan
> > plateau.cc:memscan done
> > removing duplicate plateaus
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23574B.
> > :
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23636B.
> > relabeling plateaus
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23570B.
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23601B.
> > generating plateau statistics
> > MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> > by 23570B.
> > MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> > by 23601B.
> > ----------
> > assigning directions on plateaus
> > MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> > by 19306B.
> > :
> > MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> > by 19368B.
> > --------------
> > generating watersheds and watershed graph
> > creating windows
> > warning: using slower scan
> > generateWatersheds
> > MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> > by 10438B.
> > :
> > :
> > :
> > :
> > :
> > MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> > by 11182B.
> > file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> > r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void
> > makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T =
> > waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err
> > == AMI_ERROR_NO_ERROR' failed.
> >
> > --
> > Best Regrads
> >
> > Abhay Menon
> >
> > On 4/28/06, *Shaun Walbridge* <walbridge at nceas.ucsb.edu
> > <mailto:walbridge at nceas.ucsb.edu> > wrote:
> >
> > Apologies, I forgot to re-attach the patch to this list. The only
> > changes I've found necessary are those couple spots in ami_stream.h.
> > We've successfully run Terraflow on 4GB+ datasets after the swap to
> > fseeko/ftello.
> >
> > Cheers,
> > Shaun
> >
> > --- ami_stream.h 2004-11-09 05:29:58.000000000 -0800
> > +++ ami_stream.h.edit 2006-04-27 10:44: 48.000000000 -0700
> > @@ -394,8 +394,8 @@
> > seek_offset = offset * sizeof(T);
> > }
> >
> > - if (fseek(fp, seek_offset, SEEK_SET) == -1) {
> > - cerr << "AMI_STREAM::seek offset=" << seek_offset <<
> > "failed.\n";
> > + if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
> > + cerr << "AMI_STREAM::seek offset=" << seek_offset << "
> > failed.\n";
> > assert(0);
> > exit(1);
> > }
> > @@ -463,7 +463,7 @@
> >
> > assert(fp);
> > //if we go past substream range
> > - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> > + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> > return AMI_ERROR_END_OF_STREAM;
> >
> > } else {
> > @@ -489,7 +489,7 @@
> > assert(fp);
> >
> > //if we go past substream range
> > - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> > + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> > return AMI_ERROR_END_OF_STREAM;
> >
> > } else {
> > @@ -512,7 +512,7 @@
> >
> > assert(fp);
> > //if we go past substream range
> > - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> > + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> > return AMI_ERROR_END_OF_STREAM;
> >
> > } else {
> > @@ -532,7 +532,7 @@
> >
> > assert(fp);
> > //if we go past substream range
> > - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> > + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> > return AMI_ERROR_END_OF_STREAM;
> >
> > } else {
> >
More information about the grass-dev
mailing list