[GRASSLIST:944] Re:
Shaun Walbridge
walbridge at nceas.ucsb.edu
Thu Apr 27 14:02:38 EDT 2006
Hi Nagaraj,
This error occurs because currently Terraflow uses the standard C file
stream calls. The permanent solution to this would be to add some
Makefile magic to detect the current platform, and use the appropriate
stream types. The temporary solution is to use the patch I've attached.
You'll need the grass source tree, to apply the patch do something like:
cd /grass6/raster/r.terraflow/IOStream/include
patch < ami_stream.h.patch
cd ../..
make clean
make
Then install this modified terraflow.
Do any GRASS devs have the time and expertise to make terraflow
largefile aware? In my experience this (very minor) patch does the
trick, should be just fseek / ftell which need proper detection
Cheers,
Shaun Walbridge
Nagaraj Thanushkodi wrote:
>Hi All,
>
>I am trying to process 2GB Ascii raster in Terraflow module with GRASS
>version 6.1.cvs.
>
>But it showing following error.
>
>
>
>"r.terraflow elev=newdata filled=fill direction=fdir swatershed=sws
>accumulation=facc tci=tci d8cut=infinity memory=300 STREAM_DIR=/var/tmp
>stats=stats.out -s
>r.terraflow December 2003
>region size is 19954 x 21199
>cell newdata header compatible with region header
>elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE
>--you should use r.terraflow.short
>
>STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
>DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE
>PLEASE DELETE THESE FILES MANUALLY!)
>SFD (D8) flow direction
>D8CUT= 999999986991104.000000
>memory size: 300.00M (314572800) bytes
>Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
>reading data from newdata to stream /var/tmp/STREAM_n3rZJr:
>
>total elements=423004846, nodata elements=153409740
>largest temporary files:
> FILL: 22.06G (23688271376) [423004846 elements, 56B each]
> FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
>Will need at least 44.12G (47376542752) space available in /var/tmp
>------------------------------
>COMPUTING FLOW DIRECTIONS
>classifying nodata (inner & boundary)
>finding nodata
>relabeling nodata
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17734B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17765B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17796B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17827B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17858B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17889B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17920B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17951B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>17982B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>18013B.
>MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
>18044B.
>MM warning: limit=314572800B. allocating 157117944B. limit exceeded by
>24102B.
>merging relabeled grid
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17734B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17765B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17796B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17827B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17858B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17889B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17920B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17951B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>17982B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>18013B.
>MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
>18044B.
>----------
>assigning preliminary directions
>----------
>finding flat areas (plateaus and depressions)
>plateau.cc:starting memscan
>plateau.cc:memscan done
>removing duplicate plateaus
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17726B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17757B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17788B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17819B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17850B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17881B.
>relabeling plateaus
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17718B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17749B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17780B.
>MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
>17811B.
>generating plateau statistics
>MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
>17718B.
>MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
>17749B.
>MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
>17780B.
>MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
>17811B.
>----------
>assigning directions on plateaus
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14518B.
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14549B.
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14580B.
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14611B.
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14642B.
>MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
>14673B.
>--------------
>generating watersheds and watershed graph
>creating windows
>warning: using slower scan
>r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*,
>AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T =
>waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]:
>Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.
>
>
> Any one can help?
>
>Regards
>
>Nagaraj
>
>
>
-------------- next part --------------
--- ami_stream.h 2004-11-09 05:29:58.000000000 -0800
+++ ami_stream.h.edit 2006-04-27 10:44:48.000000000 -0700
@@ -394,8 +394,8 @@
seek_offset = offset * sizeof(T);
}
- if (fseek(fp, seek_offset, SEEK_SET) == -1) {
- cerr << "AMI_STREAM::seek offset=" << seek_offset << "failed.\n";
+ if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
+ cerr << "AMI_STREAM::seek offset=" << seek_offset << " failed.\n";
assert(0);
exit(1);
}
@@ -463,7 +463,7 @@
assert(fp);
//if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
return AMI_ERROR_END_OF_STREAM;
} else {
@@ -489,7 +489,7 @@
assert(fp);
//if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
return AMI_ERROR_END_OF_STREAM;
} else {
@@ -512,7 +512,7 @@
assert(fp);
//if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
return AMI_ERROR_END_OF_STREAM;
} else {
@@ -532,7 +532,7 @@
assert(fp);
//if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
return AMI_ERROR_END_OF_STREAM;
} else {
More information about the grass-user
mailing list