[GRASS5] r.terraflow patch

[Andrew Danner]

Shaun,

 Thanks! Can someone patch the CVS tree with Shaun's patch? It is
certainly needed for large file support.

-Andy

On Thu, 2006-04-27 at 17:04 -0700, Shaun Walbridge wrote:
> Apologies, I forgot to re-attach the patch to this list.  The only 
> changes I've found necessary are those couple spots in ami_stream.h. 
> We've successfully run Terraflow on 4GB+ datasets after the swap to 
> fseeko/ftello.
> 
> Cheers,
> Shaun
> 
> Andrew Danner wrote:
> 
> >Yes, the problem is related to using fseek and ftell (which are only 32
> >bit) instead of fseeko/ftello which support off_t offsets which can be
> >64-bits long if CPPFLAGS includes -D_FILE_OFFSET_BITS=64
> >-D_LARGEFILE_SOURCE.
> >
> >I'm not sure if this completely fixes the problem. I would have to see
> >the patch, but I thought some of the terraflow types for storing offsets
> >were of type "long" instead of "off_t", so some additional tweaking
> >needs to be done. 
> >
> >I have some familiarity with Terraflow, but I have been concentrating my
> >efforts in developing a new Terraflow-like module that should be more
> >robust and have some additional features.  If you have a patch that
> >works, could you submit it to CVS?
> >
> >If I get some free time, I can look into fixes if things still do not
> >work after the patch. 
> >
> >-Andy 
> >
> >On Thu, 2006-04-27 at 13:38 -0700, Shaun Walbridge wrote:
> >  
> >
> >>My response to the same question, on GRASSLIST:
> >>
> >>This error occurs because currently Terraflow uses the standard C file 
> >>stream calls. The permanent solution to this would be to add some 
> >>Makefile magic to detect the current platform, and use the appropriate 
> >>stream types.  The temporary solution is to use the patch I've attached. 
> >>You'll need the grass source tree, to apply the patch do something like:
> >>
> >>cd /grass6/raster/r.terraflow/IOStream/include
> >>patch < ami_stream.h.patch
> >>cd ../..
> >>make clean
> >>make
> >>
> >>Then install this modified terraflow.
> >>
> >>Do any GRASS devs have the time and expertise to make terraflow 
> >>largefile aware? In my experience this (very minor) patch does the 
> >>trick, should be just fseek / ftell which need proper detection
> >>
> >>Cheers,
> >>Shaun Walbridge
> >>

> plain text document attachment (ami_stream.h.patch)
> --- ami_stream.h    	2004-11-09 05:29:58.000000000 -0800
> +++ ami_stream.h.edit	2006-04-27 10:44:48.000000000 -0700
> @@ -394,8 +394,8 @@
>      seek_offset = offset * sizeof(T);
>    }
>  
> -  if (fseek(fp, seek_offset, SEEK_SET) == -1) {
> -    cerr << "AMI_STREAM::seek offset=" << seek_offset << "failed.\n";
> +  if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
> +    cerr << "AMI_STREAM::seek offset=" << seek_offset << " failed.\n";
>      assert(0);
>      exit(1);
>    }
> @@ -463,7 +463,7 @@
>  
>    assert(fp);
>    //if we go past substream range
> -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>      return AMI_ERROR_END_OF_STREAM;
>    
>    } else {
> @@ -489,7 +489,7 @@
>    assert(fp);
>    
>    //if we go past substream range
> -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>      return AMI_ERROR_END_OF_STREAM;
>      
>    } else {
> @@ -512,7 +512,7 @@
>  
>    assert(fp);
>    //if we go past substream range
> -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>      return AMI_ERROR_END_OF_STREAM;
>    
>    } else {
> @@ -532,7 +532,7 @@
>  
>    assert(fp);
>    //if we go past substream range
> -  if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> +  if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
>      return AMI_ERROR_END_OF_STREAM;
>      
>    } else {