[GRASS-dev] Re: [GRASS-user] problems using r.proj with large dataset

[Gerald Nelson]

Here's the set of commands I use when I compile and make grass

CFLAGS="-O2 -mcpu=pentium4 -Wall" LDFLAGS="-s"                                 

 ./configure \
           --enable-largefile                                    
           --with-postgres-includes=/usr/local/pgsql/include/    
           --with-proj-includes=/usr/local/include/              
           --with-proj-share=/usr/share/proj/                    
           --with-postgres-libs=/usr/local/pgsql/lib/            
           --with-readline-libs=/usr/local/lib/                  
           --with-readline-includes=/usr/local/include/readline/ \                            
           --with-readline=no

The output from configure says largefile support is enabled. When I look in include/Make/Platform.make, I find

#Large File Support (LFS)
USE_LARGEFILES      = 1

 Is there any way I can tell whether its working when I run grass?

When I did the equivalent using gdalwarp, it took quite a while (like hours).

Jerry

---- Original message ----
>Date: Tue, 19 Dec 2006 22:30:53 +0000
>From: Glynn Clements <glynn at gclements.plus.com>  
>Subject: RE: [GRASS-dev] Re: [GRASS-user] problems using r.proj with large dataset  
>To: "Jerry Nelson" <gnelson at uiuc.edu>
>Cc: "'Maciej Sieczka'" <tutey at o2.pl>, <grassuser at grass.itc.it>, <grass-dev at grass.itc.it>
>
>
>Jerry Nelson wrote:
>
>> Here's the results of my attempt to use r.proj.seg. I'm using grass6.3cvs,
>> which I updated yesterday (19 Dec). There are some other issues with the
>> update (eg using the gui to do d.vect generates strange error message), so
>> it is possible these results are not a problem with r.proj.seg. Jerry
>
>> Allocating memory and reading input map...
>> File size limit exceeded 
>
>No, this is an r.proj.seg issue.
>
>> r.proj.seg input=AfricaHornElev location=world mapset=PERMANENT
>> output=AfricaHornElev method=cubic --overwrite 
>> 
>> Input Projection Parameters: +proj=longlat +a=6378137 +rf=298.257223563
>> +no_defs
>> Input Unit Factor: 1
>> 
>> Output Projection Parameters: +proj=utm +zone=37 +a=6378137
>> +rf=298.257223563 +n
>> o_defs
>> Output Unit Factor: 1
>> Input:
>> Cols: 22875 (25200)
>> Rows: 27201 (27600)
>
>22,875 * 27,201 = 622,222,875 cells * 4 bytes/cell = 2,488,891,500
>bytes, which will exceed the file size limit unless LFS is enabled
>(--enable-largefile) and working.
>
>I did try to make the code work with LFS, but I may well have gotten something wrong; I haven't tested that aspect yet (I don't have any maps that large).
>
>Is USE_LARGEFILES set in include/Make/Platform.make?
>
>If it is, my first guess would be that this (in r.proj.seg/Makefile):
>
>	ifneq ($(USE_LARGEFILES),)
>		EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
>	endif
>
>should have come *after* this:
>
>	include $(MODULE_TOPDIR)/include/Make/Module.make
>
>OTOH, even if you have LFS ...
>
>> Output:
>> Cols: 17067 (17067)
>> Rows: 21048 (21048)
>
>... 17067 * 21048 = 359,226,216 output cells is a rather large region
>even for "normal" GRASS modules. I would expect that to take a while,
>particularly for method=cubic, but it will certainly make a good test case.
>
>-- 
>Glynn Clements <glynn at gclements.plus.com>
Gerald Nelson
Professor, Dept. of Agricultural and Consumer Economics
University of Illinois, Urbana-Champaign
office: 217-333-6465 
cell: 217-390-7888
315 Mumford Hall
1301 W. Gregory
Urbana, IL 61801