[GRASS-dev] Re: [GRASS-user] problemsusing r.proj with large dataset

[Jerry Nelson]

I did a cvs update around midday US CST on Wed (ie a couple of hours before
I sent this email). I first ran r.proj. This time the last message I get is
"Killed". With r.proj.seg I get the "File size limit exceeded" error.

In your email you said to change the Makefile for r.proj, then do a make in
r.proj.seg. When I checked the Makefiles in both places, r.proj had what
looked to be old stuff, with no large file support but r.proj.seg had
exactly what you said to put in the r.proj makefile. So I was wondering if
you really meant to change the r.proj Makefile.

Jerry 

-----Original Message-----
From: Glynn Clements [mailto:glynn at gclements.plus.com] 
Sent: Wednesday, December 20, 2006 9:54 AM
To: Gerald Nelson
Cc: grassuser at grass.itc.it; grass-dev at grass.itc.it
Subject: RE: [GRASS-dev] Re: [GRASS-user] problemsusing r.proj with large
dataset


Gerald Nelson wrote:

> Here's the set of commands I use when I compile and make grass
> 
> CFLAGS="-O2 -mcpu=pentium4 -Wall" LDFLAGS="-s"

> 
>  ./configure \
>            --enable-largefile                                    
>            --with-postgres-includes=/usr/local/pgsql/include/    
>            --with-proj-includes=/usr/local/include/              
>            --with-proj-share=/usr/share/proj/                    
>            --with-postgres-libs=/usr/local/pgsql/lib/            
>            --with-readline-libs=/usr/local/lib/                  
>            --with-readline-includes=/usr/local/include/readline/ \

>            --with-readline=no
> 
> The output from configure says largefile support is enabled. When I 
> look in include/Make/Platform.make, I find
> 
> #Large File Support (LFS)
> USE_LARGEFILES      = 1
> 
>  Is there any way I can tell whether its working when I run grass?

If you can read/write raster maps whose cell/fcell file is >2GiB, then LFS
is working in libgis. For testing, the easiest way to create a large raster
is to use e.g. "r.mapcalc test = rand(-1000000,-1)" with region of >~500K
cells.

[The compression techniques used for integer maps do badly on both random
data and negative numbers, so the cell file will end up using 4 bytes per
cell in the above example.]

If LFS is enabled, the "File size limit exceeded" error from r.proj.seg
suggests that I got the Makefile logic wrong.

Try changing raster/r.proj/Makefile to:

------------------------------------------------------------
MODULE_TOPDIR = ../..

PGM = r.proj.seg

LIBES =  $(GPROJLIB) $(GISLIB)
DEPENDENCIES = $(GPROJDEP) $(GISDEP)
EXTRA_INC = $(PROJINC)

include $(MODULE_TOPDIR)/include/Make/Module.make

ifneq ($(USE_LARGEFILES),)
	EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
endif

default: cmd
------------------------------------------------------------

(i.e. moving the USE_LARGEFILES test to after the inclusion of
Module.make) then recompiling it with:

	cd raster/r.proj.seg
	make clean
	make
	make INST_NOW=y

I'll change it in CVS anyhow; it certainly can't harm, and it may fix the
LFS issue.

> When I did the equivalent using gdalwarp, it took quite a while (like 
> hours).

That's to be expected. It has to evaluate the UTM->Lat/Lon projection
equations for each output cell, which is a fair amount of FP arithmetic; for
method=cubic, it also needs to evaluate a bicubic polynomial for each cell.

--
Glynn Clements <glynn at gclements.plus.com>