[GRASS-dev] Re: [GRASS-user] problems using r.proj with large dataset

[Glynn Clements]

Gerald Nelson wrote:

> Here's the set of commands I use when I compile and make grass
> 
> CFLAGS="-O2 -mcpu=pentium4 -Wall" LDFLAGS="-s"                                 
> 
>  ./configure \
>            --enable-largefile                                    
>            --with-postgres-includes=/usr/local/pgsql/include/    
>            --with-proj-includes=/usr/local/include/              
>            --with-proj-share=/usr/share/proj/                    
>            --with-postgres-libs=/usr/local/pgsql/lib/            
>            --with-readline-libs=/usr/local/lib/                  
>            --with-readline-includes=/usr/local/include/readline/ \                            
>            --with-readline=no
> 
> The output from configure says largefile support is enabled. When I look in include/Make/Platform.make, I find
> 
> #Large File Support (LFS)
> USE_LARGEFILES      = 1
> 
>  Is there any way I can tell whether its working when I run grass?

If you can read/write raster maps whose cell/fcell file is >2GiB, then
LFS is working in libgis. For testing, the easiest way to create a
large raster is to use e.g. "r.mapcalc test = rand(-1000000,-1)" with
region of >~500K cells.

[The compression techniques used for integer maps do badly on both
random data and negative numbers, so the cell file will end up using 4
bytes per cell in the above example.]

If LFS is enabled, the "File size limit exceeded" error from
r.proj.seg suggests that I got the Makefile logic wrong.

Try changing raster/r.proj/Makefile to:

------------------------------------------------------------
MODULE_TOPDIR = ../..

PGM = r.proj.seg

LIBES =  $(GPROJLIB) $(GISLIB)
DEPENDENCIES = $(GPROJDEP) $(GISDEP)
EXTRA_INC = $(PROJINC)

include $(MODULE_TOPDIR)/include/Make/Module.make

ifneq ($(USE_LARGEFILES),)
	EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
endif

default: cmd
------------------------------------------------------------

(i.e. moving the USE_LARGEFILES test to after the inclusion of
Module.make) then recompiling it with:

	cd raster/r.proj.seg
	make clean
	make
	make INST_NOW=y

I'll change it in CVS anyhow; it certainly can't harm, and it may fix
the LFS issue.

> When I did the equivalent using gdalwarp, it took quite a while
> (like hours).

That's to be expected. It has to evaluate the UTM->Lat/Lon projection
equations for each output cell, which is a fair amount of FP
arithmetic; for method=cubic, it also needs to evaluate a bicubic
polynomial for each cell.

-- 
Glynn Clements <glynn at gclements.plus.com>