[GRASS-dev] Re: [GRASS-user] problems using r.proj with large dataset

[Glynn Clements]

Jerry Nelson wrote:

> Here's the results of my attempt to use r.proj.seg. I'm using grass6.3cvs,
> which I updated yesterday (19 Dec). There are some other issues with the
> update (eg using the gui to do d.vect generates strange error message), so
> it is possible these results are not a problem with r.proj.seg. Jerry

> Allocating memory and reading input map...
> File size limit exceeded 

No, this is an r.proj.seg issue.

> r.proj.seg input=AfricaHornElev location=world mapset=PERMANENT
> output=AfricaHornElev method=cubic --overwrite 
> 
> Input Projection Parameters: +proj=longlat +a=6378137 +rf=298.257223563
> +no_defs
> Input Unit Factor: 1
> 
> Output Projection Parameters: +proj=utm +zone=37 +a=6378137
> +rf=298.257223563 +n
> o_defs
> Output Unit Factor: 1
> Input:
> Cols: 22875 (25200)
> Rows: 27201 (27600)

22,875 * 27,201 = 622,222,875 cells * 4 bytes/cell = 2,488,891,500
bytes, which will exceed the file size limit unless LFS is enabled
(--enable-largefile) and working.

I did try to make the code work with LFS, but I may well have gotten
something wrong; I haven't tested that aspect yet (I don't have any
maps that large).

Is USE_LARGEFILES set in include/Make/Platform.make?

If it is, my first guess would be that this (in r.proj.seg/Makefile):

	ifneq ($(USE_LARGEFILES),)
		EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
	endif

should have come *after* this:

	include $(MODULE_TOPDIR)/include/Make/Module.make

OTOH, even if you have LFS ...

> Output:
> Cols: 17067 (17067)
> Rows: 21048 (21048)

... 17067 * 21048 = 359,226,216 output cells is a rather large region
even for "normal" GRASS modules. I would expect that to take a while,
particularly for method=cubic, but it will certainly make a good test
case.

-- 
Glynn Clements <glynn at gclements.plus.com>