[GRASS-dev] Re: [GRASS-user] problems using r.proj with large dataset

[Jerry Nelson]

Here's the results of my attempt to use r.proj.seg. I'm using grass6.3cvs,
which I updated yesterday (19 Dec). There are some other issues with the
update (eg using the gui to do d.vect generates strange error message), so
it is possible these results are not a problem with r.proj.seg. Jerry

r.proj.seg input=AfricaHornElev location=world mapset=PERMANENT
output=AfricaHornElev method=cubic --overwrite 

Input Projection Parameters: +proj=longlat +a=6378137 +rf=298.257223563
+no_defs
Input Unit Factor: 1

Output Projection Parameters: +proj=utm +zone=37 +a=6378137
+rf=298.257223563 +n
o_defs
Output Unit Factor: 1
Input:
Cols: 22875 (25200)
Rows: 27201 (27600)
North: 17.996667 (18.000000)
South: -4.670833 (-5.000000)
West: 32.745833 (31.000000)
East: 51.808333 (52.000000)
ew-res: 0.000833
ns-res: 0.000833

Output:
Cols: 17067 (17067)
Rows: 21048 (21048)
North: 1989664.482410 (1989664.482410)
South: -516147.358936 (-516147.358936)
West: -163395.495312 (-163395.495312)
East: 1868393.231623 (1868393.231623)
ew-res: 119.047796
ns-res: 119.052254
Allocating memory and reading input map...
File size limit exceeded 

-----Original Message-----
From: Glynn Clements [mailto:glynn at gclements.plus.com] 
Sent: Sunday, December 17, 2006 7:15 PM
To: Maciej Sieczka; grassuser at grass.itc.it; JerryNelson;
grass-dev at grass.itc.it
Subject: Re: [GRASS-dev] Re: [GRASS-user] problems using r.proj with large
dataset


Glynn Clements wrote:

> > > Is this a problem with large files that I will just have to work 
> > > around or is it something to do with my setup?
> > 
> > Propably the same, very old issue:
> > http://intevation.de/rt/webrt?serial_num=241
> 
> I looked into this a while ago. Unfortunately, you can't use rowio (or 
> a home-grown equivalent), as libgis doesn't allow the projection to be 
> changed while maps are open. So, you have to read the entire input 
> map, close it, change the projection, then write the output map.
> 
> To get around the memory issues, you would first need to copy the 
> relevant portion of the input map to a temporary file, then use a 
> cache backed by that file.

I've added to CVS a modified version of r.proj named r.proj.seg. This
behaves identically to r.proj, but uses a tile cache rather than reading the
entire map into memory.

Currently, each tile is 64 * 64 cells, and the size of the cache is fixed at
2 * (nx + ny) tiles (where nx * ny is the size of the source region in
tiles), which I would expect to suffice for any realistic transformation.

[It will definitely suffice for any affine or "near"-affine transformation.
It would be useful to know what level of distortion needs to be accommodated
in practice.]

Cache replacement is random (i.e. when a new tile is loaded, the tile which
it replaces is selected at random). In practice, random replacement tends to
work almost as well as an "optimal" algorithm, but without the overhead of
maintaining usage statistics, as well as not having any degenerate cases.

Suggestions for improvements and/or performance statistics for "real-world"
usage are welcome.

--
Glynn Clements <glynn at gclements.plus.com>