[GRASS-dev] r.proj.seg disk space usage problem

[Glynn Clements]

William Kyngesburye wrote:

> I had a chance today to try a VERY large raster with r.proj.seg.   
> 2.7GB raster, 55000x47000 cells.  The memory issue of r.proj have now  
> been moved to a HD space issue.  With 7.75GB free, it chokes at about  
> 75% in the allocating memory stage, while free space slowly drops to  
> zero, with an "Error writing segment file".
> 
> It seems r.proj.seg, for its segmentation, creates an uncompressed  
> copy of the whole raster on the HD (instead of in memory).

Correct.

55000 * 47000 * 4 bytes/cell = 10,340,000,000 bytes (~10GB).

> I'm guessing this has something to do with random access speed in
> rasters?

Correct. Re-projection involves accessing the source data in a
non-linear fashion, so the data needs to be stored in a form which
allows random access.

[We could potentially do something else for the case of
transformations with low levels of rotation and curvature, but I'm not
sure whether the case of not having enough free disk space to hold one
uncompressed map is sufficiently common to make it worthwhile.]

> So I'm back to projecting pieces of the raster, but now so they fit  
> in my disk free space, and patching them together.  Disappointing,  
> but if that's the way it must be, I can live with it.

Buy a larger hard drive. 250GB IDE = £43, 250GB SATA = £46, so that
extra 10GB equates to ~£5 worth of disk space. Which is a lot cheaper
than another 10GB of RAM (which you would need for the same task using
r.proj).

-- 
Glynn Clements <glynn at gclements.plus.com>