[GRASS-user] v.clean process killed itselt!?

[Glynn Clements]

Markus Neteler wrote:

> >> > > How should on go about and clean this map?
> >> >
> >> > On a 64-bit system with a lot of RAM. A 32-bit system limits each
> >> > process to a 4GiB address space, some of which is reserved.
> >>
> >> :-)
> >>
> >> My system is 64-bit, CoreDuo 2,53GHz, and I run Ubuntu Intrepid 64-bit.
> >> Unfortunately I can't install more RAM.
> >>
> >> Will more swap space help?
> >
> > I doubt it.
> 
> I wonder if we have a memory leak in the vector library.
> Compare
>  http://trac.osgeo.org/grass/ticket/14
> 
> If it is there bit small normal usage won't trigger it in a significant
> way. But this huge map would do.

Possibly. But with a map this large, you don't need a leak. The raw
data will barely fit into memory, and any per-vertex, per-edge etc
data could easily push it over the limit.

AFAICT from the output and the code, it's dying in Vect_snap_lines(). 

Looking into it more, I don't think that it's a leak; I just think
that it's trying to store an "expanded" (i.e. bloated) version of a
2.7GiB map in RAM on a system which only has 4GiB.

E.g. for each line vertex, it stores a bounding rectangle (actually a
cube, 6 doubles, 48 bytes). If there are 122 million vertices and only
~2 million are centroids, that could be 120 million line segments,
which would be ~5.4GiB.

Then there's the vertices themselves, and it's storing a significant
fraction of those at 2*8+4 = 20 bytes each, which could consume
anything up to 2.4GiB (the extra 4 bytes per vertex accounts for the
difference to the size of the "coor" file).

Add onto that the additional data used for e.g. the current boundary
(which could be most of the map if it's a long, detailed stretch of
intricate coastline), new vertices created during snapping, other
housekeeping data etc and it could easily exceed RAM.

-- 
Glynn Clements <glynn at gclements.plus.com>