[GRASSLIST:7177] Re: v.in.ascii out of memory
hamish_nospam at yahoo.com
Wed Jun 15 23:42:02 EDT 2005
> I'm trying to import a somewhat large file of 24.8 million LIDAR
> points (866MB) into GRASS 6.1 (11 Jun CVS) using v.in.ascii
Status: (all due to memory limitations [ie leaks])
GRASS 6.0.0's v.in.ascii ~ 100k points
GRASS 6.1-cvs v.in.ascii ~ 3M points (you)
> File format is just a list of easting, northing, and elevation in NC
> I'm using
> v.in.ascii -z input=strip_09.txt output=g6Npts fs=, z=3
try with -t too. No need to create a table if you have no attributes!
> and I get the following:
> Maximum input row length: 34
> Maximum number of columns: 3
> Minimum number of columns: 3
> Building topology ...
> Registering lines: ERROR: G_realloc: out of memory
> There shouldn't be any lines to register. This is just a list of
> points. Any ideas?
I've seen the same thing. Not sure if it has to know everything to
build topology or if it is a memory leak. I think it's a leak.
I did some tests with valgrind back in March. Radim plugged the biggest
leak, but I think "Registering Lines.." still has one too.
see this thread:
> I have imported close to 500 million points in GRASS 5.4 using
> s.in.ascii, but I don't know why things are crashing in the new vector
GRASS 5.4 stored sites in a simple ASCII file-- the limit was whatever
your filesystem/OS could handle. Vector points in GRASS 6 is a 100%
> I have tested v.in.ascii upto 3 million points.
> I'm running on a Fedora Core 3 Linux with a 2.6 kernel, 1GB of
> physical memory and 8GB of swap.
Note you can add more swap memory to Linux on the fly, see:
> At three million points, I noticed that I'm using over 1GB of memory.
> Is it trying to build the entire topology in memory? I don't need to
> build any topology for a set of points. Is there a way to turn this
> In 5.4 I was developing a scalable version of s.surf.rst that could
> interpolate surfaces on over 500 million points, but now I can't even
> import that many points into a vector/site format.
a) Try adding more swap.
b) We need to fix the leak. (more valgrind)
let us know how you get on.
More information about the grass-user