[GRASS5] [bug #2903] (grass) v.in.ascii for larger data sets with attributes does not work

[Request Tracker]

this bug's URL: http://intevation.de/rt/webrt?serial_num=2903
-------------------------------------------------------------------------

Subject: v.in.ascii for larger data sets with attributes does not work

Platform: GNU/Linux/i386
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: 5.7 CVS updated 20040111

v.in.ascii has a memory problem for dbf option for data sets that include
500000 points or more.
As an example -
it reads 50000 points for all options in about 10 seconds
it reads 200000 points in minutes  using cca 100MB memory,
it cannot read 500000 points: it fills 1.5GB memory and starts filling swap too.
I was unable to get it finish importing any file over 500K points
after many hours and the machine gets frozen.

The problem occurs also with -z option if there are 4 columns in the file,
the third one is ID (or category) - such files are created by s.in.ascii in 5.4
Examples of runs that won't finish
v.in.ascii -z testlid560K.ascii out=testlid560Kxyz z=4 fs=space
v.in.ascii testlid560K.ascii out=testlid560K fs=space

For a 280000 point data set it is slow but it does the job:
This takes 30 minutes:
v.in.ascii -z atmlid99280K.ascii out=atmlid99280Kxyz z=4 fs=space
Maximum input row length: 45
Maximum number of columns: 4
Minimum number of columns: 4
column: 1  type: double
column: 2  type: double
column: 3  type: integer
column: 4  type: double
Building topology ...
282294 primitives registered
...etc

The same file without attributes takes 40 seconds to read
v.in.ascii -t atmlid99280K.ascii out=atmlid99280Kxy fs=space
Maximum input row length: 45
Maximum number of columns: 4
Minimum number of columns: 4
Building topology ...
282294 primitives registered
...etc

The test data are available at http://mpa.itc.it/grasstutor/data_menu2nd.phtml
Jockey's ridge LIDAR data, but a sequence of test data can be easily created
by applying r.random to any DEM(e.g. spearfish) in GRASS5.4.

Helena

-------------------------------------------- Managed by Request Tracker