[GRASS5] 5.7.1

[Markus Neteler]

Helena,
(cc grass5)

On Sun, Dec 05, 2004 at 04:07:22PM -0500, Helena wrote:
> Markus,
> 
> as I said already all we need for now is to prevent the code to go into 
> swapping and
> freeze the users machines when their file is too big for their memory (this 
> seems to
> be needed for both v.in.sites and v.in.ascii run without flags). 

I don't think that we can prevent GRASS from swapping as it is
done by the operating system (however, better to discuss this in 'grass5' with
the other experts).

I tried v.in.sites on
http://mpa.itc.it/grasstutor/data_menu2nd.phtml
nclidar-utm.tar.gz: Ready to use GRASS LOCATION in UTM (39.5MB) for North Carolina

and it swaps too much for me as well (1GB RAM). There seems to be a memory
leak either in v.in.sites or the underlying DBMI engine.

Radim is out of town currently, and I have zero experience to track
down memory leaks.
Maybe someone skilled to replicate n times a local sites file and
use v.in.sites? No need to download the large nclidar-utm.tar.gz for that.

> If there is
> enough memory, it runs just fine. The problem is that I cannot import the 
> files
> created in 5.3 into 5.7 on the same machine - I am not trying bigger files, 
> I am just asking
> for 5.7 being able to read the same size site file as 5.3 without buying a 
> new computer
> or installing more memory and if that is not possible, have the program say 
> that.

I clearly understand what you wrote.
But if v.in.sites fails, maybe better v.in.ascii?

cat `g.gisenv GISDBASE`/`g.gisenv LOCATION_NAME`/`g.gisenv MAPSET`/site_lists/lidar99 |\
     grep -v '^#' | sed 's+#++g' | sed 's+ %+|+g' | sed 's+ $++g' |\
     v.in.ascii -z out=lidar99 xcol=1 ycol=2 catcol=3 zcol=4 \
                columns='cat int, x double, y double, height double' fs='|'

If this approach works for you, we could even retire v.in.sites and make
above code a script with identical name.

> We will look at the vector code with Jaro and maybe
> others that I work with to see what we can do to get around this issue both 
> for importing the site file (skipping the table? do I need it for sites in a 
> way that
> it is done right now?) and reading the points in v.surf.rst. But this has 
> been an unexpected issue and it is delaying the updates to v.surf.rst
>
> As for trying s.out.ascii | v.in.ascii combination - yes I tried it, as you 
> might have noticed
> in my previous emails - it reads it without problems if I use -t flag.

It should also work without -t flag.

> I will try -z, this may help because there will be no attributes in case I 
> don't have
> variable smoothing parameter. But I really don't want to run any of the 
> v.in.* programs
> until there is an error message and the program ends in some decent way 
> when it runs out of memory and starts swapping.

I assume that there is a (small, but accumulated with large data sets)
memory leak somewhere in the vector engine/DBMI. But as not being real
programmer I don't have the slightest idea how to track this down.

Markus