[GRASS5] 5.7.1

[Michael Barton]

Helena,

Given that Jaro mentioned s.in.ascii bombing out with a dataset half the
size of mine and you mentioned v.in.ascii bombing out with table creation,
this makes me wonder about the dataset it is reading, or possibly a recent
change in GRASS. 

My data has a total of 15 fields: cat, x,y, and 12 others. All are double
precision (no text or integer). One file has 133,200 points and the other
495,000 points. I had no trouble with either. I read these in with a
September 2004 version of GRASS 5.7 using v.in.ascii.

Maybe this information will help debug the problem.

Michael




On 12/3/04 12:17 PM, "Helena" <hmitaso at unity.ncsu.edu> wrote:

> Michael Barton wrote:
>> I know this started out as a s.in.ascii discussion. But it is focused on
>> GRASS 5.7 and v.in.ascii is the more relevant module to be considering here.
> 
> s.in.ascii never had this problem, it is the table creation that is causing it
> (see below).
>> 
>> I've imported hundreds of thousands of points using v.in.ascii. I don't have
>> a time, but I am remembering about 15-30 minutes for about a half-million
>> points. I'm using a Mac G5 single processor at 1.8 Mhz and 512 Mb RAM. This
>> is a nice, but not extremely high powered system. While I'd like it to go
>> faster, I don't think this is too bad. >
>> v.in.ascii can also import sites files. Just use a text editor to get rid of
>> the # and % characters. I haven't used v.in.sites with enough real data to
>> get a feel for whether or not it is faster. However, v.in.ascii has more
>> options for controlling how the sites are to be translated to vector points.
> 
> Michael, thanks for the hint - I tried v.in.ascii on the same data set -
> with -t option it imported in seconds, without -t it went into swapping and
> I had to kill it. So apparently it is the table creation that eats-up the
> memory.
> Radim, can the table creation be done on smaller subsets of the data to avoid
> the
> swapping ? Or maybe there is a simpler solution?
> 
> Helena
> 
>> 
>> Michael
>> 
>> 
>> On 12/3/04 5:06 AM, "Jaro Hofierka" <hofierka at geomodel.sk> wrote:
>> 
>> 
>>> 
>>> Radim Blazek wrote:
>>> 
>>> 
>>> 
>>>> 
>>>> 5.7 is used more by normal users than by developers and they want stable
>>>> version in their distributions. If 5.7 is slow with large data sets it
>>>> does not mean that it cannot become stable version. It is ok with me to
>>>> write in announcement that 6.0 is not suitable for making dems from
>>>> datasets > ?00000 points.
>>> 
>>> I don't think hundreds of thousand points are large datasets. They are
>>> quite common if you do real projects. I routinely work with millions of
>>> points. This must be fixed otherwise many people (including me) will
>>> have to stick to 5.4 for a longer time than previously thought.
>>> 
>>> Jaro
>>> 
>>> 
>>> 
>>> -=x=-
>>> Skontrolovan? antiv?rov?m programom NOD32
>>> 
>> 
>> 
>> ______________________________
>> Michael Barton, Professor of Anthropology
>> School of Human, Evolution and Social Change
>> Arizona State University
>> Tempe, AZ  85287-2402
>> USA
>> 
>> voice: 480-965-6262; fax: 480-965-7671
>> www: http://www.public.asu.edu/~cmbarton
>> 
> 
> 

______________________________
Michael Barton, Professor of Anthropology
School of Human, Evolution and Social Change
Arizona State University
Tempe, AZ  85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton