[GRASS5] 5.7.1

[Helena]

Michael Barton wrote:
> Helena,
> 
> Given that Jaro mentioned s.in.ascii bombing out with a dataset half the
> size of mine

I don't think that Jaro had problem with s.in.ascii.

  and you mentioned v.in.ascii bombing out with table creation,
> this makes me wonder about the dataset it is reading, or possibly a recent
> change in GRASS.

we are doing it completely independently and I have different data than he does.
But my data sets were from dos, so I wanted to make sure that is not the problem
and ran v.in.sites on a lidar site file with 1.5 mil points (55MB) that was created by
GRASS5.3 - I should not have done it because it took me one hour just to get it killed.
Radim, if you can prevent it from swapping
and rather give an error message, that would help me a lot for now.

I put the GRASS-generated site file that I tried with v.in.ascii here
  http://skagit.meas.ncsu.edu/~helena/grasswork/grassprobl/lidar99.gz
(you can use it with x,y location, or get the ready to use location here:
http://mpa.itc.it/grasstutor/data_menu2nd.phtml
Jockey's Ridge LIDAR data

If you have some time and could try it on your Mac and let me know how it goes,
that would be great.
(Lubos just bought Mac laptop as several of his physics colleagues
did and he loves it and tries to talk me into getting one too, so if
you have any advice - off grass5 list, I will appreaciate it).

> 
> My data has a total of 15 fields: cat, x,y, and 12 others. All are double
> precision (no text or integer). One file has 133,200 points and the other
> 495,000 points. I had no trouble with either. I read these in with a
> September 2004 version of GRASS 5.7 using v.in.ascii.

I used 5.7 downloaded just few days ago - I wanted to try the new scripts
v.in.sites.all and v.convert.all to get all my data into 5.7 at once.
I am not sure what has changed between sept. and now.

Helena
> 
> Maybe this information will help debug the problem.
> 
> Michael
> 
> 
> 
> 
> On 12/3/04 12:17 PM, "Helena" <hmitaso at unity.ncsu.edu> wrote:
> 
> 
>>Michael Barton wrote:
>>
>>>I know this started out as a s.in.ascii discussion. But it is focused on
>>>GRASS 5.7 and v.in.ascii is the more relevant module to be considering here.
>>
>>s.in.ascii never had this problem, it is the table creation that is causing it
>>(see below).
>>
>>>I've imported hundreds of thousands of points using v.in.ascii. I don't have
>>>a time, but I am remembering about 15-30 minutes for about a half-million
>>>points. I'm using a Mac G5 single processor at 1.8 Mhz and 512 Mb RAM. This
>>>is a nice, but not extremely high powered system. While I'd like it to go
>>>faster, I don't think this is too bad. >
>>>v.in.ascii can also import sites files. Just use a text editor to get rid of
>>>the # and % characters. I haven't used v.in.sites with enough real data to
>>>get a feel for whether or not it is faster. However, v.in.ascii has more
>>>options for controlling how the sites are to be translated to vector points.
>>
>>Michael, thanks for the hint - I tried v.in.ascii on the same data set -
>>with -t option it imported in seconds, without -t it went into swapping and
>>I had to kill it. So apparently it is the table creation that eats-up the
>>memory.
>>Radim, can the table creation be done on smaller subsets of the data to avoid
>>the
>>swapping ? Or maybe there is a simpler solution?
>>
>>Helena
>>
>>
>>>Michael
>>>
>>>
>>>On 12/3/04 5:06 AM, "Jaro Hofierka" <hofierka at geomodel.sk> wrote:
>>>
>>>
>>>
>>>>Radim Blazek wrote:
>>>>
>>>>
>>>>
>>>>
>>>>>5.7 is used more by normal users than by developers and they want stable
>>>>>version in their distributions. If 5.7 is slow with large data sets it
>>>>>does not mean that it cannot become stable version. It is ok with me to
>>>>>write in announcement that 6.0 is not suitable for making dems from
>>>>>datasets > ?00000 points.
>>>>
>>>>I don't think hundreds of thousand points are large datasets. They are
>>>>quite common if you do real projects. I routinely work with millions of
>>>>points. This must be fixed otherwise many people (including me) will
>>>>have to stick to 5.4 for a longer time than previously thought.
>>>>
>>>>Jaro
>>>>
>>>>
>>>>
>>>>-=x=-
>>>>Skontrolovan? antiv?rov?m programom NOD32
>>>>
>>>
>>>
>>>______________________________
>>>Michael Barton, Professor of Anthropology
>>>School of Human, Evolution and Social Change
>>>Arizona State University
>>>Tempe, AZ  85287-2402
>>>USA
>>>
>>>voice: 480-965-6262; fax: 480-965-7671
>>>www: http://www.public.asu.edu/~cmbarton
>>>
>>
>>
> 
> ______________________________
> Michael Barton, Professor of Anthropology
> School of Human, Evolution and Social Change
> Arizona State University
> Tempe, AZ  85287-2402
> USA
> 
> voice: 480-965-6262; fax: 480-965-7671
> www: http://www.public.asu.edu/~cmbarton
>