[GRASS5] 5.7.1

[Helena]

Radim,

I did some testing for v.in.sites, importing a file x|y|%z with gradually increasing
number of points. 5000, 8000 point files are no problem, 200000-300000 points (11MB)
reads the sites in seconds, writes the number of points and then does something
for 5 minutes (this is still OK), but with 627000 point file it freezes the machine
and I believe that it is swapping, I could not even kill it and I cannot even get
through the "Transfering sites to vector file" to get the number of points.
If it is swapping, it will depend on the size of your memory when this happens.
I will post some larger point files on the web if you don't have any.

And Jaro is right - this is not some special advanced developers issue -
site files with over million of data points are now a common place and you can
get plenty of them for free on the Web. We cannot have a GRASS release which freezes
your machine with a point file that has the size which is now common. (You may remember
recent emails users talking about tens even hundred million of points).
Below, I have inserted your recent exchange with Jaro - I am wondering whether
there may be something related going on in v.in.sites as we had with reading
point vector data in v.surf.rst.

Helena

 >>>>> >>>Also I had a couple of problems in grass57. Currently I am not sure if
 >>>>> >>>these problems are associated with my version of grass57 or it is more
 >>>>> >>>general. For example, I couldn't read input vector file with ~200K
 >>>>> >>>points,
 >>
 >>> >
 >>> > You cannot read existing vector? Where? In v.surf.rst?
 >>> > What does it mean 'couldn't read'? Does it crash or is it slow?
 >
 >>
 >> It stops without any progress. Smaller files were OK.


It was slow because IL_vector_input_data_2d() was using
db_select_value() for each line/node, which is especially slow
if there is no index for category column in the table.

I have replaced db_select_value() by db_select_CatValArray() +
db_CatValArray_get_value_int/double(), which reduced the time
for vector loading (v.surf.rst until the end of IL_vector_input_data_2d() )
for 200000 points to 42s on 2x1.5GHz. Progress is printed for lines and nodes,
but for now, no way to print progress for (slower) db_select_CatValArray().

Radim


  Blazek wrote:
> Jaro Hofierka wrote:
> 
>>
>>
>> Markus Neteler wrote:
>>
>>>> so I created a small elevation site file - it imported nicely, 
>>>> v.info and v.univar
>>>> worked OK but v.surf.rst interpolated categories instead of fp values
>>>> by default (I guess that is what I need to fix).
>>>> and there are lot of little things, such as various messages e.g.
>>>> Reading lines from vector map ...  100%
>>>> Reading nodes from vector map ...  100%
>>>> that may be confusing for users
>>>>
>>
>> Helena,
>>
>> This combination works for me:
>>
>> v.in.sites input=u_sm output=u2
>> v.surf.rst input=u2 elev=u2 tension=20 zcol=flt1 scol=flt2
>>
>> i.e. it looks that you did not set flt1 and instead of it you 
>> interpolated a category field which is a default option.
>>
>> The real problem is that the devi option doesn't produce a vector file 
>> with atributes. 
> 
> 
> The reason is that sites (map) has no structure defined (only individual 
> sites) so it is impossible to create table when sites (->vector) is 
> opened. It could be possible to create table when first site is written 
> and I'll consider that for 6.0.
> 
>> I don't remember if I fixed this in updated v.surf.rst[.cv]  but it 
>> should work for a CV file). I found there just old code for site format.
>> I am looking now at recent grass5.7 code and I am seeing 2 "site" 
>> commands (s.in.ascii and s.out.ascii). Is it a change in anticipated 
>> transition from old site format to vectors?
> 
> 
> site commands are not compiled and will not appear in stable.
> s.in.ascii and s.out.ascii can only be used for sites-vector
> library testing.
> 
> 
> Radim
> 
> _______________________________________________
> grass5 mailing list
> grass5 at grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass5