[GRASS-user] v.to.db/v.what rast on large vector sets

[patrick s.]

Here the example on the North Carolina Workframe using random data. Its 
a Shell-Script to loop the process. Hope this is ok?
Scaling can be done on the variable in the first line. Would be great to 
get feedback on processing time other experience on large datasets.

Patrick


####################################################
RES=100  #resolution
RAST=10  #nr of rasters to match
PTNR=100 #nr of points to create

#____INIT__________________________________________________________
g.region rast=elevation_shade res=$RES -a #to keep extend and change 
resolution
v.random out=pt_grid npoints=$PTNR --o
v.db.addtable map=pt_grid

#generate rasters
i=0
while ((i<=RAST)) #`g.list type=rast`
do
     echo $i
     let i++
     r.mapcalc expr="mymap${i}=rand(1,1000)" --o -s #random raster
done

#____join to points________________________________________________
#add x,y
v.db.addcolumn map=pt_grid columns="x double precision, y double precision"
v.to.db map=pt_grid opt=coor columns="x,y"

#iterate through all rasters
#NOTE This leads does not allow to control for type, if raster is wrong 
coded
#NOTE Raster can be of type CELL (integer), DCELL (double prec.), 
FCELL(single prec.)
for i in `g.list type=rast`
do
     echo "adding layer '$i'"
     eval `r.info -g $i`
     if [ $datatype == "CELL" ]
     then
         v.db.addcolumn map=pt_grid column="$i integer"
     else #DCELL; FCELL
         v.db.addcolumn map=pt_grid column="$i double"
     fi
     v.what.rast map=pt_grid rast=$i col=$i
done;

##########################################################



On 11.08.2015 23:33, Markus Neteler wrote:
> On Mon, Aug 10, 2015 at 9:51 AM, patrick s. <patrick_gis at gmx.net> wrote:
>> Dear grass-users
>>
>> First off all- sorry for "spamming" this user-list recently with questions.
> That's fine!
>
>> I don't know any grass-users that I could ask. So this list is the only
>> feedback I can get for support and I am happy it works so well.  So my
>> Thanks to all of you, for the recommendations given!
>>
>> Now my issue:
>> I have a dataset with 30Mio Vectorpoints that need to get attributes added:
>> the coordinates and the values of approx.60 rasters (resolution of 25m
>> across Switzerland). While the processing of the raster-data was fast, this
>> final join is very slow. Only adding x and y -coordinate is at 14% after 15h
>> of processing; v.to.db currently still using 100%CPU. Is this expected
>> behavior or an error of my system (Grass70 with SQLITE, Ubuntu 15.04)?
> For easier testing, could you provide a simple cmd line example?
> Ideally with the North Carolina dataset?
>
> To simulate 30 mio Vector points, just set the raster resolution to cm
> or the like.
> An example with way less points is also fine, scaling it to more
> points is easier than writing it from scratch...
>
> thanks
> Markus