I think that will be the parallel version of the code given in the link. Sorry, I dont have the resources right now to check the validity of the code, my laptop is fried, working from a common terminal.<br><br>Just add -fopenmp flag in the make file for the code (This line is funny, seems like a packed food adv).<br>
<br>I have a small problem in understanding how the <span style="color: rgb(255, 102, 102);">vect</span> variable is being declared... in function <b><span class="code-func">tcholInv</span></b> and <b><span class="code-func">tcholSolveInv.<br>
<br><br></span></b><div class="gmail_quote">On Fri, Jun 19, 2009 at 4:40 PM, Hamish <span dir="ltr"><<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
ok, I had a chance to look using a 1km^2 block of LIDAR data I had around.<br>
(0.5M pts, uses ~ 750mb RAM on 64bit Linux)<br>
<br>
I've never used the v.lidar tools much (I usually just do something<br>
simple with r.in.xyz) so I can't offer too much expert help. But...<br>
<br>
[half of what follows is aimed at developers]<br>
<br>
<br>
John:<br>
> I have recently joined the user group and I'm working on LiDAR data.<br>
> I am running GRASS GIS 6.4.0svn (the one that came out last week) on<br>
> an Intel quad core 6600 with 2gb RAM.<br>
>> OS = XP Pro<br>
><br>
> I have seperated my data into 1000mx1000m which have around<br>
> 1.5 million points per tile.<br>
><br>
> I use the -tbzr on v.in.ascii and v.build after for topology.<br>
> I have also utilised the sqlite db.<br>
<br>
AFAICT topology is not required for v.lidar modules.<br>
If you only use 3D x,y,z data then database is not needed and will just<br>
take up space and memory. (v.in.ascii -z)<br>
<br>
<br>
> After running v.outlier on first and last returns, the time come for<br>
> edge detection.<br>
><br>
> I run this module (v.lidar.edgedetection) and it works (only using<br>
> 25% of cpu as per usual (guessing there is no quad cpu support = shame))<br>
<br>
Nope, awaiting a volunteer -- could be a really nice threading project<br>
for someone.<br>
<br>
- the v.lidar code is reasonably straight forward & compartmentalized<br>
- experimental support for Pthreads is already added to r.mapcalc in<br>
grass 7 as a template, but this could be an even better example as<br>
r.mapcalc is typically hard drive-limited not CPU-limited.<br>
- IIUC there is an OpenMP version of i.atcorr by Yann Chemin which<br>
may provide a OpenMP example.<br>
- not really sure if Pthreads or OpenMP is more appropriate here<br>
<br>
<br>
tcholDec() could probably be split up into multiple threads at the<br>
for(i;;) stage. (??)<br>
<a href="https://trac.osgeo.org/grass/browser/grass/trunk/vector/lidar/lidarlib/TcholBand.c#L6" target="_blank">https://trac.osgeo.org/grass/browser/grass/trunk/vector/lidar/lidarlib/TcholBand.c#L6</a><br>
<br>
see also <a href="http://grass.osgeo.org/wiki/OpenMP" target="_blank">http://grass.osgeo.org/wiki/OpenMP</a><br>
<br>
<br>
<br>
> However, I have not finished a process yet as it writes at about<br>
> 1kb per second, using 1% of cpu.<br>
<br>
I can only assume that is due to page swapping. Running without a DB<br>
and with no topology may help to save quite a bit of memory/time there.<br>
<br>
no idea why it would drop to 1% of CPU if there is no page swapping<br>
going on.<br>
<br>
> As it is writing the file (was at 6Mb after 3hrs), the mem usage is<br>
> on 114Mb with 1.4Gb free at the moment. It is not using the PageFile.<br>
<br>
.... is that at 100%/4(25%) or 1% CPU?<br>
<br>
<br>
> Is it supposed to be this slow? is there a bug? The tile as .txt/.las<br>
> is c.47Mb.<br>
<br>
the vector engine doesn't scale as well as the raster engine (which can<br>
easily handle gigabytes of data), but it should handle 47mb with ease.<br>
<br>
<br>
> Does the edge file equate to a similar size, e.g. will it<br>
> take about 47000 seconds to write the file? Is there a rough percentage<br>
> of the size of edge file compared to original txt file? I notice it is<br>
> also writing to the sqlite.db at the same time.<br>
<br>
no idea about file sizes. In 6.5svn and 7svn I've just added some<br>
verbose messages (add --verbose) and some percent done indicators<br>
so it will seem like less of a black box now.<br>
<br>
It turns out that ~95% of the computational time is spent in the 3-deep<br>
nested for loops of the Tcholetsky decomposition function.<br>
(lidarlib/tcholDec())<br>
<br>
for my data it ran 16 loops of:<br>
{<br>
1. Bilinear interpolation<br>
2. Bicubic interpolation<br>
3. Point classification (fast)<br>
}<br>
<br>
maybe these loops are another thing that could be sent out to different<br>
threads? (not sure if these are multiple passes or independent segments)<br>
Better to concentrate on the library level first? i.e. lidarlib/tcholDec()<br>
or is tcholDec() just written in a very inefficient way?<br>
<br>
<br>
also you can turn on some debug messages to follow the action:<br>
g.gisenv set="DEBUG=1"<br>
<br>
<br>
> Have I done something wrong?<br>
<br>
one thing to check: did you run 'g.region vect=lidar_points res=2 -ap'<br>
before starting? v.lidar.edgedetection seems to use the current<br>
computational region settings for something.<br>
<br>
<br>
> The book and manual says it needs topology for lidar tools to work.<br>
> Does it need the database?<br>
<br>
I don't think it needs either. Where abouts does it say that?<br>
<br>
> v.build and thought a db could help.<br>
<br>
.... I think it just adds overheads to the process.<br>
<font color="#888888"><br>
<br>
Hamish<br>
<br>
<br>
<br>
<br>
<br>
</font></blockquote></div><br><br clear="all"><br>-- <br>JYOTHISH SOMAN<br>MS-2008-CS<br>Center for Security, Theory And Algorithm Research (CSTAR)<br>International Institute of Information Technology<br>Hyderabad<br>India<br>
Phone:+91-9966222626<br><a href="http://www.iiit.ac.in/">http://www.iiit.ac.in/</a><br>--------------------------------------------------------------<br>The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.<br>
George Bernard Shaw<br>--------------------------------------------------------<br><br>