[GRASS-dev] lidar tools update in grass7

[Markus Metz]

On Mon, May 3 Soeren Gebbert wrote:
> Hello Markus,
>
> 2010/5/3 Markus Metz:
>>
>> IMHO, it doesn't really make sense to replace a function with another
>> function that's 10+ times slower and produces worse results... If you
>> need a parallelized version and a 10+ core system to achieve the same
>> speed, and the results are not as good as with the original tchol
>> solver, I would suggest to keep using the original tchol solver until
>> the CG solver is faster and produces identical results (also for
>> asymmetrical matrices).
>
> Sorry for my misunderstanding words, i not suggest to use the parallel
> CG algorithm instead of the cholesky. I just wanted to say that i
> tried to find an alternative and failed.
>
Um, don't give up that quickly;-) The cholesky decomposition is the
main bottleneck, it would be great to get it faster!

[snip]
>  I was hoping that there are simpler overlapping strategies
> avoiding edge effects (large overlapping edges which are cut to
> smaller ones when patching the large tiles using a pixel averaging
> strategy ...).

Me too, I tried simpler tiling methods but it did not work. This fancy
method with two overlapping zones and weighing according to distance
from core zone seems to be the only solution to avoid edge effects and
is present nearly everywhere in the C code, both in lidarlib and in
the modules.
>
> [snip]
>> IMHO that would be a complete rewrite of the modules in
>> python. Maybe because I am more familiar with C. Changing the C code
>> so that tiles can be processed in parallel should not be too hard.
>
> I see. That will mean to re-fracture the C-code, putting the
> consolidated tiling algorithms in the lidar library and implement MPI
> parallelism in the lidarlib and each module. Thats IMHO a lot of work
> too. :)

I agree, but at least conceptually it seems relatively easy to me to
process the tiles in parallel.
>
> [snip]
>
> [...] i am not familiar with the lidar tiling
> algorithm. So i thought the size of the overlapping edges can be
> computed in python and used as parameter for r.tileset (parameter
> overlap). After the lidar interpolation, the created tiles can be
> patched by cutting the overlapping edges and using pixel averaging.
> This will, as you mentioned above not be sufficient to avoid edge
> effects.

Pixel averaging is not enough, needed is the sum of the weighed
interpolation values, and the weighing for e.g. v.surf.bspline is done
in lidarlib/raster.c and for v.outlier in v.outlier/outlier.c.
>
> So there will be no good alternative to a native MPI lidar version?

Maybe something like this: calculate subregions in python as done in
lidarlib, with the help of r.tileset plus calculating buffer zones
plus g.region, run the chosen lidar module in parallel for each
subregion set by g.region (can you use different computational regions
in parallel?), do some r.mapcalc stuff to weigh the interpolated
values in the overlapping zones as done in the lidar tools, patch
tiles by summing up with r.series method=sum. The r.mapcalc and
r.series steps will eat up (some of the) saved time I guess. Both
meta- or supertiles in a python script and rewriting current modules
in C to process subregions in parallel is quite a bit of work I guess.
But I am sure it's worth it for very large regions.

Markus M