[GRASS-dev] Interested in parallelization of GRASS

[Hamish]

Glynn:
> The biggest advantage of the current format is that skipping entire
> rows (when the region's vertical resolution is coarser than the map's)
> is trivial.

note that is also a great feature when the map only takes up a small
part of the working region, e.g. a stamp sized map in the corner of a
big r.patch or display operation. It flies past the rows which are all
NULLs. (but doesn't help much if the map is tall and thin or diagonal,
and your region is square) ...

> If I was going to change anything about the raster format, it would be
> to allow the data to be partitioned horizontally, so that we can avoid
> reading and decompressing entire rows when the region's east-west
> bounds are only a small portion of the map's.

unrealted aside: Since the old-old bug tracker days there was a wish
to replace the i.rectify code with the same-origin GDALwarp API library,
which is much faster. Last Summer (of Code) Seth's main project was
to add OpenCL capability to gdalwarp (which also allows you to do multi-
core if you don't have a capable GPU). The OpenCL r.sun work was tacked
on to the end of that project.


> For modules which need non-sequential access, I'd suggest replacing
> the segment library with something more efficient, e.g. something like
> the code from r.proj. Or even just a flat file which can be mmap()ed,
> and require the use of a 64-bit platform if you want to run such
> modules on more than ~3 GiB of data.

Although it's an obvious target, I didn't mention the segment library
in the parallelization suggestions as I seemed to recall the
disatisfaction with the current implimentation.

If would be cool if a replacement for the segment library could be
(re)built from the ground up with parallelization in mind.


Hamish:
> > So I try to think of modules which are CPU bound.. the first
> > task is to replace inefficient algorithms with better ones (e.g.
> > Glynn's r.cost work,
>
> Huh? I haven't done anything related to r.cost.

fixing the over-by-one bug in the segment library sped up r.cost by
over 50x in my tests (YMMV). (sometime before 6.4.0RC1)  Not technically
a fix to r.cost, but it sure is nice to have it go heaps faster.
I misspoke a bit; no inefficient algorithm was replaced there.


> If you're thinking of r.grow.distance, that can't be used as a
> substitute for r.cost except in the case of constant cost, as it
> relies upon "distance" being monotonic with respect to x and y.

I still have to play with that, as it could be added as an optional
flag in v.surf.icw (from addons), where r.cost was always the slowest
step. That is a heavy user of r.mapcalc, so will be a good test for
pthreads too.



thanks,
Hamish