<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi,</p>
<p>just wanted to point that if you are interested in a "framework"
for submit jobs to a thread pool, I can point to<br>
</p>
<p><a class="moz-txt-link-freetext" href="https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.h">https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.h</a></p>
<p><a class="moz-txt-link-freetext" href="https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.c">https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.c</a></p>
<p>which is a port in C I've done from the equivalent C++ code of
GDAL
(<a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/blob/master/port/cpl_worker_thread_pool.h">https://github.com/OSGeo/gdal/blob/master/port/cpl_worker_thread_pool.h</a>).</p>
<p>It has a pthread and Win32 implementation. It could be easily
extracted from libopenjp2 (pending a opj_ -> grass_ renaming to
avoid conflicts if both are combined)</p>
<p>The high level API is the <span class="pl-c1">opj_thread_pool_*
one.<br>
</span></p>
<p>Probably not super fancy, but serves my need well. The user is
responsible for selecting the number of threads and splitting the
work load in jobs that are queued to the thread pool and consumed
by the threads as soon as they are no longer busy.</p>
<p>Even<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">Le 14/10/2022 à 11:02, Moritz Lennert a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:4d8d4d6b5c02c026044def3ecb825b8f@posteo.de">Am
09.10.2022 20:45 schrieb Brad ReDacted:
<br>
<blockquote type="cite">Those variables would be...? Is this
documented somewhere? If not, can
<br>
we get it documented?
<br>
</blockquote>
<br>
<a class="moz-txt-link-freetext" href="https://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly">https://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly</a>
<br>
<br>
<blockquote type="cite">
<br>
OpenMP can do far more than just loop "unrolling", these days.
Tasking
<br>
sections to run concurrent is also quite trivial. It can also
offload
<br>
to GPU, etc. Check out the v4.5+ spec. It's pretty impressive. I
<br>
believe it can do most of what pthreads does, but you certainly
lose
<br>
control of implementation details. Some compilers have an omp
library
<br>
while others convert to pthreads.
<br>
<br>
I do find myself rewriting algorithms so that OpenMP can handle
them.
<br>
It doesn't seem to handle nested loops with breaks very well and
I'm
<br>
not entirely sure why.
<br>
<br>
On 10/9/2022 10:43 AM, William Hargrove wrote:
<br>
<blockquote type="cite">Can still run GRASS outside the shell by
setting all of the environment variables appropriately ...
<br>
<br>
OpenMP just works by "unrolling" all of the determinate loops,
i.e., the ones that iterate a fixed number of times. No
speedups to anything else.
<br>
<br>
Speedup from OpenMP will be limited, depending on the number
of determinate loops present, and how much of the load they
represent.
<br>
<br>
pthreads are totally flexible, but the programmer has to
specify everything, very carefully ...
<br>
<br>
But pthreads can speed up lots of stuff outside of determinate
loops ...
<br>
<br>
<br>
HTH,
<br>
<br>
Bill H.
<br>
<br>
<br>
On 10/9/2022 12:37 PM, Brad ReDacted wrote:
<br>
<blockquote type="cite">Hello,
<br>
<br>
I'm working on adding parallelism to modules, but debugging
is turning out to be a logistical nightmare:
<br>
<br>
Why do I not get any reporting from GCC option
'-fsanitize=address|thread"?
<br>
<br>
I am also having trouble getting the profiler to work
properly inside GRASS (I assume due to shell?). The gmon.out
file produced has no usable data.
<br>
<br>
OpenMP is extremely poorly supported by most tools. valgrind
with helgrind reports a lot of nonsense. I can't seem to get
the Intel linux tools to work properly, either.
<br>
<br>
BTW, we are supporting both pthreads and OpenMP. While this
isn't an issue in most cases, there can be races and
deadlocks if not handled properly. Pthreads aren't entirely
portable. OpenMP is. However, pthreads gives us a more
control. May I suggest using OpenMP for most modules and
reserve Pthreads to libraries, etc? Or should we start
moving away from pthreads?
<br>
<br>
Any suggestions would be greatly appreciated!
<br>
<br>
<br>
</blockquote>
<br>
</blockquote>
--
<br>
Best Regards,
<br>
-Brad
<br>
<br>
_______________________________________________
<br>
grass-dev mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:grass-dev@lists.osgeo.org">grass-dev@lists.osgeo.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/grass-dev">https://lists.osgeo.org/mailman/listinfo/grass-dev</a>
<br>
</blockquote>
<br>
_______________________________________________
<br>
grass-dev mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:grass-dev@lists.osgeo.org">grass-dev@lists.osgeo.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/grass-dev">https://lists.osgeo.org/mailman/listinfo/grass-dev</a>
<br>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>