<div dir="ltr">Thanks Connor, I just took a look at the executor.cpp.<div><br></div><div>I see locking around stage creation which I already have. I also see explicit locking around the prepare call and the preview, which I'm not doing so I will try adding that.</div><div><br></div><div>Is there anything else that you can think of I might be missing?</div><div><br></div><div>Jason</div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Jul 20, 2017 at 3:08 PM Connor Manning <<a href="mailto:connor@hobu.co">connor@hobu.co</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Entwine is multi-threaded. Most of the PDAL-locking in Entwine happens in `entwine/util/executor.cpp` and `hpp`.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 20, 2017 at 2:02 PM, Jason Beverage <span dir="ltr"><<a href="mailto:jasonbeverage@gmail.com" target="_blank">jasonbeverage@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Andrew,<div><br></div><div>Basically I have a bunch of jobs that read from input files using PDAL, do something on them, and then write some resulting points out to an LAZ file. I've got all Stage creation wrapped in a mutex and I was hoping that the stage pipelines for each job could run independently without a lock. I'm getting random access violations in PDAL. I'm going to try to build pdal in debug and see if I can figure out what is going on. If I globally mutex any access at all to PDAL things run just fine (although not in parallel :) ).</div><div><br></div><div>Does Entwine use threading or is it a strictly multiprocess setup?</div><span class="m_-7730802544547022184HOEnZb"><font color="#888888"><div><br></div></font></span><div><span class="m_-7730802544547022184HOEnZb"><font color="#888888">Jason</font></span><div><div class="m_-7730802544547022184h5"><br><div><div><br><div class="gmail_quote"><div dir="ltr">On Thu, Jul 20, 2017 at 10:17 AM Andrew Bell <<a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The stage creation code through the PluginManager and StageManager is believed to be threadsafe. Nothing else is guaranteed. We rely on many libraries which may or may not be threadsafe. If you need thread safety, you need to implement your own locking.<div><br></div><div>If you're getting thread-related crashes in PluginManager or StageManager, please open a ticket.</div><div><br></div><div>That said, there is probably a lot you can get away with -- we don't rely much on constructs that are inherently non-threadsafe. I'm willing to look at issues up to a point, but we're not going to work hard to make things threadsafe. I'd need to know more about specific problems to be helpful.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"></div></div><div class="gmail_extra"><div class="gmail_quote">On Thu, Jul 20, 2017 at 10:06 AM, Jason Beverage <span dir="ltr"><<a href="mailto:jasonbeverage@gmail.com" target="_blank">jasonbeverage@gmail.com</a>></span> wrote:<br></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I'm running into some threading issues with PDAL that I could use some help with. I'm running into some random crashing using PDAL in a multithreaded manner. I understand that PDAL isn't entirely threadsafe, but do you have some info on what is and isn't threadsafe? <div><br></div><div>Basically what I'm trying to do is multiple threads are creating Stages using a shared StageFactory and each thread is running it's own independent pipeline. The only thing that is shared is the StageFactory which is static for the application. I've got access to the StageFactory protected by my own mutex since I was getting crashes accessing it from multiple threads. I tried creating a StageFactory per thread but that caused other issues like stages randomly not being able to be able to be created and it not loading my plugins all the time.</div><div><br></div><div>Right now I'm using the OSGEO4W 64 bit package to get PDAL and all of it's dependencies and building my application against that. It looks like the pdal version is 1.5.0 which is the latest release.</div><div><br></div><div>Thanks!</div><span class="m_-7730802544547022184m_128268951830316734m_4978355254171190689m_-7906843629220618564HOEnZb"><font color="#888888"><div><br></div><div>Jason</div></font></span></div>
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">_______________________________________________<br>
pdal mailing list<br>
<a href="mailto:pdal@lists.osgeo.org" target="_blank">pdal@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/pdal" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/pdal</a><br></blockquote></div></div><div class="gmail_extra"><br><br clear="all"><div><br></div>-- <br><div class="m_-7730802544547022184m_128268951830316734m_4978355254171190689m_-7906843629220618564gmail_signature" data-smartmail="gmail_signature">Andrew Bell<br><a href="mailto:andrew.bell.ia@gmail.com" target="_blank">andrew.bell.ia@gmail.com</a></div>
</div></blockquote></div></div></div></div></div></div></div>
<br>_______________________________________________<br>
pdal mailing list<br>
<a href="mailto:pdal@lists.osgeo.org" target="_blank">pdal@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/pdal" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/pdal</a><br></blockquote></div><br></div>
</blockquote></div>