[pdal] Threading and PDAL

[Connor Manning]

Entwine is multi-threaded.  Most of the PDAL-locking in Entwine happens in
`entwine/util/executor.cpp` and `hpp`.

On Thu, Jul 20, 2017 at 2:02 PM, Jason Beverage <jasonbeverage at gmail.com>
wrote:

> Hi Andrew,
>
> Basically I have a bunch of jobs that read from input files using PDAL, do
> something on them, and then write some resulting points out to an LAZ
> file.  I've got all Stage creation wrapped in a mutex and I was hoping that
> the stage pipelines for each job could run independently without a lock.
> I'm getting random access violations in PDAL.  I'm going to try to build
> pdal in debug and see if I can figure out what is going on.  If I globally
> mutex any access at all to PDAL things run just fine (although not in
> parallel :) ).
>
> Does Entwine use threading or is it a strictly multiprocess setup?
>
> Jason
>
>
> On Thu, Jul 20, 2017 at 10:17 AM Andrew Bell <andrew.bell.ia at gmail.com>
> wrote:
>
>> The stage creation code through the PluginManager and StageManager is
>> believed to be threadsafe.  Nothing else is guaranteed.  We rely on many
>> libraries which may or may not be threadsafe.  If you need thread safety,
>> you need to implement your own locking.
>>
>> If you're getting thread-related crashes in PluginManager or
>> StageManager, please open a ticket.
>>
>> That said, there is probably a lot you can get away with -- we don't rely
>> much on constructs that are inherently non-threadsafe.  I'm willing to look
>> at issues up to a point, but we're not going to work hard to make things
>> threadsafe.  I'd need to know more about specific problems to be helpful.
>>
>>
>> On Thu, Jul 20, 2017 at 10:06 AM, Jason Beverage <jasonbeverage at gmail.com
>> > wrote:
>>
>>> I'm running into some threading issues with PDAL that I could use some
>>> help with.  I'm running into some random crashing using PDAL in a
>>> multithreaded manner.  I understand that PDAL isn't entirely threadsafe,
>>> but do you have some info on what is and isn't threadsafe?
>>>
>>> Basically what I'm trying to do is multiple threads are creating Stages
>>> using a shared StageFactory and each thread is running it's own independent
>>> pipeline.  The only thing that is shared is the StageFactory which is
>>> static for the application.  I've got access to the StageFactory protected
>>> by my own mutex since I was getting crashes accessing it from multiple
>>> threads.  I tried creating a StageFactory per thread but that caused other
>>> issues like stages randomly not being able to be able to be created and it
>>> not loading my plugins all the time.
>>>
>>> Right now I'm using the OSGEO4W 64 bit package to get PDAL and all of
>>> it's dependencies and building my application against that.  It looks like
>>> the pdal version is 1.5.0 which is the latest release.
>>>
>>> Thanks!
>>>
>>> Jason
>>>
>>> _______________________________________________
>>> pdal mailing list
>>> pdal at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/pdal
>>>
>>
>>
>>
>> --
>> Andrew Bell
>> andrew.bell.ia at gmail.com
>>
>
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20170720/0368dc31/attachment.html>