[pdal] Threading and PDAL

[Jason Beverage]

Hi Andrew,

Basically I have a bunch of jobs that read from input files using PDAL, do
something on them, and then write some resulting points out to an LAZ
file.  I've got all Stage creation wrapped in a mutex and I was hoping that
the stage pipelines for each job could run independently without a lock.
I'm getting random access violations in PDAL.  I'm going to try to build
pdal in debug and see if I can figure out what is going on.  If I globally
mutex any access at all to PDAL things run just fine (although not in
parallel :) ).

Does Entwine use threading or is it a strictly multiprocess setup?

Jason

On Thu, Jul 20, 2017 at 10:17 AM Andrew Bell <andrew.bell.ia at gmail.com>
wrote:

> The stage creation code through the PluginManager and StageManager is
> believed to be threadsafe.  Nothing else is guaranteed.  We rely on many
> libraries which may or may not be threadsafe.  If you need thread safety,
> you need to implement your own locking.
>
> If you're getting thread-related crashes in PluginManager or StageManager,
> please open a ticket.
>
> That said, there is probably a lot you can get away with -- we don't rely
> much on constructs that are inherently non-threadsafe.  I'm willing to look
> at issues up to a point, but we're not going to work hard to make things
> threadsafe.  I'd need to know more about specific problems to be helpful.
>
>
> On Thu, Jul 20, 2017 at 10:06 AM, Jason Beverage <jasonbeverage at gmail.com>
> wrote:
>
>> I'm running into some threading issues with PDAL that I could use some
>> help with.  I'm running into some random crashing using PDAL in a
>> multithreaded manner.  I understand that PDAL isn't entirely threadsafe,
>> but do you have some info on what is and isn't threadsafe?
>>
>> Basically what I'm trying to do is multiple threads are creating Stages
>> using a shared StageFactory and each thread is running it's own independent
>> pipeline.  The only thing that is shared is the StageFactory which is
>> static for the application.  I've got access to the StageFactory protected
>> by my own mutex since I was getting crashes accessing it from multiple
>> threads.  I tried creating a StageFactory per thread but that caused other
>> issues like stages randomly not being able to be able to be created and it
>> not loading my plugins all the time.
>>
>> Right now I'm using the OSGEO4W 64 bit package to get PDAL and all of
>> it's dependencies and building my application against that.  It looks like
>> the pdal version is 1.5.0 which is the latest release.
>>
>> Thanks!
>>
>> Jason
>>
>> _______________________________________________
>> pdal mailing list
>> pdal at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/pdal
>>
>
>
>
> --
> Andrew Bell
> andrew.bell.ia at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20170720/3fa1e98b/attachment.html>