[gdal-dev] Multithreading with OGR

[Frank Warmerdam]

Martin Dobias wrote:
>>> So my only problem is to avoid concurrent reads from
>>> both main thread and worker thread. How can be this
>>> best accomplished without serializing the access?
>> Your reads are eventually going to be serialized by the disk controller
>> anyway, so what is the problem with serializing the access ?   Just do
>> it on a small enough scale that no thread gets starved.
> 
> You're right, but that is not what I had in my mind, sorry for being
> unclear. When rendering from worker thread, I have to avoid this kind
> of situations (considering one layer):
> 
> 1. worker thread calls layer's ResetReading, then starts issuing
> GetNextFeature calls
> 2. before worker thread is done, main thread calls layer's
> ResetReading and also starts reading features
> 3. both main thread and worker thread do GetNextFeature calls, but
> both get invalid results
> 
> This can happen e.g. when user tries to identify some features on map
> while rendering is in progress. With the "serialization of access" I
> meant serializing the calls: if main thread was going to read
> features, it would first wait until rendering of that layer finishes
> in worker thread. Clearly, with serialized access, there wouldn't be
> much added value when using threading.
> 
> So, my main question is, if opening the same layer for worker thread
> is a safe operation. That is, if both main and worker thread will
> access (with GetNextFeature calls) their own copy of the layer,
> whether they both end up with correct data.

Martin,

Generally speaking it is not safe to have multiple threads making calls
into a single OGRLayer at the same time.  At the very least you will
encounter the "messing up" effect on GetNextFeature(), even if you
serialize calls, and if you don't serialize the calls to the OGRLayer
you will also face potentially severe problems and crashes.

With regard to RFC 16, only a little work in the existing code with
locks was implemented.  None of the TestCapability or GetLayerClone()
work was done.  I still think that the approach in the RFC is reasonable
but I don't currently have funding to pursue the matter.  There was
also some pushback at the time though I don't recall all the details.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent