[Gdal-dev] Multi-threading Support

[Chapman, Martin]

Frank,

That rocks!!!  You are the man!  

Martin

-----Original Message-----
From: gdal-dev-bounces at xserve.flids.com
[mailto:gdal-dev-bounces at xserve.flids.com] On Behalf Of Frank Warmerdam
Sent: Monday, May 23, 2005 10:03 AM
To: gdal-dev
Cc: Steve Soule
Subject: [Gdal-dev] Multi-threading Support


Folks,

I have committed a variety of changes to GDAL in support of thread
safety.  On win32 these capabilities should now be utilized by default.
On unix you would need to configure --with-pthread and then hand edit a
definition for CPL_THREADLOCAL into cpl_config.h.  I intend to make the
use on unix smoother when I return.

I have done some fairly intensive tests with multireadtest.exe (in 
gdal/apps) and I have reasonably confidence in multi-threaded use within
the the following restrictions:

 o Any given GDALDataset is used only from one thread.
 o The GDAL driver list should be initialized only once (ie. a
    call to GDALAllRegister()). 
 o Currently error handlers are per-thread, so if you want to install
    a custom error handler, you will need to do this for each thread
    depending on it. 
 o The block cache is shared amoung all threads. 
 o My testing has been mostly in a read-only environment.  I don't
    see any obvious problem with writing, but it hasn't been tested
    in a multithreaded environment. 
  o I have not reviewed the OGRSpatialReference related code for
     threadsafety yet.  

The code is incorporated in CVS now.  Some changes went in this morning,
so last night's cvs snapshot will still be buggy.  

For background, I have made a number of additions to cpl_multiproc.cpp/h
including the addition of a mutex holder class.  

I have also modified the assumptions with regard to CPLAcquireMutex().  
It is now expected to support recursive mutex acquisition.  That is, a 
thread can safely acquire an already held mutex again.  Normal reference

counting rules apply.  I am a bit concerned about how portable this will
be on different pthreads implementations.  Currently I have tested it on
Linux and I had to use the apparently non-standard
PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP mutex initialization. It may be
that additional work will be needed for multi-threaded mutex support on
other platforms. 

The other key change was introduction of the CPL_THREADLOCAL macro.
This attribute for a variable marks it as being thread-local. That is, a
separate instance will exist for each thread.  I have applied this to a
variety of static buffers in gdal/port such as the "last error
message" buffer.   On win32 this evaluates to __declspec(thread). 
On other platforms it resolves to nothing by default, but if you have an
appropriately modern version of gcc you can have it expand to __thread.
I have tested this with gcc 3.3 on Linux successfully. I still need to
develop the appropriate configuration control around 
this definition.

Those with a strong interest in GDAL multi-threaded support are
encouraged to test it over the next two weeks while I am away. However,
as it is all quite new some teething problems are to be expected.  So
take care with the latest CVS version of GDAL.

Best regards,
-- 
---------------------------------------+--------------------------------
---------------------------------------+------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent
_______________________________________________
Gdal-dev mailing list
Gdal-dev at xserve.flids.com
http://xserve.flids.com/mailman/listinfo/gdal-dev