[mapserver-users] Making Mapserver Multithreaded

[Frank Warmerdam]

Folks,

After some discussions with Steve and Daniel I have done some work on
making MapServer work better in a multi-threaded environment.  This
isn't necessary in a cgi situation, but I gather it can be useful
from the scripting environements sometimes, where it is necessary
to have multiple maps being rendered within one process, launched
from different threads.

In particular MapServer now can keep per-thread errorObj information
and what was the ms_error global is now fetched using msGetErrorObj().

There is also a simple API for getting the thread id and locking/unlocking
mutexes in mapthread.c.  These in turn are used to protect .map file
parsing with a mutex as well as calls to pj_init() as neither of these
are thread safe.

With these changes I was successfully in running shp2img with multiple
threads generating map output at the same time.

A few notes:

  o I haven't yet added logic to configure to control whether threading
    should be supported.  It will be off by default, since it does add a
    bit of overhead, as well as potentially substantial build complexity.

  o I am currently only doing this with Unix pthreads, but it is my
    intention to also implement support on windows using the Win32 threads
    API which I have a bit of experience.  Let me know if this is of
    interest to you.

  o You can't use SHAPEPATH in multi-threaded mode.  It results in chdir()
    calls within mapserver which are not thread local and will cause all
    hell to break loose.

  o I suspect there are lots of items within MapServer that are not really
    thread safe.  For instance, I have not attempted to protect GDAL even
    though I know there are at least a few subtle thread issues with it.
    I have also not protected calls to the PROJ.4 pj_transform() even
    though I don't know whether it is safe or not.

  o My tests were with each thread operating on it's own mapObj (even if
    read from the same .map file).  I don't think we are near allowing
    different threads to work on the same mapObj.

  o I haven't try to build any of the MapScript bindings.  There may be
    some build problems since my changes ... especially related to no
    longer having a global errorObj.

  o As part of this effort I have made msSetError() take variable
    arguments.  You can now do stuff like this:

    msSetError(MS_SYMERR, "(%s):(%d)", "getSymbol()", msyytext, msyylineno);


Anyways, folks interested in multi-threaded MapServers might want to talk
to Steve or I about testing.  Anyone encountering build problems with
MapServer might have to poke me if it is my fault, since I have already
committed my work to date.

Finally, this work is very preliminary (aka experimental).  We likely have
a long way to go before we will have the same level of comfort with the
bulletproof-ness of MapServer operating in a multi-threaded environment
that we have in single threaded mode.


Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent