RFC 16: MapScript WxS Services

[Frank Warmerdam]

Tamas Szekeres wrote:
> Frank,
> 
> SWIG Mapscript exposes its functionality as object oriented interface model,
> however the mapserver code is implemented using conventional C code.
> Theoretically when an user invokes a method of an object she expects that
> this call will not violate and interfere with other invocations of other
> objects. With the current implementation one has a good reason fear of it.
> Using locks around global variables will keep up the appearance that the
> threads are separated, but kills the performance and scalability of the code
> and still not separated in effect.

Tamas,

I would claim there is a big difference between fine grained locks
around updates to static stuff (like the maperror.c threadlocal related
lock) and "big locks" such as the one around all GDAL access.  Big locks
deny any other thread access to the service for a lengthy period while
a big bunch of processing is done.  Fine grained locks only exclude
other threads long enough to do a few assignments changing around a
shared data structure.

So, I would claim that fine grained locking, as proposed for maintaining
the threadlocal "msIO" lists will not have a significant effect on performance
or scalability.

> In fact the separated mapscript objects should be really separated. That was
> the primary reason that called RFC-15 to be created. 

I'm not sure what you mean by "mapscript objects should be really separated".
To me RFC 15 is a survey of perceived multi-thread problems, with plans to
fix some of them.  In general we need to be more serious about multi-threading
issues in MapServer if we want to satisfy the multi-threaded user base.

>> Note, I have proposed making the msIO handers thread local, so that each
>> thread can have independent handling of output routing.
> 
> Using thread locals to separate object invocations as a replacement of the
> global ones are considered as a makeshift and -in quality- highly lags
> behind using local contexts. For example one day if the folks will find out
> the need to pass objects between threads we will be between hell and
> high-water. It is not too unrealistic because some of the application models
> will use different threads between subsequent requests without having the
> control of the developer. Developers might want to maintain the state
> between requests.

I will conceed that having an IO context would be highly preferrable to
using a thread-local IO context if we ever conceive of wanting this sort
of stuff to move smoothly from thread to thread.  In general, if I were
engineering MapServer from scratch I would be inclined to have IO
contexts.  However, in our current situation I'm not prepared to address
that level of re-engineering.  If you can convince the TSC that you
have a plan, are willing to do it, and will support MapServer through
the follow on problems, then go for it.  But it's a bridge-too-far for
my purposes.

I would add that the io context would need to be passed in a lot of
mapscript code, and that we (the mapserver core developers) have no easy
way to update the many mapscript applications out there in the world.

>> This would require changing and reviewing and modifying an enormous
>> amount of code.  For me it just is not on the table.  And I am not clear
>> on what the benefit would be.
> 
> I don't think so. The possible solution is inside the code since it is
> prepared to use contexts along with the io operation. To achieve the
> requested behaviour the *only* following steps should be implemented.
>  
> 1. OWSDispatch should initialize the context (msIOContext) adding a memory
> writer as readWriteFunc (instead if msIO_stdioWrite). Context.cbData should
> hold the accumulated result. 
> 2. Find and replace msIO_printf -s with for example msIO_printf_ctx -s
> having the context as a parameter. (not all of the files having msIO_printf
> should be modified eg. mapserv.c, cgiutil.c)

Well, it is this step 2 that frightens me.  I would add that some of the
output occurs in places like msSetError().  All sorts of functions at
various levels would have to be re-engineered to pass around this io
context.   It could be done, but I claim it is hard.

> Yeah, so we have the possibility to put at least one portion of the code
> into a good shape, it would be unwise to loose it.
 >
> Frank, since I am determined to carry on RFC-15, i am ready to dig into this
> issue in more deatail to help - if needed. 

I am keen to see RFC-15 carried through, and am willing to do some of
the work once the plan is made more specific.  But I only see that as
loosely related to RFC 16, and I am not prepared to address IO contexts.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org