Multiple user problem -- C# MapScript Web Application (Is mapserver thread safe?)

[Umberto Nicoletti]

Hi Tamas.

On 8/9/06, Tamas Szekeres <szekerest at gmail.com> wrote:
> Umberto,
>
> Most of us are user and/or developer of mapserver/mapscript so the
> primary objective is to make the code sufficient for using in
> multithreaded environments.(not to blame the code anyway)

So you are saying we should focus on correctness, not performance, at
least for now.

> You are willing to prove the usability and the correctness of the code
> from the experimental side. However i am a bit uncertain how to
> construct these tests and what to test against.

Before we touch the code I think we need to see a proof of error. So
far not a single line of code or a detailed mapserver setup that leads
to a more or less reproducible bug has been showed. We should also
start discussing about what components we repute unsafe: gd, oracle,
postgis, shapefile and run specific tests against it one at a time.

In the meanwhile, why don't you attempt to run your problematic map
file against the java threadtest, set it to start a large number of
threads and run it multiple times? If it crashes we have evidence of a
problem and act upon it, if it doesn't maybe it's only a problem with
csharp.
I forgot to say that also PHP users have reported positively about php
mapscript as a shared module. When used as a shared module php
mapscript should suffer all the threading issues of Java, C# and other
mapscripts (except maybe for Python).

> How can you make sure
> a problem properly manifests in an error. And how to reproduce a real
> production environment when the user is complaining about funky
> segmentation faults for example. I am slightly anxious with this
> issue.

I bet you are, I am like that myself. That's why
- we need to be super-cautious and meticulous in reviewing portions of
existing code that works at least in cgi
- we need proofs

> Therefore the best i can do is to look over the code and see how the
> global variables are handled inside. The problems were identified for
> mapserver at RFC-15 but all of the dependent libraries may cause the
> same problem. If a dependent library is not thread safe
> it will destabilize the mapserver code either. At the first post of
> this thread the problems of gd and proj4 (each one is fairly
> fundamental) were also identified. Now then, we cannot consider the
> code as being thread safe easily. It was noted for example that in
> spite of we have locks around pj_init we would have locks around
> pj_transform as well. The locking strategy of the current
> implementation is rather experimental.
>
> Here is my proposal in the beginnings:
>
> 1. We should implement an alternative  global lock at the SWIG
> interface level first. Only the method invocations should be locked
> the property manipulation is considered as thread safe by me. As a
> benefit the user could easily switch to this level of safety when
> experiencing problems can be dedicated to thread safety. (Benedikt's
> approach)
> Alternatively we could implement a COM STA style single threaded
> model. (However i cannot see the benefits so far). In the latter case
> every method invocation would imply a thread switch. The mapserver
> code would be tampered by the same thread.

Before we do this and throw away all performance with it we need to
have proofs of all the errors that this thread has so far failed to
describe in detail.

> 2. Alternative application models should be suggested for
> safety+performance. (Ned's approach)

This is possible, and it could be used even if mapscript was perfectly
thread-safe.

> 3. All of critical sections of the code should be identified and
> locked properly. I am considering currently not all of the sections
> are handled.
> 4. Global data should be eliminated at all or restricted to use as
> const variables initialized at the startup code. On demand
> initialization of the variables should be eliminated.
>

Agreed, even thouh this can quickly become a daunting task.

Umberto
>
> Tamas
>
>
> 2006/8/9, Umberto Nicoletti <umberto.nicoletti at gmail.com>:
> > All,
> > I think this discussion is not going anywhere without some code and
> > some numbers to look at, otherwise this smells like FUD to me. I don't
> > mean to be harsh, but I think we need to steer this thread into a
> > clear and agreed path if we want to come up with somethin useful to
> > our users (and ourselves).
> >
> > My proposed checklist is:
> >
> > 1) we have java code that instantiates a configurable number of
> > threads and runs them (doing map draws, queries and geos buffer())
> > 2) we can throw a number of maps at it and see if it performs
> > correctly (a diff against a sample image of the generated images is
> > enough) and fast. These maps could be collected from the msautotest
> > suite so that anyone can reproduce the results by himself or at least
> > they should be publicly available to anyone for review and usage
> > 3) I expect mapserver to perform correctly for all the components
> > listed in the thread-safety faq
> > 4) the Java thread-test can be rewritten in C# or any other language
> > we need to test, but they must be kept synchronized so that we can
> > compare the numbers they produce
> > 5) squeeze out perfomance
> >
> > Best regards,
> > Umberto
> >
>