[mapserver-dev] Thread safety [was: python mapscript and msCleanup issues]

[Even Rouault]

Selon Stephan Meißl <stephan at meissl.name>:

> Hi all,
>
> after some further investigations we think the issue is the non-thread
> safe usage of VSI IO in WCS 1.1 and 2.0. I created an issue on github
> [1] and attached a short script to reproduce the problem. As you can see
> all files are written to "/vsimem/wcsout" which is never cleaned.
>
> AFAIK MapServer is meant to be thread save, right? What is the
> recommended way for the usage of VSI IO? Is there a recommended way to
> find out process and thread ids and use them in the directory name? Any
> other ideas?

You could use the msTmpFile() function that calls msTmpFilename(), which returns
a unique filename. You are responsible of cleaning files that have been created
with VSIUnlink(). You can have a look at mapogroutput.c for example.

>
> Thanks,
> Stephan
>
> [1] https://github.com/mapserver/mapserver/issues/4369
>
>
> On Tue, 2012-07-03 at 07:25 +0200, Umberto Nicoletti wrote:
> >
> >
> > On Monday, July 2, 2012, Stephan Meißl wrote:
> >         Hi Umberto,
> >
> >         many thanks for your help. I tried both suggestions but
> >         without an
> >         optimal result.
> >
> >         Using msIO_resetHandlers() instead of msCleanup() gives me,
> >         after using
> >         a fairly big amount of memory, quite strange results. In fact
> >         sometimes
> >         results from previous requests are present in responses. It
> >         seems, that
> >         not everything gets cleared. Any suggestion how to investigate
> >         this?
> >
> >
> > msIO_resetHandlers should probably be invoked after the response has
> > been commited, could you take a look at this page:
> >
> >
> > http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode
> >
> >
> > and implement the cleanup hook as described?
> >
> >
> >         Using msIO_getStdoutBufferString() seems to work for text
> >         responses only
> >         but not for images.
> >
> >         Thanks,
> >         Stephan
> >
> >
> >         On Sun, 2012-07-01 at 12:06 +0200, Umberto Nicoletti wrote:
> >         > Before we turn on debugging mode let's try a couple of
> >         simple changes
> >         > to your script:
> >         >
> >         >
> >         > http://pastebin.ca/2165984
> >         >
> >         >
> >         > I have replaced the call to msCleanup with a call to
> >         > msIO_resetHandlers which clears all buffers.
> >         >
> >         >
> >         > http://pastebin.ca/2165986
> >         >
> >         >
> >         > same as before but with the buffer accessed as a string
> >         rather than a
> >         > byte array (string is not copied inside mapscript, array
> >         is).
> >         >
> >         >
> >         > Let us now how this goes.
> >         >
> >         >
> >         > Umberto
> >         >
> >         > On Thu, Jun 28, 2012 at 8:02 PM, Stephan Meißl
> >         <stephan at meissl.name>
> >         > wrote:
> >         >         Thomas, Umberto,
> >         >
> >         >         we created a simple (at least as simple as possible)
> >         test
> >         >         case. Here is
> >         >         a basic Python script: http://pastebin.ca/2165345
> >         which we
> >         >         deployed in
> >         >         Apache using mod_wsgi with this config:
> >         >         http://pastebin.ca/2165346
> >         >
> >         >         There is also a basic mapfile using the "road" data
> >         in
> >         >         "msautotets/wxs"
> >         >         http://pastebin.ca/2165329 but I suspect you can use
> >         whatever
> >         >         mapfile
> >         >         you have at hand.
> >         >
> >         >         When we issue multiple simultaneous requests e.g.
> >         using this
> >         >         script
> >         >         http://pastebin.ca/2165348 (note the "&" at the end
> >         of the
> >         >         curl call)
> >         >         some of the requests result in errors.
> >         >
> >         >         Either removing the call to msCleanup() or extending
> >         the
> >         >         Apache config
> >         >         to limit the number of threads to 1
> >         >         (http://pastebin.ca/2165347) seems
> >         >         to resolve the issues but of course with
> >         limitations.
> >         >
> >         >         Are we missing anything essential? How should we
> >         free
> >         >         request/map
> >         >         objects without using msCleanup() in Python?
> >         >
> >         >         Thanks for any hint,
> >         >         Stephan
> >         >
> >         >
> >         >         On Thu, 2012-06-28 at 14:15 +0200, Umberto Nicoletti
> >         wrote:
> >         >         > Fabian,
> >         >         > if you could put together a test case that I can
> >         run to
> >         >         reproduce the
> >         >         > problem I will take a look into it.
> >         >         >
> >         >         >
> >         >         > Regards,
> >         >         > Umberto
> >         >         >
> >         >         > On Thu, Jun 28, 2012 at 1:44 PM, thomas bonfort
> >         >         > <thomas.bonfort at gmail.com> wrote:
> >         >         >         Fabian,
> >         >         >         can you get a backtrace of the segfaults?
> >         >         >
> >         >         >         Aside from that, I would personally
> >         refrain from
> >         >         using threads
> >         >         >         with
> >         >         >         mapscript and stick to a prefork mpm in
> >         apache to
> >         >         accomplish
> >         >         >         this. It
> >         >         >         is also my understanding that calling m
>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>