[mapserver-dev] Thread safety [was: python mapscript and msCleanup issues]
Stephan Meißl
stephan at meissl.name
Thu Aug 30 08:49:19 PDT 2012
On Tue, 2012-07-03 at 16:48 +0200, Even Rouault wrote:
> Selon Stephan Meißl <stephan at meissl.name>:
>
> > Hi all,
> >
> > after some further investigations we think the issue is the non-thread
> > safe usage of VSI IO in WCS 1.1 and 2.0. I created an issue on github
> > [1] and attached a short script to reproduce the problem. As you can see
> > all files are written to "/vsimem/wcsout" which is never cleaned.
> >
> > AFAIK MapServer is meant to be thread save, right? What is the
> > recommended way for the usage of VSI IO? Is there a recommended way to
> > find out process and thread ids and use them in the directory name? Any
> > other ideas?
>
> You could use the msTmpFile() function that calls msTmpFilename(), which returns
> a unique filename. You are responsible of cleaning files that have been created
> with VSIUnlink(). You can have a look at mapogroutput.c for example.
Even, just seen that I never replied to your mail. Many thanks for your
suggestion following which we successfully resolved the issue (see [1]).
cu
Stephan
[1] https://github.com/mapserver/mapserver/commit/10f619589244fdb8f1192c72c3d9174235addd58
> >
> > Thanks,
> > Stephan
> >
> > [1] https://github.com/mapserver/mapserver/issues/4369
> >
> >
> > On Tue, 2012-07-03 at 07:25 +0200, Umberto Nicoletti wrote:
> > >
> > >
> > > On Monday, July 2, 2012, Stephan Meißl wrote:
> > > Hi Umberto,
> > >
> > > many thanks for your help. I tried both suggestions but
> > > without an
> > > optimal result.
> > >
> > > Using msIO_resetHandlers() instead of msCleanup() gives me,
> > > after using
> > > a fairly big amount of memory, quite strange results. In fact
> > > sometimes
> > > results from previous requests are present in responses. It
> > > seems, that
> > > not everything gets cleared. Any suggestion how to investigate
> > > this?
> > >
> > >
> > > msIO_resetHandlers should probably be invoked after the response has
> > > been commited, could you take a look at this page:
> > >
> > >
> > > http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode
> > >
> > >
> > > and implement the cleanup hook as described?
> > >
> > >
> > > Using msIO_getStdoutBufferString() seems to work for text
> > > responses only
> > > but not for images.
> > >
> > > Thanks,
> > > Stephan
> > >
> > >
> > > On Sun, 2012-07-01 at 12:06 +0200, Umberto Nicoletti wrote:
> > > > Before we turn on debugging mode let's try a couple of
> > > simple changes
> > > > to your script:
> > > >
> > > >
> > > > http://pastebin.ca/2165984
> > > >
> > > >
> > > > I have replaced the call to msCleanup with a call to
> > > > msIO_resetHandlers which clears all buffers.
> > > >
> > > >
> > > > http://pastebin.ca/2165986
> > > >
> > > >
> > > > same as before but with the buffer accessed as a string
> > > rather than a
> > > > byte array (string is not copied inside mapscript, array
> > > is).
> > > >
> > > >
> > > > Let us now how this goes.
> > > >
> > > >
> > > > Umberto
> > > >
> > > > On Thu, Jun 28, 2012 at 8:02 PM, Stephan Meißl
> > > <stephan at meissl.name>
> > > > wrote:
> > > > Thomas, Umberto,
> > > >
> > > > we created a simple (at least as simple as possible)
> > > test
> > > > case. Here is
> > > > a basic Python script: http://pastebin.ca/2165345
> > > which we
> > > > deployed in
> > > > Apache using mod_wsgi with this config:
> > > > http://pastebin.ca/2165346
> > > >
> > > > There is also a basic mapfile using the "road" data
> > > in
> > > > "msautotets/wxs"
> > > > http://pastebin.ca/2165329 but I suspect you can use
> > > whatever
> > > > mapfile
> > > > you have at hand.
> > > >
> > > > When we issue multiple simultaneous requests e.g.
> > > using this
> > > > script
> > > > http://pastebin.ca/2165348 (note the "&" at the end
> > > of the
> > > > curl call)
> > > > some of the requests result in errors.
> > > >
> > > > Either removing the call to msCleanup() or extending
> > > the
> > > > Apache config
> > > > to limit the number of threads to 1
> > > > (http://pastebin.ca/2165347) seems
> > > > to resolve the issues but of course with
> > > limitations.
> > > >
> > > > Are we missing anything essential? How should we
> > > free
> > > > request/map
> > > > objects without using msCleanup() in Python?
> > > >
> > > > Thanks for any hint,
> > > > Stephan
> > > >
> > > >
> > > > On Thu, 2012-06-28 at 14:15 +0200, Umberto Nicoletti
> > > wrote:
> > > > > Fabian,
> > > > > if you could put together a test case that I can
> > > run to
> > > > reproduce the
> > > > > problem I will take a look into it.
> > > > >
> > > > >
> > > > > Regards,
> > > > > Umberto
> > > > >
> > > > > On Thu, Jun 28, 2012 at 1:44 PM, thomas bonfort
> > > > > <thomas.bonfort at gmail.com> wrote:
> > > > > Fabian,
> > > > > can you get a backtrace of the segfaults?
> > > > >
> > > > > Aside from that, I would personally
> > > refrain from
> > > > using threads
> > > > > with
> > > > > mapscript and stick to a prefork mpm in
> > > apache to
> > > > accomplish
> > > > > this. It
> > > > > is also my understanding that calling m
> >
> >
> > _______________________________________________
> > mapserver-dev mailing list
> > mapserver-dev at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> >
>
>
More information about the mapserver-dev
mailing list