[mapserver-dev] python mapscript and msCleanup issues

[thomas bonfort]

Fabian,
can you get a backtrace of the segfaults?

Aside from that, I would personally refrain from using threads with
mapscript and stick to a prefork mpm in apache to accomplish this. It
is also my understanding that calling msCleanup() should be limited to
when a process is actually exiting, and not at the end of each
request, i.e. if I were you I would investigate where memory is
leaking when freeing your request/map objects.

regards,
thomas

On Thu, Jun 28, 2012 at 11:28 AM, Fabian Schindler
<fabian.schindler at eox.at> wrote:
> Hi all,
>
> Has anyone had issues with Python MapScript and the msCleanup() in
> WSGI-Apache applications? Using msCleanup() results in strange
> errors/segfaults in multi-threaded setups but not using it quickly eats up
> memory.
>
> We are developing a Django application to serve enhanced WCS responses
> (according to the EO-WCS extension). For this purpose, we use MapScript to
> receive a basic WCS response, which we are extending with additional
> metadata.
>
> Our mapserver dispatch code is like this:
>
> mapscript.msIO_installStdoutToBuffer()
> dispatch_status = map.OWSDispatch(ows_req)
> content_type = mapscript.msIO_stripStdoutBufferContentType()
> mapscript.msIO_stripStdoutBufferContentHeaders()
> result = mapscript.msIO_getStdoutBufferBytes()
> mapscript.msCleanup(1)
> return result, content_type, dispatch_status
>
> Unfortunately we experienced several weird errors (segfaults, hit assertions
> within GEOS, etc) when dispatching requests, especially in multi-threaded
> environments (such as a deployment with Apache). We traced this down and
> found that a call to `msCleanup` right after the OWSDispatch was causing the
> problems.
>
> Taken from the mapscript documentation:
>
> "msCleanup() attempts to recover all dynamically allocated resources
> allocated by MapServer code and dependent libraries. It it used primarily
> for final cleanup in scripts that need to do memory leak testing to get rid
> of “noise” one-time allocations. It should not normally be used by
> production code."
>
> When we remove the call to msCleanup, no segfaults and other errors occur
> within Apache. Unfortunately, while running our test cases (including
> several hundreds MapServer dispatches) we quickly run out of memory (e.g:
> memory consumption after 30 test cases was over 2.2 GiB instead of about 150
> MB).
>
> Any ideas are welcome :)
>
> Thanks,
> Fabian
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev