[mapserver-dev] python mapscript and msCleanup issues

[Fabian Schindler]

Hi all,

Has anyone had issues with Python MapScript and the msCleanup() in 
WSGI-Apache applications? Using msCleanup() results in strange 
errors/segfaults in multi-threaded setups but not using it quickly eats 
up memory.

We are developing a Django application to serve enhanced WCS responses 
(according to the EO-WCS extension). For this purpose, we use MapScript 
to receive a basic WCS response, which we are extending with additional 
metadata.

Our mapserver dispatch code is like this:

mapscript.msIO_installStdoutToBuffer()
dispatch_status = map.OWSDispatch(ows_req)
content_type = mapscript.msIO_stripStdoutBufferContentType()
mapscript.msIO_stripStdoutBufferContentHeaders()
result = mapscript.msIO_getStdoutBufferBytes()
mapscript.msCleanup(1)
return result, content_type, dispatch_status

Unfortunately we experienced several weird errors (segfaults, hit 
assertions within GEOS, etc) when dispatching requests, especially in 
multi-threaded environments (such as a deployment with Apache). We 
traced this down and found that a call to `msCleanup` right after the 
OWSDispatch was causing the problems.

Taken from the mapscript documentation:

"msCleanup() attempts to recover all dynamically allocated resources 
allocated by MapServer code and dependent libraries. It it used 
primarily for final cleanup in scripts that need to do memory leak 
testing to get rid of “noise” one-time allocations. It should not 
normally be used by production code."

When we remove the call to msCleanup, no segfaults and other errors 
occur within Apache. Unfortunately, while running our test cases 
(including several hundreds MapServer dispatches) we quickly run out of 
memory (e.g: memory consumption after 30 test cases was over 2.2 GiB 
instead of about 150 MB).

Any ideas are welcome :)

Thanks,
Fabian