[mapserver-users] mapserver defunct

Wed Jan 12 07:23:42 EST 2011

The problem definately seems related to concurrency.

Load benching using 1 concurrent requests: 0 fails.
Load benching using 2 concurrent requests: +- 30% fails
more concurrent = more fails.

The failed seem to return with HTTP 500.

Running a shp2img during this time, still returns a good image.

I'll post more info about this topic and a resolution, if i find one :). Do
note that we're dealing with shapefiles with 3 000 000 linestrings. While
they consist only of 2 coordinates (begin end obviously), it's still a big
thing to load.

On 12 January 2011 06:18, Stephen Woodbridge <woodbri at swoodbridge.com>wrote:

> Wim,
>
> I'm not sure I have any magic bullet for this problem, because I'm not sure
> what is causing it. If this was my project, I would tackle it like this:
>
> 1. alert people we have a problem that might impact the release date if you
> have not already done that
> 2. divide the problem to try and localize what is causing it
>
> a. do the tiles that fail, always fail? this points at a potential
> mapserver related issue as opposed to a load issue
> a.1 have you turned on mapfile DEBUG output? can you identify the layer
> a.2 you can get some maybe useful output from:
>
> strace /path/to/mapserv QUERY_STRING='<url arguments>'
>
> gdb /path/to/mapserv
> run QUERY_STRING='<url arguments>'
> where
> quit
>
> valgrind /path/to/mapserv QUERY_STRING='<url arguments>'
>
> a.3 you can run it with shp2img ...
>
> b. is it fastcgi, do you see the same problem if you build it as cgi
> c. is it lighttpd, do you see the same problem if you run it with apache
> fastcgi? what about apache cgi?
>
> Intermittent problems are the hardest to diagnose because they often are
> not reproducable in a debuging environment. We might be able to help you
> with more information.
>
> I have done a lot of load testing on various versions of mapserver and
> tiled all of US and Canada about 4-5 different times using mapserver. I have
> not used FastCGI/Lighty and lighttpd, so I'm not sure about what influence
> that might have on the problem.
>
> Hope this gives you some ideas.
>
> Best regards,
>  -Steve W
>   http://imaptools.com/
>
>
> On 1/11/2011 3:15 PM, Wim Vanbelle wrote:
>
>> Well, the problem is that the requests are not always served.
>>
>> During load testing, say about 10 concurrent requests I also load the
>> map. But there are randomly tiles that are never loaded. I thought this
>> would be related to the defunct processes, but that is not a certainty.
>>
>> My setup is lighttpd + fastcgi.
>> To be honest I'm in a pretty bad situation now, considering we go live
>> with this in 9 days.
>>
>> Not sure what else I can do honestly.
>>
>>
>> On 11 January 2011 19:05, Stephen Woodbridge <woodbri at swoodbridge.com
>> <mailto:woodbri at swoodbridge.com>> wrote:
>>
>>    Here is an explanation of defunct processes:
>>
>>    http://www.webmasterworld.com/forum40/1032.htm
>>
>>    I'm sure there are others. So since you are running fastcgi, it is
>>    the responsibility of the fastcgi parent process to clean up dead
>>    child processes. It is likely that it is busy because you are doing
>>    load testing or there is some subtle issue that is occurring. If you
>>    are getting good responses from mapserver, you probably do not need
>>    to worry about it unless you are getting tons of them.
>>
>>    If you are using apache, I believe there is a parameter that you can
>>    set for fastcgi processes which will let them die after N requests.
>>    This should clean those up and apache will spawn a new process if
>>    needed to replace the that died.
>>
>>    -Steve W
>>
>>
>>    On 1/11/2011 10:04 AM, Wim Vanbelle wrote:
>>
>>        Hi all,
>>
>>        I'm getting a lot of
>>
>>        27098 x  20   0     0    0    0 Z    1  0.0   0:00.03 mapserv
>>        <defunct>
>>        27104 x  20   0     0    0    0 Z    1  0.0   0:00.03 mapserv
>>        <defunct>
>>
>>        while load testing mapserver. Is there any way I can go about
>>        checking
>>        out why this is happening? Especially when doing concurrency
>>        tests it
>>        seems to go sideways.
>>
>>        Currently i'm using :
>>
>>        MapServer version 5.6.5 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG
>>        OUTPUT=WBMP
>>        OUTPUT=SWF OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE
>>        SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
>>        SUPPORTS=WMS_CLIENT
>>        SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
>>        SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS
>>        SUPPORTS=RGBA_PNG INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR INPUT=GDAL
>>        INPUT=SHAPEFILE
>>
>>        using FastCGI/Lighty.
>>
>>        Do note that the mapfile itself seems to be fine, since it does
>>        render
>>        content when i'm the single user using it.
>>
>>        Thanks for your expert insight :D.
>>
>>
>>
>>        _______________________________________________
>>        mapserver-users mailing list
>>        mapserver-users at lists.osgeo.org
>>        <mailto:mapserver-users at lists.osgeo.org>
>>
>>        http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>>
>>    _______________________________________________
>>    mapserver-users mailing list
>>    mapserver-users at lists.osgeo.org <mailto:
>> mapserver-users at lists.osgeo.org>
>>
>>    http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-users/attachments/20110112/7e827609/attachment-0001.html