[mapserver-users] mapserver defunct

Wed Jan 12 08:19:35 EST 2011

Hi,

Maybe my experience is somehow related. I have just been seeding GeoWebCache from our Mapserver this week and using more than two threads with Apache fast-cgi proved to be very unreliable.  Some of the GWC threads were dying soon with message:
ERROR [seed.MTSeeder] - Unexpected response code from backend:
 500 for http://....

I turned back to normal Mapserver cgi and seeding is progressing much faster for me now because I can keep more GeoWebCache seeder threads up this way.

-Jukka Rahkonen-

thomas bonfort wrote:

> 
> http://trac.osgeo.org/mapserver/ticket/3099 might be related. I don't
> know how fastcgi in lighty works, but mapserver in fastcgi definitely
> does not play nicely with fastcgi implementations that don't expect
> child processes to ever exit().
> As Stpehen pointed out, I'd try with apache/fastcgi , if not only to
> confirm that the problem is in the fastcgi rather than pure mapserver
> side.
> 
> best regards,
> thomas
> 
> On Wed, Jan 12, 2011 at 13:23, Wim Vanbelle 
> <wimvanbelle at gmail.com> wrote:
> > The problem definately seems related to concurrency.
> >
> > Load benching using 1 concurrent requests: 0 fails.
> > Load benching using 2 concurrent requests: +- 30% fails
> > more concurrent = more fails.
> >
> > The failed seem to return with HTTP 500.
> >
> > Running a shp2img during this time, still returns a good image.
> >
> > I'll post more info about this topic and a resolution, if i 
> find one :). Do
> > note that we're dealing with shapefiles with 3 000 000 
> linestrings. While
> > they consist only of 2 coordinates (begin end obviously), 
> it's still a big
> > thing to load.
> >
> >
> > On 12 January 2011 06:18, Stephen Woodbridge 
> <woodbri at swoodbridge.com>
> > wrote:
> >>
> >> Wim,
> >>
> >> I'm not sure I have any magic bullet for this problem, 
> because I'm not
> >> sure what is causing it. If this was my project, I would 
> tackle it like
> >> this:
> >>
> >> 1. alert people we have a problem that might impact the 
> release date if
> >> you have not already done that
> >> 2. divide the problem to try and localize what is causing it
> >>
> >> a. do the tiles that fail, always fail? this points at a potential
> >> mapserver related issue as opposed to a load issue
> >> a.1 have you turned on mapfile DEBUG output? can you 
> identify the layer
> >> a.2 you can get some maybe useful output from:
> >>
> >> strace /path/to/mapserv QUERY_STRING='<url arguments>'
> >>
> >> gdb /path/to/mapserv
> >> run QUERY_STRING='<url arguments>'
> >> where
> >> quit
> >>
> >> valgrind /path/to/mapserv QUERY_STRING='<url arguments>'
> >>
> >> a.3 you can run it with shp2img ...
> >>
> >> b. is it fastcgi, do you see the same problem if you build 
> it as cgi
> >> c. is it lighttpd, do you see the same problem if you run 
> it with apache
> >> fastcgi? what about apache cgi?
> >>
> >> Intermittent problems are the hardest to diagnose because 
> they often are
> >> not reproducable in a debuging environment. We might be 
> able to help you
> >> with more information.
> >>
> >> I have done a lot of load testing on various versions of 
> mapserver and
> >> tiled all of US and Canada about 4-5 different times using 
> mapserver. I have
> >> not used FastCGI/Lighty and lighttpd, so I'm not sure 
> about what influence
> >> that might have on the problem.
> >>
> >> Hope this gives you some ideas.
> >>
> >> Best regards,
> >>  -Steve W
> >>   http://imaptools.com/
> >>
> >> On 1/11/2011 3:15 PM, Wim Vanbelle wrote:
> >>>
> >>> Well, the problem is that the requests are not always served.
> >>>
> >>> During load testing, say about 10 concurrent requests I 
> also load the
> >>> map. But there are randomly tiles that are never loaded. 
> I thought this
> >>> would be related to the defunct processes, but that is 
> not a certainty.
> >>>
> >>> My setup is lighttpd + fastcgi.
> >>> To be honest I'm in a pretty bad situation now, 
> considering we go live
> >>> with this in 9 days.
> >>>
> >>> Not sure what else I can do honestly.
> >>>
> >>>
> >>> On 11 January 2011 19:05, Stephen Woodbridge 
> <woodbri at swoodbridge.com
> >>> <mailto:woodbri at swoodbridge.com>> wrote:
> >>>
> >>>    Here is an explanation of defunct processes:
> >>>
> >>>    http://www.webmasterworld.com/forum40/1032.htm
> >>>
> >>>    I'm sure there are others. So since you are running 
> fastcgi, it is
> >>>    the responsibility of the fastcgi parent process to 
> clean up dead
> >>>    child processes. It is likely that it is busy because 
> you are doing
> >>>    load testing or there is some subtle issue that is 
> occurring. If you
> >>>    are getting good responses from mapserver, you 
> probably do not need
> >>>    to worry about it unless you are getting tons of them.
> >>>
> >>>    If you are using apache, I believe there is a 
> parameter that you can
> >>>    set for fastcgi processes which will let them die 
> after N requests.
> >>>    This should clean those up and apache will spawn a new 
> process if
> >>>    needed to replace the that died.
> >>>
> >>>    -Steve W
> >>>
> >>>
> >>>    On 1/11/2011 10:04 AM, Wim Vanbelle wrote:
> >>>
> >>>        Hi all,
> >>>
> >>>        I'm getting a lot of
> >>>
> >>>        27098 x  20   0     0    0    0 Z    1  0.0   
> 0:00.03 mapserv
> >>>        <defunct>
> >>>        27104 x  20   0     0    0    0 Z    1  0.0   
> 0:00.03 mapserv
> >>>        <defunct>
> >>>
> >>>        while load testing mapserver. Is there any way I 
> can go about
> >>>        checking
> >>>        out why this is happening? Especially when doing 
> concurrency
> >>>        tests it
> >>>        seems to go sideways.
> >>>
> >>>        Currently i'm using :
> >>>
> >>>        MapServer version 5.6.5 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG
> >>>        OUTPUT=WBMP
> >>>        OUTPUT=SWF OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG 
> SUPPORTS=FREETYPE
> >>>        SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
> >>>        SUPPORTS=WMS_CLIENT
> >>>        SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
> >>>        SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS
> >>> SUPPORTS=GEOS
> >>>        SUPPORTS=RGBA_PNG INPUT=EPPL7 INPUT=POSTGIS 
> INPUT=OGR INPUT=GDAL
> >>>        INPUT=SHAPEFILE
> >>>
> >>>        using FastCGI/Lighty.
> >>>
> >>>        Do note that the mapfile itself seems to be fine, 
> since it does
> >>>        render
> >>>        content when i'm the single user using it.
> >>>
> >>>        Thanks for your expert insight :D.
> >>>
> >>>
> >>>
> >>>        _______________________________________________
> >>>        mapserver-users mailing list
> >>>        mapserver-users at lists.osgeo.org
> >>>        <mailto:mapserver-users at lists.osgeo.org>
> >>>        http://lists.osgeo.org/mailman/listinfo/mapserver-users
> >>>
> >>>
> >>>    _______________________________________________
> >>>    mapserver-users mailing list
> >>>    mapserver-users at lists.osgeo.org
> >>> <mailto:mapserver-users at lists.osgeo.org>
> >>>    http://lists.osgeo.org/mailman/listinfo/mapserver-users
> >>>
> >>>
> >>
> >
> >
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> >
> >
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 