[mapserver-users] mapserver defunct
Stephen Woodbridge
woodbri at swoodbridge.com
Wed Jan 12 07:52:01 PST 2011
Have you built mapserver with threads? If so disable this and see if the
problem goes away.
HTTP 500 probably means that your process is getting a SIGV. Also unless
you are using a database connection, using fastcgi is not going to give
you a huge benefit because of the way mapserver works. And threads will
probably not help either. OK, I admit I am somewhat biased against
threads :) Tuning for performance in my experience is:
1. start by keep it simple
2. measure where your bottlenecks are
3. fix these bottlenecks
4. repeat
Don't throw everything at it because you think think it will help. It
just makes a complicated mess and you have no way to know if any given
pieces is helping or hurting.
Using a tmpfs is good if you manage it, I prefer a ramfs, the difference
being the a ramfs is fixed size and you have to explicitly load it, but
you have more control over size and behavior for a little more planning
and it does not have down side of being able to eat all your memory or
forcing linux to do more swapping which can slow down things.
Also if you need to squeeze an additional 1-1.5% performance and can
live without swap, disable your swap partition. Linux is a swappy OS and
swaps to disk before it needs to in anticipation, this is costly if you
have a lot of memory and do not NEED to swap.
-Steve W
On 1/12/2011 7:23 AM, Wim Vanbelle wrote:
> The problem definately seems related to concurrency.
>
> Load benching using 1 concurrent requests: 0 fails.
> Load benching using 2 concurrent requests: +- 30% fails
> more concurrent = more fails.
>
> The failed seem to return with HTTP 500.
>
> Running a shp2img during this time, still returns a good image.
>
> I'll post more info about this topic and a resolution, if i find one :).
> Do note that we're dealing with shapefiles with 3 000 000 linestrings.
> While they consist only of 2 coordinates (begin end obviously), it's
> still a big thing to load.
>
>
> On 12 January 2011 06:18, Stephen Woodbridge <woodbri at swoodbridge.com
> <mailto:woodbri at swoodbridge.com>> wrote:
>
> Wim,
>
> I'm not sure I have any magic bullet for this problem, because I'm
> not sure what is causing it. If this was my project, I would tackle
> it like this:
>
> 1. alert people we have a problem that might impact the release date
> if you have not already done that
> 2. divide the problem to try and localize what is causing it
>
> a. do the tiles that fail, always fail? this points at a potential
> mapserver related issue as opposed to a load issue
> a.1 have you turned on mapfile DEBUG output? can you identify the layer
> a.2 you can get some maybe useful output from:
>
> strace /path/to/mapserv QUERY_STRING='<url arguments>'
>
> gdb /path/to/mapserv
> run QUERY_STRING='<url arguments>'
> where
> quit
>
> valgrind /path/to/mapserv QUERY_STRING='<url arguments>'
>
> a.3 you can run it with shp2img ...
>
> b. is it fastcgi, do you see the same problem if you build it as cgi
> c. is it lighttpd, do you see the same problem if you run it with
> apache fastcgi? what about apache cgi?
>
> Intermittent problems are the hardest to diagnose because they often
> are not reproducable in a debuging environment. We might be able to
> help you with more information.
>
> I have done a lot of load testing on various versions of mapserver
> and tiled all of US and Canada about 4-5 different times using
> mapserver. I have not used FastCGI/Lighty and lighttpd, so I'm not
> sure about what influence that might have on the problem.
>
> Hope this gives you some ideas.
>
> Best regards,
> -Steve W
> http://imaptools.com/
>
>
> On 1/11/2011 3:15 PM, Wim Vanbelle wrote:
>
> Well, the problem is that the requests are not always served.
>
> During load testing, say about 10 concurrent requests I also
> load the
> map. But there are randomly tiles that are never loaded. I
> thought this
> would be related to the defunct processes, but that is not a
> certainty.
>
> My setup is lighttpd + fastcgi.
> To be honest I'm in a pretty bad situation now, considering we
> go live
> with this in 9 days.
>
> Not sure what else I can do honestly.
>
>
> On 11 January 2011 19:05, Stephen Woodbridge
> <woodbri at swoodbridge.com <mailto:woodbri at swoodbridge.com>
> <mailto:woodbri at swoodbridge.com
> <mailto:woodbri at swoodbridge.com>>> wrote:
>
> Here is an explanation of defunct processes:
>
> http://www.webmasterworld.com/forum40/1032.htm
>
> I'm sure there are others. So since you are running fastcgi,
> it is
> the responsibility of the fastcgi parent process to clean up
> dead
> child processes. It is likely that it is busy because you
> are doing
> load testing or there is some subtle issue that is
> occurring. If you
> are getting good responses from mapserver, you probably do
> not need
> to worry about it unless you are getting tons of them.
>
> If you are using apache, I believe there is a parameter that
> you can
> set for fastcgi processes which will let them die after N
> requests.
> This should clean those up and apache will spawn a new
> process if
> needed to replace the that died.
>
> -Steve W
>
>
> On 1/11/2011 10:04 AM, Wim Vanbelle wrote:
>
> Hi all,
>
> I'm getting a lot of
>
> 27098 x 20 0 0 0 0 Z 1 0.0 0:00.03
> mapserv
> <defunct>
> 27104 x 20 0 0 0 0 Z 1 0.0 0:00.03
> mapserv
> <defunct>
>
> while load testing mapserver. Is there any way I can go
> about
> checking
> out why this is happening? Especially when doing concurrency
> tests it
> seems to go sideways.
>
> Currently i'm using :
>
> MapServer version 5.6.5 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG
> OUTPUT=WBMP
> OUTPUT=SWF OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG
> SUPPORTS=FREETYPE
> SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
> SUPPORTS=WMS_CLIENT
> SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER
> SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS
> SUPPORTS=GEOS
> SUPPORTS=RGBA_PNG INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR
> INPUT=GDAL
> INPUT=SHAPEFILE
>
> using FastCGI/Lighty.
>
> Do note that the mapfile itself seems to be fine, since
> it does
> render
> content when i'm the single user using it.
>
> Thanks for your expert insight :D.
>
>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> <mailto:mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>>
>
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> <mailto:mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>>
>
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>
>
>
>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
More information about the MapServer-users
mailing list