The problem definately seems related to concurrency. <br><br>Load benching using 1 concurrent requests: 0 fails.<br>Load benching using 2 concurrent requests: +- 30% fails<br>more concurrent = more fails.<br><br>The failed seem to return with HTTP 500.<br>
<br>Running a shp2img during this time, still returns a good image.<br><br>I'll post more info about this topic and a resolution, if i find one :). Do note that we're dealing with shapefiles with 3 000 000 linestrings. While they consist only of 2 coordinates (begin end obviously), it's still a big thing to load.<br>
<br><br><div class="gmail_quote">On 12 January 2011 06:18, Stephen Woodbridge <span dir="ltr"><<a href="mailto:woodbri@swoodbridge.com">woodbri@swoodbridge.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Wim,<br>
<br>
I'm not sure I have any magic bullet for this problem, because I'm not sure what is causing it. If this was my project, I would tackle it like this:<br>
<br>
1. alert people we have a problem that might impact the release date if you have not already done that<br>
2. divide the problem to try and localize what is causing it<br>
<br>
a. do the tiles that fail, always fail? this points at a potential mapserver related issue as opposed to a load issue<br>
a.1 have you turned on mapfile DEBUG output? can you identify the layer<br>
a.2 you can get some maybe useful output from:<br>
<br>
strace /path/to/mapserv QUERY_STRING='<url arguments>'<br>
<br>
gdb /path/to/mapserv<br>
run QUERY_STRING='<url arguments>'<br>
where<br>
quit<br>
<br>
valgrind /path/to/mapserv QUERY_STRING='<url arguments>'<br>
<br>
a.3 you can run it with shp2img ...<br>
<br>
b. is it fastcgi, do you see the same problem if you build it as cgi<br>
c. is it lighttpd, do you see the same problem if you run it with apache fastcgi? what about apache cgi?<br>
<br>
Intermittent problems are the hardest to diagnose because they often are not reproducable in a debuging environment. We might be able to help you with more information.<br>
<br>
I have done a lot of load testing on various versions of mapserver and tiled all of US and Canada about 4-5 different times using mapserver. I have not used FastCGI/Lighty and lighttpd, so I'm not sure about what influence that might have on the problem.<br>
<br>
Hope this gives you some ideas.<br>
<br>
Best regards,<br>
-Steve W<br>
<a href="http://imaptools.com/" target="_blank">http://imaptools.com/</a><div class="im"><br>
<br>
On 1/11/2011 3:15 PM, Wim Vanbelle wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
Well, the problem is that the requests are not always served.<br>
<br>
During load testing, say about 10 concurrent requests I also load the<br>
map. But there are randomly tiles that are never loaded. I thought this<br>
would be related to the defunct processes, but that is not a certainty.<br>
<br>
My setup is lighttpd + fastcgi.<br>
To be honest I'm in a pretty bad situation now, considering we go live<br>
with this in 9 days.<br>
<br>
Not sure what else I can do honestly.<br>
<br>
<br>
On 11 January 2011 19:05, Stephen Woodbridge <<a href="mailto:woodbri@swoodbridge.com" target="_blank">woodbri@swoodbridge.com</a><br></div><div><div></div><div class="h5">
<mailto:<a href="mailto:woodbri@swoodbridge.com" target="_blank">woodbri@swoodbridge.com</a>>> wrote:<br>
<br>
Here is an explanation of defunct processes:<br>
<br>
<a href="http://www.webmasterworld.com/forum40/1032.htm" target="_blank">http://www.webmasterworld.com/forum40/1032.htm</a><br>
<br>
I'm sure there are others. So since you are running fastcgi, it is<br>
the responsibility of the fastcgi parent process to clean up dead<br>
child processes. It is likely that it is busy because you are doing<br>
load testing or there is some subtle issue that is occurring. If you<br>
are getting good responses from mapserver, you probably do not need<br>
to worry about it unless you are getting tons of them.<br>
<br>
If you are using apache, I believe there is a parameter that you can<br>
set for fastcgi processes which will let them die after N requests.<br>
This should clean those up and apache will spawn a new process if<br>
needed to replace the that died.<br>
<br>
-Steve W<br>
<br>
<br>
On 1/11/2011 10:04 AM, Wim Vanbelle wrote:<br>
<br>
Hi all,<br>
<br>
I'm getting a lot of<br>
<br>
27098 x 20 0 0 0 0 Z 1 0.0 0:00.03 mapserv<br>
<defunct><br>
27104 x 20 0 0 0 0 Z 1 0.0 0:00.03 mapserv<br>
<defunct><br>
<br>
while load testing mapserver. Is there any way I can go about<br>
checking<br>
out why this is happening? Especially when doing concurrency<br>
tests it<br>
seems to go sideways.<br>
<br>
Currently i'm using :<br>
<br>
MapServer version 5.6.5 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG<br>
OUTPUT=WBMP<br>
OUTPUT=SWF OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE<br>
SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER<br>
SUPPORTS=WMS_CLIENT<br>
SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER<br>
SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS<br>
SUPPORTS=RGBA_PNG INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR INPUT=GDAL<br>
INPUT=SHAPEFILE<br>
<br>
using FastCGI/Lighty.<br>
<br>
Do note that the mapfile itself seems to be fine, since it does<br>
render<br>
content when i'm the single user using it.<br>
<br>
Thanks for your expert insight :D.<br>
<br>
<br>
<br>
_______________________________________________<br>
mapserver-users mailing list<br>
<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br></div></div>
<mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a>><div class="im"><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a><br>
<br>
<br>
_______________________________________________<br>
mapserver-users mailing list<br></div>
<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a> <mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a>><div class="im">
<br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a><br>
<br>
<br>
</div></blockquote>
<br>
</blockquote></div><br>