<div dir="ltr">Tamas: Curious if you were able to diagnose/fix this? --Steve</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 4:15 PM Lime, Steve D (MNIT) <<a href="mailto:steve.lime@state.mn.us">steve.lime@state.mn.us</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_8991572914255972688WordSection1">
<p class="MsoNormal">Ouch, this looks tough to debug. Are the processes all getting stuck in the same spot or could that be a red herring? Is there any way to re-create the behavior outside of ASP.Net/MapScript using just straight MapServer calls and maybe
FastCGI? What sort of lexer/parser interactions are there on each request – there would be the map load but a point query shouldn’t use the lexer/parser unless there are layer filters or expressions involved.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> mapserver-dev <<a href="mailto:mapserver-dev-bounces@lists.osgeo.org" target="_blank">mapserver-dev-bounces@lists.osgeo.org</a>>
<b>On Behalf Of </b>Tamas Szekeres<br>
<b>Sent:</b> Friday, August 13, 2021 5:05 AM<br>
<b>To:</b> <a href="mailto:mapserver-dev@lists.osgeo.org" target="_blank">mapserver-dev@lists.osgeo.org</a><br>
<b>Subject:</b> [mapserver-dev] Mapserver lexer/parser issues<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt">Dear Developers,<br>
<br>
We're currently having an issue with MapServer and I am hoping you can give some idea how to work around this.
<br>
<br>
We use <a href="https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fasp.net%2F&data=04%7C01%7Csteve.lime%40state.mn.us%7C146e3f692ac0429140db08d95e41e13e%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637644461111528028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=kbfLO5GflI5L6q6CsAmRNhIkoz1Aw10n5aGhRrrPl7s%3D&reserved=0" target="_blank">
asp.net</a> and IIS, where the web requests are being served from a pool of threads within a single process. The application runs fine if we just use it manually, but when we run a load test which involves 20 concurrent users or so, the worker process CPU gets
stuck at 25% after a while (there are 4 virtual cpu on this box, so it looks like it is maxing one of the vCPU). Every subsequent requests on that particular worker process times out. <br>
<br>
The test consists of loading a map, doing a selection (ie queryBypoint) and generate a map, then move to another area in a grid like pattern and do another selection and map export.<br>
Once finished the test starts again automatically, load the map, selection, etc... for each concurrent user. The test complete successfully a few iterations but after a while the CPU gets stuck.<br>
Note that the same request executed successfully many times before.<br>
<br>
After the process was getting stucked this way, we took a memory snapshot with some debugging information which shows the following results:<br>
<br>
1. One of the threads is operating in an endless loop in maplexer.c inside the following fragment:<br>
<br>
<i>do<br>
{<br>
YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ;<br>
if ( yy_accept[yy_current_state] )<br>
{<br>
(yy_last_accepting_state) = yy_current_state;<br>
(yy_last_accepting_cpos) = yy_cp;<br>
}<br>
while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )<br>
{<br>
yy_current_state = (int) yy_def[yy_current_state];<br>
if ( yy_current_state >= 2034 )<br>
yy_c = yy_meta[yy_c];<br>
}<br>
yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];<br>
++yy_cp;<br>
}<br>
while ( yy_current_state != 2033 );</i><br>
<br>
It looks like yy_current_state never gets the expected value to get out of the loop, but it is not too trivial for me how to interpret this situation in the lexer generated code. It may probably be a bug in the lexer itself or we didn't manage to protect the
static variables against the multiple thread access well enough (?)<br>
<br>
2. I can also see in the memory dmp that more than 75% of the threads are waiting for the parser lock at msLoadMapFromString which significantly degrades the performance by consuming the available threads from the pool. Do we have any ongoing effort to make
the mapfile/expression lexer/parser reentrant?<br>
<br>
The application is using a 7.4.x version of MapServer and it doesn't seem to be trivial for us to upgrade in short term, but if you can confirm this specific problem has already been addressed in a later version we'll consider to upgrade.<br>
<br>
Thanks,<br>
<br>
Tamas<br>
<br>
<br>
<u></u><u></u></p>
</div>
</div>
</div>
</div>
_______________________________________________<br>
mapserver-dev mailing list<br>
<a href="mailto:mapserver-dev@lists.osgeo.org" target="_blank">mapserver-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/mapserver-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/mapserver-dev</a><br>
</blockquote></div>