<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Ouch, this looks tough to debug. Are the processes all getting stuck in the same spot or could that be a red herring? Is there any way to re-create the behavior outside of ASP.Net/MapScript using just straight MapServer calls and maybe
FastCGI? What sort of lexer/parser interactions are there on each request – there would be the map load but a point query shouldn’t use the lexer/parser unless there are layer filters or expressions involved.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> mapserver-dev <mapserver-dev-bounces@lists.osgeo.org>
<b>On Behalf Of </b>Tamas Szekeres<br>
<b>Sent:</b> Friday, August 13, 2021 5:05 AM<br>
<b>To:</b> mapserver-dev@lists.osgeo.org<br>
<b>Subject:</b> [mapserver-dev] Mapserver lexer/parser issues<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Dear Developers,<br>
<br>
We're currently having an issue with MapServer and I am hoping you can give some idea how to work around this.
<br>
<br>
We use <a href="https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fasp.net%2F&data=04%7C01%7Csteve.lime%40state.mn.us%7C146e3f692ac0429140db08d95e41e13e%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637644461111528028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=kbfLO5GflI5L6q6CsAmRNhIkoz1Aw10n5aGhRrrPl7s%3D&reserved=0">
asp.net</a> and IIS, where the web requests are being served from a pool of threads within a single process. The application runs fine if we just use it manually, but when we run a load test which involves 20 concurrent users or so, the worker process CPU gets
stuck at 25% after a while (there are 4 virtual cpu on this box, so it looks like it is maxing one of the vCPU). Every subsequent requests on that particular worker process times out. <br>
<br>
The test consists of loading a map, doing a selection (ie queryBypoint) and generate a map, then move to another area in a grid like pattern and do another selection and map export.<br>
Once finished the test starts again automatically, load the map, selection, etc... for each concurrent user. The test complete successfully a few iterations but after a while the CPU gets stuck.<br>
Note that the same request executed successfully many times before.<br>
<br>
After the process was getting stucked this way, we took a memory snapshot with some debugging information which shows the following results:<br>
<br>
1. One of the threads is operating in an endless loop in maplexer.c inside the following fragment:<br>
<br>
<i>do<br>
{<br>
YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ;<br>
if ( yy_accept[yy_current_state] )<br>
{<br>
(yy_last_accepting_state) = yy_current_state;<br>
(yy_last_accepting_cpos) = yy_cp;<br>
}<br>
while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )<br>
{<br>
yy_current_state = (int) yy_def[yy_current_state];<br>
if ( yy_current_state >= 2034 )<br>
yy_c = yy_meta[yy_c];<br>
}<br>
yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];<br>
++yy_cp;<br>
}<br>
while ( yy_current_state != 2033 );</i><br>
<br>
It looks like yy_current_state never gets the expected value to get out of the loop, but it is not too trivial for me how to interpret this situation in the lexer generated code. It may probably be a bug in the lexer itself or we didn't manage to protect the
static variables against the multiple thread access well enough (?)<br>
<br>
2. I can also see in the memory dmp that more than 75% of the threads are waiting for the parser lock at msLoadMapFromString which significantly degrades the performance by consuming the available threads from the pool. Do we have any ongoing effort to make
the mapfile/expression lexer/parser reentrant?<br>
<br>
The application is using a 7.4.x version of MapServer and it doesn't seem to be trivial for us to upgrade in short term, but if you can confirm this specific problem has already been addressed in a later version we'll consider to upgrade.<br>
<br>
Thanks,<br>
<br>
Tamas<br>
<br>
<br>
<o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>