[mapserver-dev] Mapserver lexer/parser issues

[Steve Lime]

Tamas: Curious if you were able to diagnose/fix this? --Steve

On Fri, Aug 13, 2021 at 4:15 PM Lime, Steve D (MNIT) <steve.lime at state.mn.us>
wrote:

> Ouch, this looks tough to debug. Are the processes all getting stuck in
> the same spot or could that be a red herring? Is there any way to re-create
> the behavior outside of ASP.Net/MapScript using just straight MapServer
> calls and maybe FastCGI? What sort of lexer/parser interactions are there
> on each request – there would be the map load but a point query shouldn’t
> use the lexer/parser unless there are layer filters or expressions involved.
>
>
>
> *From:* mapserver-dev <mapserver-dev-bounces at lists.osgeo.org> *On Behalf
> Of *Tamas Szekeres
> *Sent:* Friday, August 13, 2021 5:05 AM
> *To:* mapserver-dev at lists.osgeo.org
> *Subject:* [mapserver-dev] Mapserver lexer/parser issues
>
>
>
> Dear Developers,
>
> We're currently having an issue with MapServer and I am hoping you can
> give some idea how to work around this.
>
> We use asp.net
> <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fasp.net%2F&data=04%7C01%7Csteve.lime%40state.mn.us%7C146e3f692ac0429140db08d95e41e13e%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637644461111528028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=kbfLO5GflI5L6q6CsAmRNhIkoz1Aw10n5aGhRrrPl7s%3D&reserved=0>
> and IIS, where the web requests are being served from a pool of threads
> within a single process. The application runs fine if we just use it
> manually, but when we run a load test which involves 20 concurrent users or
> so, the worker process CPU gets stuck at 25% after a while (there are 4
> virtual cpu on this box, so it looks like it is maxing one of the vCPU).
> Every subsequent requests on that particular worker process times out.
>
> The test consists of loading a map, doing a selection (ie queryBypoint)
> and generate a map, then move to another area in a grid like pattern and do
> another selection and map export.
> Once finished the test starts again automatically, load the map,
> selection, etc... for each concurrent user. The test complete successfully
> a few iterations but after a while the CPU gets stuck.
> Note that the same request executed successfully many times before.
>
> After the process was getting stucked this way, we took a memory snapshot
> with some debugging information which shows the following results:
>
> 1.  One of the threads is operating in an endless loop in maplexer.c
> inside the following fragment:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *do { YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ; if (
> yy_accept[yy_current_state] ) { (yy_last_accepting_state) =
> yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while (
> yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) {
> yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >=
> 2034 ) yy_c = yy_meta[yy_c]; } yy_current_state =
> yy_nxt[yy_base[yy_current_state] + yy_c]; ++yy_cp; } while (
> yy_current_state != 2033 );*
>
> It looks like yy_current_state never gets the expected value to get out of
> the loop, but it is not too trivial for me how to interpret this situation
> in the lexer generated code. It may probably be a bug in the lexer itself
> or we didn't manage to protect the static variables against the multiple
> thread access well enough (?)
>
> 2. I can also see in the memory dmp that more than 75% of the threads are
> waiting for the parser lock at msLoadMapFromString which significantly
> degrades the performance by consuming the available threads from the pool.
> Do we have any ongoing effort to make the mapfile/expression lexer/parser
> reentrant?
>
> The application is using a 7.4.x version of MapServer and it doesn't seem
> to be trivial for us to upgrade in short term, but if you can confirm this
> specific problem has already been addressed in a later version we'll
> consider to upgrade.
>
> Thanks,
>
> Tamas
>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20210901/923e60f1/attachment.html>