[mapserver-dev] Mapserver lexer/parser issues

[Tamas Szekeres]

Steve,

No progress, I don't have the environment to reproduce this.
We could probably try with other lexer options if possible, but it is faily
out of my knowlegde.

Best regards,

Tamas


Steve Lime <sdlime at gmail.com> ezt írta (időpont: 2021. szept. 1., Sze,
16:09):

> Tamas: Curious if you were able to diagnose/fix this? --Steve
>
> On Fri, Aug 13, 2021 at 4:15 PM Lime, Steve D (MNIT) <
> steve.lime at state.mn.us> wrote:
>
>> Ouch, this looks tough to debug. Are the processes all getting stuck in
>> the same spot or could that be a red herring? Is there any way to re-create
>> the behavior outside of ASP.Net/MapScript using just straight MapServer
>> calls and maybe FastCGI? What sort of lexer/parser interactions are there
>> on each request – there would be the map load but a point query shouldn’t
>> use the lexer/parser unless there are layer filters or expressions involved.
>>
>>
>>
>> *From:* mapserver-dev <mapserver-dev-bounces at lists.osgeo.org> *On Behalf
>> Of *Tamas Szekeres
>> *Sent:* Friday, August 13, 2021 5:05 AM
>> *To:* mapserver-dev at lists.osgeo.org
>> *Subject:* [mapserver-dev] Mapserver lexer/parser issues
>>
>>
>>
>> Dear Developers,
>>
>> We're currently having an issue with MapServer and I am hoping you can
>> give some idea how to work around this.
>>
>> We use asp.net
>> <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fasp.net%2F&data=04%7C01%7Csteve.lime%40state.mn.us%7C146e3f692ac0429140db08d95e41e13e%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637644461111528028%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=kbfLO5GflI5L6q6CsAmRNhIkoz1Aw10n5aGhRrrPl7s%3D&reserved=0>
>> and IIS, where the web requests are being served from a pool of threads
>> within a single process. The application runs fine if we just use it
>> manually, but when we run a load test which involves 20 concurrent users or
>> so, the worker process CPU gets stuck at 25% after a while (there are 4
>> virtual cpu on this box, so it looks like it is maxing one of the vCPU).
>> Every subsequent requests on that particular worker process times out.
>>
>> The test consists of loading a map, doing a selection (ie queryBypoint)
>> and generate a map, then move to another area in a grid like pattern and do
>> another selection and map export.
>> Once finished the test starts again automatically, load the map,
>> selection, etc... for each concurrent user. The test complete successfully
>> a few iterations but after a while the CPU gets stuck.
>> Note that the same request executed successfully many times before.
>>
>> After the process was getting stucked this way, we took a memory snapshot
>> with some debugging information which shows the following results:
>>
>> 1.  One of the threads is operating in an endless loop in maplexer.c
>> inside the following fragment:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *do { YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ; if (
>> yy_accept[yy_current_state] ) { (yy_last_accepting_state) =
>> yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while (
>> yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) {
>> yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >=
>> 2034 ) yy_c = yy_meta[yy_c]; } yy_current_state =
>> yy_nxt[yy_base[yy_current_state] + yy_c]; ++yy_cp; } while (
>> yy_current_state != 2033 );*
>>
>> It looks like yy_current_state never gets the expected value to get out
>> of the loop, but it is not too trivial for me how to interpret this
>> situation in the lexer generated code. It may probably be a bug in the
>> lexer itself or we didn't manage to protect the static variables against
>> the multiple thread access well enough (?)
>>
>> 2. I can also see in the memory dmp that more than 75% of the threads are
>> waiting for the parser lock at msLoadMapFromString which significantly
>> degrades the performance by consuming the available threads from the pool.
>> Do we have any ongoing effort to make the mapfile/expression lexer/parser
>> reentrant?
>>
>> The application is using a 7.4.x version of MapServer and it doesn't seem
>> to be trivial for us to upgrade in short term, but if you can confirm this
>> specific problem has already been addressed in a later version we'll
>> consider to upgrade.
>>
>> Thanks,
>>
>> Tamas
>>
>>
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20210902/7d731979/attachment.html>