[mapserver-dev] Mapserver lexer/parser issues

[Tamas Szekeres]

Dear Developers,

We're currently having an issue with MapServer and I am hoping you can give
some idea how to work around this.

We use asp.net and IIS, where the web requests are being served from a pool
of threads within a single process. The application runs fine if we just
use it manually, but when we run a load test which involves 20 concurrent
users or so, the worker process CPU gets stuck at 25% after a while (there
are 4 virtual cpu on this box, so it looks like it is maxing one of the
vCPU). Every subsequent requests on that particular worker process times
out.

The test consists of loading a map, doing a selection (ie queryBypoint) and
generate a map, then move to another area in a grid like pattern and do
another selection and map export.
Once finished the test starts again automatically, load the map, selection,
etc... for each concurrent user. The test complete successfully a few
iterations but after a while the CPU gets stuck.
Note that the same request executed successfully many times before.

After the process was getting stucked this way, we took a memory snapshot
with some debugging information which shows the following results:

1.  One of the threads is operating in an endless loop in maplexer.c inside
the following fragment:


















*do{YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ;if (
yy_accept[yy_current_state] ){(yy_last_accepting_state) =
yy_current_state;(yy_last_accepting_cpos) = yy_cp;}while (
yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state
){yy_current_state = (int) yy_def[yy_current_state];if ( yy_current_state
>= 2034 )yy_c = yy_meta[yy_c];}yy_current_state =
yy_nxt[yy_base[yy_current_state] + yy_c];++yy_cp;}while ( yy_current_state
!= 2033 );*

It looks like yy_current_state never gets the expected value to get out of
the loop, but it is not too trivial for me how to interpret this situation
in the lexer generated code. It may probably be a bug in the lexer itself
or we didn't manage to protect the static variables against the multiple
thread access well enough (?)

2. I can also see in the memory dmp that more than 75% of the threads are
waiting for the parser lock at msLoadMapFromString which significantly
degrades the performance by consuming the available threads from the pool.
Do we have any ongoing effort to make the mapfile/expression lexer/parser
reentrant?

The application is using a 7.4.x version of MapServer and it doesn't seem
to be trivial for us to upgrade in short term, but if you can confirm this
specific problem has already been addressed in a later version we'll
consider to upgrade.

Thanks,

Tamas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20210813/0a767ab7/attachment.html>