<div dir="ltr"><div>If there is no layer named "Height of Buildings (2013)" then lexer would see that as an attribute binding and with no quotes around it (the outer "'s are stripped off beforehand) it would be interpreted as a numeric binding. I guess the initial questions I have are:</div><div><ol><li>is there a layer with that name in the mapfile.</li><li>if yes, does that layer also have the same LABELREQUIRES attribute? I believe the current layer name is skipped with processing its own LABELREQUIRES expression.</li></ol></div>Can you share the mapfile privately? </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 22, 2022 at 2:22 PM Tamas Szekeres <<a href="mailto:szekerest@gmail.com">szekerest@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi Steve,<div><br></div><div>I've got the corresponding mapfile causing the crash, it looks like each LABELREQUIRES expression has a corresponding layer with the same name. The only thing I see is that some of the expressions contain brackets (either in the expression and in the layer name) like so:</div><div><br></div><div><i><font color="#ff0000">LABELREQUIRES "[Height of Buildings (2013)]"</font></i><br></div><div><br></div><div>Is that something that is considered as an attribute binding by the lexer?</div><div><br></div><div>The crash location is looking like:<br></div><div><br></div><div><i>case MS_TOKEN_BINDING_DOUBLE:<br> case MS_TOKEN_BINDING_INTEGER:<br> token = NUMBER;<br> <font color="#ff0000"> (*lvalp).dblval = atof(p->shape->values[p->expr->curtoken->tokenval.bindval.index]);</font><br> break;</i><br></div><div><i><br></i></div>However the p->shape has been initialized to NULL (in msEvalContext), definitely.<div><br></div><div>Thanks,</div><div><br></div><div>Tamas</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Steve Lime <<a href="mailto:sdlime@gmail.com" target="_blank">sdlime@gmail.com</a>> ezt írta (időpont: 2022. febr. 22., K, 15:21):<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Tamas: Ok, reacquainting myself with that code. There are actually two lexers in play - msyylex() is the main one that is used to parse a mapfile and also expression strings - it is not thread-safe. With regards to expressions, msyylex() is used to generate a series of tokens - see msTokenizeExpression() in maplayer.c. There is a lock present when msyylex() is used as part of that function. The Bison parser has its own lexer (yylex(), defined in mapparser.y) that is used to traverse the series of tokens. That function (and yyparse()) operate on a parseObj that fully encapsulates expression evaluation. So the parser should be reentrant...<div><br></div><div>Assuming that LABELREQUIRES is in play here, the simplest explanation is that its value contains a bad layer name in the expression. That would leave something like "[badlayer]" remaining in the string passed to msTokenizeString() and would result in the error you noted originally. We could/should add a check in msEvalContext() to examine the expression string to catch this condition. Using strchr to check for remaining '[' or ']' characters would probably do it.</div><div><br></div><div>--Steve </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 19, 2022 at 12:19 PM Tamas Szekeres <<a href="mailto:szekerest@gmail.com" target="_blank">szekerest@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Steve,</div><div><br></div><div>According to the call stack below, yyparse also calling yylex internally, which is not considered to be reentrant I think.</div><div><pre style="white-space:pre-wrap"><i><font color="#ff0000">mapserver!yylex+0x1b5 [e:\builds-t1\src\mapserver-intramaps\vc15x64\mapparser.y @ 858]
mapserver!yyparse+0x247 [e:\home\even\mapserver\mapserver\mapparser.c @ 1461]
mapserver!msEvalContext+0x178 [e:\builds-t1\src\mapserver-intramaps\maputil.c @ 442]</font></i></pre></div><div><br></div><div>Best regards,</div><div><br></div><div>Tamas</div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Steve Lime <<a href="mailto:sdlime@gmail.com" target="_blank">sdlime@gmail.com</a>> ezt írta (időpont: 2022. febr. 19., Szo, 17:53):<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">I thought generating tokens for an expression needed a lock since it uses the lexer but that the parser did not. I could be wrong though.</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 19, 2022 at 10:21 AM Steve Lime <<a href="mailto:sdlime@gmail.com" target="_blank">sdlime@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">I don’t remember offhand and will have to check. That said if there is a attribute level token in a layer level expression than that would explain your research. Is there really no way to get info on the expression(s) in question?</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 19, 2022 at 9:22 AM Tamas Szekeres <<a href="mailto:szekerest@gmail.com" target="_blank">szekerest@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'm not too familiar in the current parser/lexer implementation, but is msTokenizeExpression and yyparse supposed to be protected by the same parser lock in <span style="color:rgb(0,0,0)">msEvalContext</span>? Or if
msTokenizeExpression doesn't leave any global variables behind, should yyparse be protected by it's own?<div><br></div><div>Thanks,</div><div><br></div><div>Tamas</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Steve Lime <<a href="mailto:sdlime@gmail.com" target="_blank">sdlime@gmail.com</a>> ezt írta (időpont: 2022. febr. 19., Szo, 16:02):<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Requires and labelrequires aren’t operating at the feature level, rather the layer level so they definitely don’t support attribute binding. We’d need to whip up a test case to figure out a method to trap this. The workaround is to fix the expression. I mean, obviously we don’t want to crash but it’s never going to work if written as you suspect.</div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 19, 2022 at 7:55 AM Tamas Szekeres <<a href="mailto:szekerest@gmail.com" target="_blank">szekerest@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Devs,<div><br></div><div>I got a crash log which shows that the access violation is happening in the following location:</div><div><br></div><div><pre style="white-space:pre-wrap"><i><font color="#ff0000">mapserver!yylex+0x1b5 [e:\builds-t1\src\mapserver-intramaps\vc15x64\mapparser.y @ 858]
mapserver!yyparse+0x247 [e:\home\even\mapserver\mapserver\mapparser.c @ 1461]
mapserver!msEvalContext+0x178 [e:\builds-t1\src\mapserver-intramaps\maputil.c @ 442]
mapserver!msDrawVectorLayer+0xb0 [e:\builds-t1\src\mapserver-intramaps\mapdraw.c @ 918]
mapserver!msDrawLayer+0x3a3 [e:\builds-t1\src\mapserver-intramaps\mapdraw.c @ 813]
mapserver!msDrawMap+0x415 [e:\builds-t1\src\mapserver-intramaps\mapdraw.c @ 403]
mapscript!CSharp_mapObj_draw+0xd</font></i></pre>By looking into the corresponding code, the problem has happened in the following location in yylex():<pre style="white-space:pre-wrap"><i><font color="#ff0000"> case MS_TOKEN_BINDING_INTEGER:<br> token = NUMBER;<br> ----> (*lvalp).dblval = atof(p->shape->values[p->expr->curtoken->tokenval.bindval.index]);<br> break;</font></i><font color="#000000"><br></font></pre>In maputil.c the call stack location is here (msEvalContext):<pre style="white-space:pre-wrap"><i><font color="#ff0000">msTokenizeExpression(&e, NULL, NULL);<br><br> p.shape = NULL;<br> p.expr = &e;<br> p.expr->curtoken = p.expr->tokens; /* reset */<br> p.type = MS_PARSE_TYPE_BOOLEAN;<br><br>-----> status = yyparse(&p);<br><br> msFreeExpression(&e);</font></i><font color="#000000"><br></font></pre>And in mapdraw.c<pre style="white-space:pre-wrap"><i><font color="#ff0000"> /* TODO TBT: draw as raster layer in vector renderers */<br><br>-----> annotate = msEvalContext(map, layer, layer->labelrequires);<br> if(map->scaledenom > 0) {<br> if((layer->labelmaxscaledenom != -1) && (map->scaledenom >= layer->labelmaxscaledenom)) annotate = MS_FALSE;<br> if((layer->labelminscaledenom != -1) && (map->scaledenom < layer->labelminscaledenom)) annotate = MS_FALSE;<br> }</font></i><font color="#000000"><br></font></pre>I'm not sure if the LABELREQUIRES option supports attribute binding (probably not) and in that case the p->expr->curtoken->tokenval.bindval.index would probably have undefined value. I don't have any information about the corresponding LABELREQUIRES expression , but if I assume it doesn't contain attribute binding, then this crash might probably happen due to a missing parser lock around <i style="color:rgb(0,0,0);white-space:pre-wrap">yyparse(&p). </i>What do you think about the problem and how could we work around this?</div><div><br></div><div>Thanks,</div><div><br></div><div>Tamas<br><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre></div></div>
_______________________________________________<br>
MapServer-dev mailing list<br>
<a href="mailto:MapServer-dev@lists.osgeo.org" target="_blank">MapServer-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/mapserver-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/mapserver-dev</a><br>
</blockquote></div></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div></div>
</blockquote></div></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>