[MapServer-dev] GNU Regex Replacement?

Sat Jun 8 09:23:26 PDT 2024

Maybe a 9.0 change then? Sounds like the underlying issues should be fixed regardless though.

From: MapServer-dev <mapserver-dev-bounces at lists.osgeo.org> On Behalf Of Even Rouault via MapServer-dev
Sent: Saturday, June 8, 2024 10:37 AM
To: Paul Ramsey <pramsey at cleverelephant.ca>; Steve Lime <sdlime at gmail.com>
Cc: MapServer Devs <mapserver-dev at lists.osgeo.org>
Subject: Re: [MapServer-dev] GNU Regex Replacement?

This message may be from an external email source.
Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center.

________________________________

UPDATE: msautotest now pass against PCRE2 with a few extra fixes.

This was actually quite helpful to uncover 2 underlying issues, not related to PCRE2 itself:

- one (likely) wrong default regex with ms_map_bad_pattern_default in msCGILoadMap() where escaping antislash was missing (probably tolerated by GNU regex , but PCRE2 is more picky about that). Easy to fix

- and more embarrassing, our redefined MS_REG_xxxx constants in mapregex.h happen to not match the values of GNU regex (they might perhaps have matched in earlier times, but things have drifted apart since then)... In particular, MS_REG_NOSUB = 4 was thus wrongly interpreted as REG_NEWLINE = (1 << 2). I loose some extra hair on figuring that out. The symptom was in rfc62.map where a query string parameter whose value is "foo%0Abar" (foo newline bar) happened to validate (with GNU regex) the following expression '^[a-z]{1,16}$' whereas it shouldn't have (our expected result was incorrect). The value matched because msEvalRegex() sets MS_REG_NOSUB which was wrongly interpreted as REG_NEWLINE, thus making the caret / dollar to apparently match any line instead of the whole string. When using PCRE2, the match didn't occur because PCRE2 and GNU regex have exactly subtle different semantics regarding newline matching (I tried to understand https://www.pcre.org/current/doc/html/pcre2posix.html#SEC5, but beyond my mastering)

So all in all, it seems PCRE2 could be a viable alternative. I would expect "subtle" differences, so any mapfile having regular expressions in it that was tested with GNU/BSD/POSIX/whatever regex should be carefully checked for correct execution if switching to PCRE2

Even
Le 08/06/2024 à 15:35, Even Rouault via MapServer-dev a écrit :

Hi,

my attempt at https://github.com/MapServer/MapServer/pull/7073 . It links, but PCRE2 doesn't seem to speak the same regex dialect as GNU regex...

Even
Le 08/06/2024 à 05:31, Paul Ramsey via MapServer-dev a écrit :
Looks like it was in Centos as far back as 7, and is in base Ubuntu. It’s in MacPorts and Homebrew. So, fairly common?
P.

On Jun 7, 2024, at 8:06 PM, Steve Lime <sdlime at gmail.com><mailto:sdlime at gmail.com> wrote:

One would think replacement wouldn’t be that hard. It’s pretty localized in MapServer. Is pcre2 widely available?

On Fri, Jun 7, 2024 at 2:40 PM Paul Ramsey via MapServer-dev <mapserver-dev at lists.osgeo.org<mailto:mapserver-dev at lists.osgeo.org>> wrote:
We dragged around a dependency on pcre which got updated to pcre2 lately.

> On Jun 7, 2024, at 12:38 PM, Howard Butler via MapServer-dev <mapserver-dev at lists.osgeo.org<mailto:mapserver-dev at lists.osgeo.org>> wrote:
>
> I was playing around with getting Conda Forge builds of MapServer working, and when I got to Windows, I realized we need the GNU Regex library
>
> https://github.com/conda-forge/mapserver-feedstock/issues/47#issue-2341018003
>
> It is ancient, not released as a standalone library anymore, and causes anything linking against it to also be GPL (which we're doing in the Windows build situation). Maybe it is not worth the trouble to change anything, but if one were to do so, what should be used in its place?
>
> Howard
> _______________________________________________
> MapServer-dev mailing list
> MapServer-dev at lists.osgeo.org<mailto:MapServer-dev at lists.osgeo.org>
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev

_______________________________________________
MapServer-dev mailing list
MapServer-dev at lists.osgeo.org<mailto:MapServer-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/mapserver-dev

_______________________________________________

MapServer-dev mailing list

MapServer-dev at lists.osgeo.org<mailto:MapServer-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/mapserver-dev

--

http://www.spatialys.com<http://www.spatialys.com/>

My software is free, but my time generally not.

_______________________________________________

MapServer-dev mailing list

MapServer-dev at lists.osgeo.org<mailto:MapServer-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/mapserver-dev

--

http://www.spatialys.com<http://www.spatialys.com/>

My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20240608/40a1f196/attachment-0001.htm>