<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Paul,</p>
<p>This sounds like you have error in your mapfile. you might try
posting it to the list if you can't figure it out yourself.</p>
<p>There also might be some mapfile tools or sites that can validate
the mapfile, but I've never needed them so I don't have them
bookmarked anywhere. Can anyone on the list make a suggestion to
Paul?</p>
<p>-Steve W<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 7/23/2020 12:48 PM, English Paul
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:0729D2BB-60C5-4159-89AB-3CAD157069EA@vaisala.com">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<div class="WordSection1">
<p class="MsoNormal">Thanks again Steve – progress:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">$ /usr/libexec/mapserv -nh
QUERY_STRING="MAP=wmst?FORMAT=image%2Fpng&LAYERS=winter1km_5min&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&TIME=2020-07-23T16%3A40%3A00Z&SRS=EPSG%3A900913&BBOX=-9392582.0328932,4383204.9527744,-9236038.9989652,4539747.9867024&WIDTH=256&HEIGHT=256"
> junk.png</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Gives the following in junk.png:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Content-type: text/html</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><HTML></p>
<p class="MsoNormal"><HEAD><TITLE>MapServer
Message</TITLE></HEAD></p>
<p class="MsoNormal"><!-- MapServer version 6.0.1 OUTPUT=GIF
OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG
SUPPORTS=CAIRO SUPPORTS=FREETYPE S</p>
<p class="MsoNormal">UPPORTS=ICONV SUPPORTS=FRIBIDI
SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPP</p>
<p class="MsoNormal">ORTS=SOS_SERVER SUPPORTS=FASTCGI
SUPPORTS=THREADS SUPPORTS=GEOS INPUT=POSTGIS INPUT=OGR
INPUT=GDAL INPUT=SHAPEFILE --></p>
<p class="MsoNormal"><BODY BGCOLOR="#FFFFFF"></p>
<p class="MsoNormal">msLoadMap(): Regular expression error.
MS_DEFAULT_MAPFILE_PATTERN validation failed.</p>
<p class="MsoNormal">msEvalRegex(): R</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal"><b><span>From:
</span></b><span>mapserver-users
<a class="moz-txt-link-rfc2396E" href="mailto:mapserver-users-bounces@lists.osgeo.org"><mapserver-users-bounces@lists.osgeo.org></a> on behalf
of Stephen Woodbridge
<a class="moz-txt-link-rfc2396E" href="mailto:stephenwoodbridge37@gmail.com"><stephenwoodbridge37@gmail.com></a><br>
<b>Date: </b>Wednesday, July 22, 2020 at 19:53<br>
<b>To: </b><a class="moz-txt-link-rfc2396E" href="mailto:mapserver-users@lists.osgeo.org">"mapserver-users@lists.osgeo.org"</a>
<a class="moz-txt-link-rfc2396E" href="mailto:mapserver-users@lists.osgeo.org"><mapserver-users@lists.osgeo.org></a><br>
<b>Subject: </b>Re: [mapserver-users] Slow and degrading
performance with radar (but not satellite) on
mapserver-6.0.1-3_0.el6.x86_64</span></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<p>you need to include in your QUERY_STRING=MAP=....&</p>
<p>it is probably hard coded in the fcgi config.</p>
<p>-Steve W</p>
<p> </p>
<div>
<p class="MsoNormal">On 7/22/2020 7:15 PM, English Paul wrote:</p>
</div>
<blockquote>
<div>
<p>So you have a complex historical "mess" and its not clear
where the performance issue is. So you need to divide the
problem into small problems that you can verify are or are
not contributing. I would start with something like this:</p>
<p>Thank you – I would not have thought to try it outside of
CGI entirely!</p>
<p>Take one slow image request and try that as cgi or cli
and not fcgi and turn on debugging.</p>
<p> - copy and rename you mapfile so it doesn't mess with
the production requests</p>
<p>Not a major worry in this case – I’m working in an
entirely dev environment. I did make backup copies though!</p>
<p> - turn on the debugging and send it to stderr in the
debug mapfile</p>
<p> - you can manually run that image request from the
commandline like:</p>
<p>if mapserv is not in your path you might need to find it
and your the path to it below</p>
<p>mapserv -nh QUERY_STRING="everything after the ? in the
original query" >junk.png 2>error.txt</p>
<p>-nh suppresses headers from being output before the image
data</p>
<p>error.txt will be stderr output and should contain the
debug messages</p>
<p>I couldn’t quite get there – which this is what I ended
up running:</p>
<p># /usr/libexec/mapserv -nh
QUERY_STRING="FORMAT=image%2Fpng&LAYERS=winter1km_5min&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&TIME=2020-07-22T22%3A30%3A00Z&SRS=EPSG%3A900913&BBOX=-11584184.507886,4070118.8849183,-11271098.44003,4383204.9527744&WIDTH=256&HEIGHT=256"
> junk.png 2>error.txt</p>
<p>I first tried it with our current map file(s), and then
with MS_ERRORFILE set to stderr and DEBUG set to 5, in
both cases, I got an empty error.txt, and the following in
junk.png:</p>
<p>Content-type: text/html</p>
<p> </p>
<p><HTML></p>
<p><HEAD><TITLE>MapServer
Message</TITLE></HEAD></p>
<p><!-- MapServer version 6.0.1 OUTPUT=GIF OUTPUT=PNG
OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG
SUPPORTS=CAIRO SUPPORTS=FREETYPE</p>
<p>SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SU</p>
<p>PPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS
SUPPORTS=GEOS INPUT=POSTGIS INPUT=OGR INPUT=GDAL
INPUT=SHAPEFILE --></p>
<p><BODY BGCOLOR="#FFFFFF"></p>
<p>loadMap(): Web application error. CGI variable
"map" is not set.</p>
<p></BODY></HTML></p>
<p>I noticed that if I ran it with just -nh, I get the
following:</p>
<p>#/usr/libexec/mapserv -nh</p>
<p>This script can only be used to decode form results and</p>
<p>should be initiated as a CGI process via a httpd server.</p>
<p>So maybe this older version just really doesn’t want to
be run on the CLI? Or maybe I’m messing up the
QUERY_STRING, or need to set the CGI variable to something
as the junk.png output suggests?</p>
<p>Thanks again and thanks SO MUCH for the super quick
reply!!</p>
<p>Paul</p>
<p> </p>
<div>
<p class="MsoNormal">
On 7/22/2020 1:47 PM, English Paul wrote:</p>
</div>
<blockquote>
<div>
<p class="MsoNormal">
Hi,</p>
<p class="MsoNormal">
Newbie here, just got a work assignment to look
into some performance issues with
mapserver-6.0.1-3_0.el6.x86_64 – specifically, at one
time, it rendered radar images very fast, then it
degraded and seemed to consume a lot more CPU. So –
the EC2 instance side was upgraded significantly – and
it still performs badly and uses a lot of CPU. It also
seems to be getting slowly worse over time
(days/months, not seconds/minutes). I asked this on
IRC, but it looks like email might be a better route.</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
During all of this, the same mapserver instance
renders satellite images quickly. These seem to be a
similar, or in some cases larger size png to start
with, rendered onto the same final map/size.</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
So – the obvious answer is that it isn’t using the
CPU to render/re-render, but rather spending it on
something else – I/O most likely – eg: a network
request, disk I/O, SQL query?</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
The previous person working on it tried turning on
debug at various levels – but unfortunately that made
it even slower, making it tricky to answer “what is
making it slow when debug is turned off?”</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
My first instinct was to try an strace and nothing
was obvious. Next up – a flame graph from strace,
and/or trying dtrace – but my understanding is that
dtrace is a little weak on RHEL/CentOS/Amazon Linux
6.0 *<b>and</b>* I’m not particularly good at that.
Also, we’re using fcgi, so attaching to the correct
process is a bit tricky.</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
My next instinct was to look at release notes and
see if the current stable has anything
fixes/improvements that directly address this – there
aren’t any that are obvious to my eyes, but you
developers have been busy! So many things! Including
some performance fixes and one “significant”
performance fix.</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
Current config file:</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
AddHandler fcgid-script fcgi</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
FcgidIPCDir /var/run/mod_fcgid</p>
<p class="MsoNormal">
FcgidProcessTableFile /var/run/mod_fcgid/fcgid_shm</p>
<p class="MsoNormal">
FcgidMaxProcesses 10</p>
<p class="MsoNormal">
FcgidMaxProcessesPerClass 10</p>
<p class="MsoNormal">
FcgidMaxRequestInMem 196608</p>
<p class="MsoNormal">
FcgidInitialEnv PROJ_LIB /usr/share/proj</p>
<p class="MsoNormal">
FcgidInitialEnv LD_LIBRARY_PATH
"/usr/local/lib:/usr/pgsql-9.1/lib"</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
So – suggestions for my next move? I currently plan to
take a quick swing at building 7.6 for RHEL 6.0,
knowing there might be old libraries and whatnot that
make that a non-starter. Of course – we’ve got other
infra running on this same instance, so upgrading
everything is a much bigger task.</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
Strace flame graph? </p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
Stretch and try dtrace?</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
A better way to use debug?</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
Something else I’m missing – eg: differences between
the image types that make them perform so differently?
</p>
<p class="MsoNormal">
</p>
<p class="MsoNormal">
Thanks,</p>
<p class="MsoNormal">
Paul</p>
</div>
<p class="MsoNormal">
</p>
<pre>_______________________________________________</pre>
<pre>mapserver-users mailing list</pre>
<pre><a href="mailto:mapserver-users@lists.osgeo.org" moz-do-not-send="true">mapserver-users@lists.osgeo.org</a></pre>
<pre><a href="https://lists.osgeo.org/mailman/listinfo/mapserver-users" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a></pre>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br>
</p>
<pre>_______________________________________________</pre>
<pre>mapserver-users mailing list</pre>
<pre><a href="mailto:mapserver-users@lists.osgeo.org" moz-do-not-send="true">mapserver-users@lists.osgeo.org</a></pre>
<pre><a href="https://lists.osgeo.org/mailman/listinfo/mapserver-users" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a></pre>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
mapserver-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/mapserver-users">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a></pre>
</blockquote>
</body>
</html>