[mapserver-users] Slow and degrading performance with radar (but not satellite) on mapserver-6.0.1-3_0.el6.x86_64

Wed Jul 22 19:53:07 PDT 2020

you need to include in your QUERY_STRING=MAP=....&

it is probably hard coded in the fcgi config.

-Steve W

On 7/22/2020 7:15 PM, English Paul wrote:
>
> So you have a complex historical "mess" and its not clear where the 
> performance issue is. So you need to divide the problem into small 
> problems that you can verify are or are not contributing. I would 
> start with something like this:
>
> Thank you – I would not have thought to try it outside of CGI entirely!
>
> Take one slow image request and try that as cgi or cli and not fcgi 
> and turn on debugging.
>
>   - copy and rename you mapfile so it doesn't mess with the production 
> requests
>
> Not a major worry in this case – I’m working in an entirely dev 
> environment. I did make backup copies though!
>
>   - turn on the debugging and send it to stderr in the debug mapfile
>
>  - you can manually run that image request from the commandline like:
>
> if mapserv is not in your path you might need to find it and your the 
> path to it below
>
> mapserv -nh QUERY_STRING="everything after the ? in the original 
> query" >junk.png 2>error.txt
>
> -nh suppresses headers from being output before the image data
>
> error.txt will be stderr output and should contain the debug messages
>
> I couldn’t quite get there – which this is what I ended up running:
>
> # /usr/libexec/mapserv -nh 
> QUERY_STRING="FORMAT=image%2Fpng&LAYERS=winter1km_5min&TRANSPARENT=TRUE&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&TIME=2020-07-22T22%3A30%3A00Z&SRS=EPSG%3A900913&BBOX=-11584184.507886,4070118.8849183,-11271098.44003,4383204.9527744&WIDTH=256&HEIGHT=256" 
> > junk.png 2>error.txt
>
> I first tried it with our current map file(s), and then with 
> MS_ERRORFILE set to stderr and DEBUG set to 5, in both cases, I got an 
> empty error.txt, and the following in junk.png:
>
> Content-type: text/html
>
> <HTML>
>
> <HEAD><TITLE>MapServer Message</TITLE></HEAD>
>
> <!-- MapServer version 6.0.1 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG 
> OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=CAIRO SUPPORTS=FREETYPE
>
> SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER 
> SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT 
> SUPPORTS=WCS_SERVER SU
>
> PPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS 
> INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE -->
>
> <BODY BGCOLOR="#FFFFFF">
>
> loadMap(): Web application error. CGI variable "map" is not set.
>
> </BODY></HTML>
>
> I noticed that if I ran it with just -nh, I get the following:
>
> #/usr/libexec/mapserv -nh
>
> This script can only be used to decode form results and
>
> should be initiated as a CGI process via a httpd server.
>
> So maybe this older version just really doesn’t want to be run on the 
> CLI? Or maybe I’m messing up the QUERY_STRING, or need to set the CGI 
> variable to something as the junk.png output suggests?
>
> Thanks again and thanks SO MUCH for the super quick reply!!
>
> Paul
>
> On 7/22/2020 1:47 PM, English Paul wrote:
>
>     Hi,
>
>        Newbie here, just got a work assignment to look into some
>     performance issues with mapserver-6.0.1-3_0.el6.x86_64 –
>     specifically, at one time, it rendered radar images very fast,
>     then it degraded and seemed to consume a lot more CPU. So – the
>     EC2 instance side was upgraded significantly – and it still
>     performs badly and uses a lot of CPU. It also seems to be getting
>     slowly worse over time (days/months, not seconds/minutes). I asked
>     this on IRC, but it looks like email might be a better route.
>
>       During all of this, the same mapserver instance renders
>     satellite images quickly. These seem to be a similar, or in some
>     cases larger size png to start with, rendered onto the same final
>     map/size.
>
>       So – the obvious answer is that it isn’t using the CPU to
>     render/re-render, but rather spending it on something else – I/O
>     most likely – eg: a network request, disk I/O, SQL query?
>
>       The previous person working on it tried turning on debug at
>     various levels – but unfortunately that made it even slower,
>     making it tricky to answer “what is making it slow when debug is
>     turned off?”
>
>      My first instinct was to try an strace and nothing was obvious.
>     Next up – a flame graph from strace, and/or trying dtrace – but my
>     understanding is that dtrace is a little weak on
>     RHEL/CentOS/Amazon Linux 6.0 **and** I’m not particularly good at
>     that. Also, we’re using fcgi, so attaching to the correct process
>     is a bit tricky.
>
>       My next instinct was to look at release notes and see if the
>     current stable has anything fixes/improvements that directly
>     address this – there aren’t any that are obvious to my eyes, but
>     you developers have been busy! So many things! Including some
>     performance fixes and one “significant” performance fix.
>
>     Current config file:
>
>     AddHandler fcgid-script fcgi
>
>     FcgidIPCDir /var/run/mod_fcgid
>
>     FcgidProcessTableFile /var/run/mod_fcgid/fcgid_shm
>
>     FcgidMaxProcesses 10
>
>     FcgidMaxProcessesPerClass 10
>
>     FcgidMaxRequestInMem 196608
>
>     FcgidInitialEnv PROJ_LIB /usr/share/proj
>
>     FcgidInitialEnv LD_LIBRARY_PATH "/usr/local/lib:/usr/pgsql-9.1/lib"
>
>     So – suggestions for my next move? I currently plan to take a
>     quick swing at building 7.6 for RHEL 6.0, knowing there might be
>     old libraries and whatnot that make that a non-starter. Of course
>     – we’ve got other infra running on this same instance, so
>     upgrading everything is a much bigger task.
>
>      Strace flame graph?
>
>       Stretch and try dtrace?
>
>       A better way to use debug?
>
>       Something else I’m missing – eg: differences between the image
>     types that make them perform so differently?
>
>     Thanks,
>
>     Paul
>
>
>
>     _______________________________________________
>
>     mapserver-users mailing list
>
>     mapserver-users at lists.osgeo.org  <mailto:mapserver-users at lists.osgeo.org>
>
>     https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20200722/99634ce6/attachment.htm>