[mapserver-users] Comparing cgi and fcgi for serving aerial images

Fri Dec 18 09:47:32 EST 2009

Hi Daniel,

Your explanation makes sense. I think that the following logfile excerpt
means that it is just so heavy to process my images that playing with
cgi/fcgi is peanuts. If you wonder, non-square pixels are intentional. I
made my WMS requests to reproject images and squeese pixels to avoid
over-optimistic results from benchmarking. Server speed is good enough
for the load we have at them moment so I am going to keep it running as
cgi and acquire a second server if this one chokes.  All the simple
things have been done: images are tiled, uncompressed and with
overviews, and mapfile and epsg files are minimal.

[Fri Dec 18 15:41:49 2009].909000 CGI Request 1 on process 4428
[Fri Dec 18 15:41:49 2009].909000 msWMSLoadGetMapParams(): enabling
non-square pixels.[Fri Dec 18 15:41:49 2009].909000 msDrawMap(): kicking
into non-square pixel preserving mode.[Fri Dec 18 15:41:49 2009].909000
msDrawRasterLayerLow(uilmakuvat2009): entering.
[Fri Dec 18 15:41:49 2009].924000 msResampleGDALToMap in effect:
cellsize = 1.876250
[Fri Dec 18 15:41:49 2009].924000 msDrawGDAL(uilmakuvat2009): using
RAW_WINDOW=8998 0 3002 2602
[Fri Dec 18 15:41:49 2009].924000 msDrawGDAL(): red,green,blue,alpha
bands = 1,2,3,0
[Fri Dec 18 15:41:50 2009].284000 msResampleGDALToMap in effect:
cellsize = 1.876250
[Fri Dec 18 15:41:50 2009].299000 msDrawGDAL(uilmakuvat2009): using
RAW_WINDOW=8998 6599 3002 5401
[Fri Dec 18 15:41:50 2009].299000 msDrawGDAL(): red,green,blue,alpha
bands = 1,2,3,0
[Fri Dec 18 15:41:51 2009].393000 msResampleGDALToMap in effect:
cellsize = 1.875625
[Fri Dec 18 15:41:51 2009].409000 msDrawGDAL(uilmakuvat2009): using
RAW_WINDOW=0 0 3001 2602
[Fri Dec 18 15:41:51 2009].409000 msDrawGDAL(): red,green,blue,alpha
bands = 1,2,3,0
[Fri Dec 18 15:41:51 2009].768000 msResampleGDALToMap in effect:
cellsize = 1.875625
[Fri Dec 18 15:41:51 2009].768000 msDrawGDAL(uilmakuvat2009): using
RAW_WINDOW=0 6599 3001 5401
[Fri Dec 18 15:41:51 2009].768000 msDrawGDAL(): red,green,blue,alpha
bands = 1,2,3,0
[Fri Dec 18 15:41:52 2009].815000 msDrawMap(): Layer 32
(uilmakuvat2009), 2.906s
[Fri Dec 18 15:41:52 2009].815000 msDrawMap(): Drawing Label Cache,
0.000s
[Fri Dec 18 15:41:52 2009].815000 msDrawMap() total time: 2.906s
[Fri Dec 18 15:41:52 2009].924000 msSaveImage() total time: 0.109s
[Fri Dec 18 15:41:52 2009].924000 mapserv request processing time
(msLoadMap not incl.): 3.015s

-Jukka-

Daniel Morissette wrote:

> It's hard to give a definite answer without seeing your actual setup, 
> mapfile, data, etc.
> 
> However, as a general rule, you will gain from using fastcgi 
> only once 
> you get to the point where the processing time for each 
> request (to read 
> data and render the map) becomes close to or smaller than the time it 
> takes to launch CGI processes. So if you get around 5 hits/sec with 8 
> cores, we can presume that each request takes over 1 second of 
> processing time (and/or disk I/O) which is several times 
> larger than the 
> time to launch a CGI process, and you need to do some work on 
> your data 
> and mapfile before you can benefit from fastcgi.
> 
> Of course I could be completely wrong, but I'd suggest you start by 
> verifying where the time is spent by enabling 
> MS_ERRORFILE/DEBUG at both 
> the mapfile and layer level, look at the logs to identify 
> where the time 
> is spent and optimize your mapfile/data accordingly.
> 
> HTH
> 
> Daniel
> 
> Rahkonen Jukka wrote:
> > I made some tests by letting JMeter run a few thousand WMS GetMap 
> > requests with varying BBOX and WIDTH and HEIGHT parameter.  
> I tested 
> > three alternatives:
> >  
> > MS4W 2.2.8 and Mapserver 5.2.1
> > MS4W 3.0 beta 7 and Mapserver 5.6.0 as cgi
> > MS4W 3.0 beta 7 and Mapserver 5.6.0 a fcgi
> >  
> > Server is Windows 2003 server with 8 cores and fast disks. 
> I configured 
> > JMeter to use 20 threads
> >  
> > I feel I was able to get fcgi running by following the 
> instructiuons of 
> > MS4W README_INSTALL document.  I believe it is convinced by a) 
> > calls addressed to  /fcgi-bin/mapserv.exe work and b) after 
> receiving an 
> > image from WMS service the process remains on the server 
> for some time. 
> > However, all these three alternatives are giving me about the same 
> > speed that is  5 images per second or 500 KB/sec with jpeg output.  
> > Bandwidth is not the limiting factor because with 24-bit 
> png output I 
> > get a throughput of 4400 KB/sec.  It looks like I am having 
> some  base 
> > load in my system that takes about 150 milliseconds per 
> each successful 
> > WMS request and using fcgi does not help in my case.
> >  
> > Can anybody suggest why my server is so much slower than 
> the one used in 
> > Mapserver/Geoserver shootout  
> > 
> http://www.slideshare.net/gatewaygeomatics.com/wms-performance
> -shootout and 
> > why I do not get any advantage from using Fast-CGI?  Could 
> it be because 
> > my tileindex shapefile is rather large with more than 11000 
> polygons?  
> > Images themselves are processed in a standard Mapserver way 
> and they 
> > are uncompressed, tiled geotiffs with external uncompressed 
> overviews.  
> > Some evidence about tileindex handling eating time is that 
> I can get 
> > double speed with 10 images per second from a single ecw 
> file but this 
> > test is not very comparable because my ecw file is black 
> and white and 
> > not 3-channel as my geotiff layer.
> >  
> > -Jukka Rahkonen-
> >  
> >  
> > 
> > 
> > 
> --------------------------------------------------------------
> ----------
> > 
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 
> 
> -- 
> Daniel Morissette
> http://www.mapgears.com/
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 