[MapProxy] SV: Response time question

Mon Apr 11 07:56:00 EDT 2011

Hi,

Making a proper WMS performance test is not easy and also the experts have had problems. This was a good trial
http://www.slideshare.net/gatewaygeomatics.com/wms-performance-shootout-2010

However, I am not sure about what was really measured because the winners were finally keeping all the source data in cache memory.
This is also good reading:
http://www.tmapy.cz/docs/software/erdas/mapserver_comparsion.pdf

I do not mean to say that there is nothign wrong with your results, but it is not always so easy to know what really was measured.

-Jukka Rahkonen-

> -----Alkuperäinen viesti-----
> Lähettäjä: Johan Sandberg [mailto:johs at kth.se] 
> Lähetetty: 11. huhtikuuta 2011 14:30
> Vastaanottaja: Rahkonen Jukka
> Kopio: mapproxy at lists.osgeo.org
> Aihe: SV: [MapProxy] SV: Response time question
> 
> Hey!
> 
> I am performing tests with the software LoadUI. I have 
> pregenerated URLS from a listw which the programe reads.
> The backend WMS are a geoserver which is pretty fast. 
> MapProxy is installed on a linux computer running on apache2 
> with the WGSImode.
> 
> The test scenario are: 400*300, 800*600,1200*900 and 
> 1600*1200 for 10 concurrent users.
> Then test the 800*600 for 20,50 and 100 concurrent users.
> 
> I test two different cases when the cache is seeded and when 
> its dynamically filled.
> 
> The results (avarage time milliseconds) on  directly to WMS 
> and to seeded cache are as follows
> 
> WMS 400*300 10 request/sec	1157,276546
> WMS 800*600 10 request/sec	2027,072179
> WMS 1200*900 10 request/sec	3463,510585
> WMS 1600*1200 10 request/sec	5571,929988
> WMS 800*600 20 request/sec	4045,260605
> WMS 800*600 50 request/sec	10349,9973
> WMS 800*600 100 requst/sec	20279,21129
> 
> 
> MapProxy 400*300 10 request/sec	229,7247875
> MapProxy 800*600 10 request/sec	1322,603837
> MapProxy 1200*900 10 request/sec	2753,405889
> MapProxy 1600*1200 10 request/sec	6793,848616
> MapProxy 800*600 20 request/sec	2678,599481
> MapProxy 800*600 50 request/sec	6545,8422
> MapProxy 800*600 100 request/sec	19226,09422
> 
> For the fourth test (1600*1200) the WMS the avarage time is 
> better for WMS than for MapProxy. 
> The tests are fetching the same data which are severall 
> vector layers compose as one Map layer.
> 
> / johan
> 
> ________________________________________
> Från: Rahkonen Jukka [Jukka.Rahkonen at mmmtike.fi]
> Skickat: den 11 april 2011 13:12
> Till: Johan Sandberg
> Kopia: mapproxy at lists.osgeo.org
> Ämne: Re: [MapProxy] SV: Response time question
> 
> Hi,
> 
> In my tests MapProxy has been either much or very much faster 
> than my backend WMS but that depends on many things.  With 
> extremely simple vector layers with simple styles and with 
> just a few concurrent users my WMS can be almost as fast. 
> Biggest speedup (at least ten times faster)is when rendering 
> complicated vector layers like OpenStreetMap data at small 
> scales . Speedup with raster maps or aerial images is 
> something in between, let's say 5 to times faster.
> 
> What is your use case like? What data you have on WMS 
> (vectors/rasters)? I suppose you are testing with WMS calls. 
> Do you use the same projection than your Mapproxy tiles are, 
> or does Mapproxy do reprojection for you? How do you test, 
> are you running the same script with reasonably many requests 
> with jmeter or something?
> 
> WMS can be a bit slow to start, especially Mapserver in CGI 
> mode is such. Therefore it is normal that throughput as bits 
> per second with big WMS requests is higher than with small 
> requests but this depends on the WMS software and also on the 
> data. Mapproxy, on the other hand, is using prerendered tiles 
> of fixed size as a raw material and therefore it is showing a 
> very stable throughput with varying image sizes. With the 
> backend data it is naturally totally ignorant (when serving 
> from the cache), heavy OSM tiles are rendered just as fast as 
> a simple point layer. This is also making it easier to manage 
> the WMS server. It is not so important to use all the tricks 
> I know for making the heavy WMS bulk layers 10 per cent 
> faster because WMS is used just once after each update and 
> thereafter Mapproxy is taking the load.
> 
> Can you give any exact numbers about your tests? How many 
> 400x300 sized images per second you can get out from your WMS 
> and how many through Mapproxy?
> 
> Johan Sandberg wrote:
> 
> > Hej!
> > Yes that's offcourse true. This leads to not so greate
> > performance on bigger image sizes with Map Proxy.
> > Measuring directly to the WMS I see that the scaling is only
> > twice the time.
> > To a certain image size there is no use of using M.P for
> > better perfromance according to the tests I perform.
> >
> > / Johan
> > ________________________________________
> > Från: Rahkonen Jukka [Jukka.Rahkonen at mmmtike.fi]
> > Skickat: den 11 april 2011 11:07
> > Till: Johan Sandberg
> > Kopia: mapproxy at lists.osgeo.org
> > Ämne: Re: [MapProxy] SV: Response time question
> >
> > Hi,
> >
> > He means that if you'll take four times more pixels
> > (800x600=480000 vs 400x300=120000)
> > then you'll need approximately four times longer time.
> > Rather logical and to my experience pretty accurate approximation.
> >
> > -Jukka Rahkonen-
> >
> >
> >
> > Johan Sandberg wrote:
> > >
> > > Hej!
> > >
> > > "MapProxy should scale linear with the number of pixels. So
> > > you should expect 1/4 requests when changing from 400x300 to
> > > 800x600. You should measure the times with one concurrent
> > > requests at first and then you should look how you can scale
> > > that with increased concurrency, see below."
> > >
> > > So do you mean that I could expect four times the time to
> > > fetch an image that comes from seeded tiles when the
> > > requested image size double?
> > >
> > > Is there any good documenations on this? Would be pretty helpful!
> > >
> > > / Johan
> > >
> > >
> > > ________________________________________
> > > Från: Oliver Tonnhofer [olt at omniscale.de]
> > > Skickat: den 31 mars 2011 17:34
> > > Till: Johan Sandberg
> > > Kopia: mapproxy at lists.osgeo.org
> > > Ämne: Re: Response time question
> > >
> > > Hej Johan,
> > >
> > > On 31.03.2011, at 15:07, Johan Sandberg wrote:
> > >
> > > > hey!
> > > >
> > > > I'm measuring responses to Map Proxy when its seeded.
> > > > I dont get that satisfying response time as a wish. When I
> > > perform requests of a image size of 400*300, 10 request at
> > > the time I get low response time(around 50, 60ms).
> > > >
> > > > When I do image size of 800*600 the response time is up
> > to 1-2 sec.
> > > > And even bigger image size give response time up to 5-6 sec.
> > > > Is this normal response time when requesting seeded tiles?
> > >
> > > MapProxy should scale linear with the number of pixels. So
> > > you should expect 1/4 requests when changing from 400x300 to
> > > 800x600. You should measure the times with one concurrent
> > > requests at first and then you should look how you can scale
> > > that with increased concurrency, see below.
> > >
> > > > The request should be alligned with the resolution that is
> > > seeded so they don't have to re scaled.
> > > > How much is the normal cost in time of rescaling an image?
> > > Do the request have to be perfectly alligned with the cached
> > > resolution to achieve the low response time that I wish?
> > >
> > > Depends on the resampling method, you have to check for your
> > > self. The performance can degrade dramatically when you
> > > request a lower resolution than your first/upper level,
> > > because it then needs to merge lots of tiles. So make sure
> > > you don't run into that.
> > >
> > > > I have installed map proxy on a Linux computer with httpd
> > > apache server. I followed the wgsi example.
> > >
> > > You should run mod_wsgi with multiple daemon processes when
> > > you have more than one CPU:
> > > http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives
> > >
> > > Regards,
> > > Oliver
> > >
> > > --
> > > Oliver Tonnhofer    | Omniscale GmbH & Co KG    |
> > http://omniscale.de
> > > http://mapproxy.org | https://bitbucket.org/olt | @oltonn
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > MapProxy mailing list
> > > MapProxy at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/mapproxy
> > >
> > _______________________________________________
> > MapProxy mailing list
> > MapProxy at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapproxy
> >
> 