BENCHMARKING (was: [mapserver-users] professional MapServer services list? / ArcIMS-MapServer choice)

Tue Sep 17 20:09:11 EDT 2002

I have already used a tool call Siege see

    http://www.joedog.org/siege/index.shtml

that works great at simulating multiple users, etc. The key is to 
come up with a set of urls the you can execute in sequence or 
randomly that simulate user activity. The way I did this was to 
generate random hits at different location over the spatial area of 
the map and at random zoom scales. This seemed to work well and to 
avoid problems like getting all your hits out of the files system 
cache which is about 10-20 times faster than out of cache hits.

Siege measures the user response times of the server. I ran tests 
both from the server (Siege itself does not generate much load on the 
system) and from a remote host to compare results to look at the 
potential impact on network latency.

For diagnostic evaluation of how time was being spend on the server I 
wrote a simple shell script like:

time --append -o /path/to/logfile  /path/to/php $*

and then analyzed the logfile to look at the process stats. 

This information is only good for a given configuration but can be 
used to tune it and to predict what the maximum potential through put 
is going to be. If you could standardize a dataset and mapfile 
configuration, it would be possible to compare mapserver version to 
version if you keep the hardware config the same, or evaluate 
hardware configs, if you keep mapserver version constant. There are 
probably a few other things that need to be monitored also, but you 
probably already know that.

Just my $0.02,
  -Steve W.

On 17 Sep 2002 at 16:37, Tyler Mitchell wrote:

> 
> I too would be interested in discussing benchmarks - obviously, we
> would first have to develop the criteria that goes into a benchmark. 
> What kinds of processes or functions would need to be done? Off the
> top of my head: -time (t) from request to image output  (has anyone
> already created php scripts, etc. to benchmark mapserver like this -
> I'd bet someone has!) -average t / # of requests / min -what else...?
> 
> 
> |---------+--------------------------------------->
> |         |           Dave McIlhagga              |
> |         |           <mcilhagga at dmsolutions.ca>  |
> |         |           Sent by:                    |
> |         |           owner-mapserver-users at lists.|
> |         |           gis.umn.edu                 |
> |         |                                   |
> |         |                                   |
> |         |           09/16/2002 04:11 PM         |
> |         |                                   |
> |---------+--------------------------------------->
>   >-------------------------------------------------------------------
>   --------------------------------------------------------------------
>   -----------| |                                   | |       To:      
>   john1zastrow at yahoo.com                                   | |      
>   cc:       Mark Phillips <mbp at geomtech.com>,
>   mapserver-users at lists.gis.umn.edu                                  
>   | |       Fax to:                                   | |      
>   Subject:  Re: [mapserver-users] professional MapServer services
>   list? /  ArcIMS-MapServer choice                                   |
>   >-------------------------------------------------------------------
>   --------------------------------------------------------------------
>   -----------|
> 
> 
> 
> 
> John,
> 
> In fact, there are numerous examples that if anything may indicate the
> contrary.  Ourselves, we have been involved with a project that can
> comfortably generate over 100,000 map draws a day.  There was also an
> example a couple of years ago covering the Canadian Federal election
> where a MapServer based application was the only one able to handle
> the enormous traffic that resulted.
> 
> I believe there are a few other groups using MapServer on this list
> who have also indicated that they have been using it for high volumes.
> 
> I'm not sure if anyone has done a thorough comparison of ArcIMS with
> Oracle vs. MapServer with either tuned flat files, connection to
> Oracle Spatial, SDE or PostGIS, but I suspect that MapServer would
> come out on top hands down.
> 
> Now that this has been brought up - has anyone done any kind of
> benchmarking like this?  I'd love to have that at my disposal in the
> ongoing argument to potential clients for using MapServer.
> 
> Dave
> 
> 
> 
> John Zastrow wrote:
> > Mark and List,
> >
> > I'm a similar boat, except that it's my own group that I'd be
> > 'selling' Mapserver to. It's my impression that for small to
> > moderate sized
> projects,
> > mapserver does just fine. But moderate to semi-large projects (size
> > being
> user
> > load and data load) one needs to use a commercial map server (ArcIMS
> > on Oracle).
> >
> > Has anyone done a feature matrix of the various mapsevers out there?
> > I
> just
> > started lurking on this list. Standard apologies if this has been
> > covered already. I'd be interested to see Mark's compilation and
> > information from
> folks
> > who have comparative information.
> >
> >
> > --- Mark Phillips <mbp at geomtech.com> wrote:
> >
> >>Hi List,
> >>
> >>I'm interested in compiling a list of companies and individuals who
> >>provide MapServer support and development services.  If you are such
> >>a company or individual, could you drop me a note?
> >
> >
> > __________________________________________________
> > Do you Yahoo!?
> > Yahoo! News - Today's headlines
> > http://news.yahoo.com
> >
> 
> 
> --
> Dave McIlhagga
> President, DM Solutions Group
> 
> http://www.dmsolutions.ca
> EMail : mcilhagga at dmsolutions.ca
> Phone : 613-565-5056 x15
> Fax : 613-565-0925
> 
> 
> 
> 
> 
> 
> 
> 