[Mapserver-dev] Performance loss between 3.5 and 4.2

[Ed]

Sean -

While I don't object to the goal, the chief difficulty is that MapServer covers such a very wide range of capabilities, with a very wide range of non-MapServer supporting libraries, that it's very difficult (IMHO) to create a test that is either comprehensive or useful.  It may be useful to have test suites to look for (a) validation of alleged performance-improving fixes or (b) changes that inadvertently cause a serious performance problem.  But the value of those suites is limited to keeping developers on their toes, not helping end users understand whether they should expect MapServer X to perform differently than MapServer Y for THEIR application - which is usually the only one they care about <g>.

	- Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
ed at topozone.com
(978) 251-4242  

-----Original Message-----
From: mapserver-dev-admin at lists.gis.umn.edu [mailto:mapserver-dev-admin at lists.gis.umn.edu] On Behalf Of Sean Gillies
Sent: Friday, September 24, 2004 2:21 PM
To: Daniel Morissette
Cc: mapserver-dev at lists.gis.umn.edu
Subject: Re: [Mapserver-dev] Performance loss between 3.5 and 4.2

On Sep 24, 2004, at 11:10 AM, Daniel Morissette wrote:

> Sean Gillies wrote:
>> Until the mapserver project adopts some regularly run performance 
>> benchmarks, all evidence that mapserver is slower or faster than it 
>> used to be will be only anecdotal.  Anyone interested?
>
> I think we'd be interested, but what do you have in mind exactly?
>

Nothing concrete, just ideas ...

For starters, could be as simple as running shp2img every night for each CVS branch using the same test data (or as identical as possible).  One platform would be enough to start.  Write output to a file that is published through the web.  Maybe have a chart of performance versus calendar time with a line for each branch.  So when somebody sees a drastic dip in performance over a few nights, they can look in CVS to see what's been committed -- and see that I've been working on brush caching in mapgd.c (for example).  I don't think we'd necessarily even need to use a nightly profiler, just time the process as accurately and consistently as we can.

That takes care of the future performance history.  For the past, we could travel back in time with CVS snapshots :).

Sean

_______________________________________________
Mapserver-dev mailing list
Mapserver-dev at lists.gis.umn.edu
http://lists.gis.umn.edu/mailman/listinfo/mapserver-dev