[Mapserver-dev] Performance loss between 3.5 and 4.2

[Daniel Morissette]

Ed McNierney wrote:
> Sean -
> 
> While I don't object to the goal, the chief difficulty is that MapServer covers such a very wide range of capabilities, with a very wide range of non-MapServer supporting libraries, that it's very difficult (IMHO) to create a test that is either comprehensive or useful.  It may be useful to have test suites to look for (a) validation of alleged performance-improving fixes or (b) changes that inadvertently cause a serious performance problem.  But the value of those suites is limited to keeping developers on their toes, not helping end users understand whether they should expect MapServer X to perform differently than MapServer Y for THEIR application - which is usually the only one they care about <g>.
> 

We're on the -dev list here, not on the -users list.  ;)  I agree that 
benchmarks don't mean much to users since the performance depends on too 
many factors. I think what Sean is talking about is a tool for 
developers: tests for typical scenarios, combined with timing 
measurements to automatically find out if things were broken or 
performance was made worse by a given change to the code. Over time as 
we add more tests to this suite we may end up being able to cover all 
aspects of the software.

Daniel
-- 
------------------------------------------------------------
  Daniel Morissette               dmorissette at dmsolutions.ca
  DM Solutions Group              http://www.dmsolutions.ca/
------------------------------------------------------------