[Benchmarking] Software veersions to be used

[Andrea Aime]

Hi all,

Jeff and I have been looking into the current benchmarking situation today.
The good news is that we have the benchmarking tools setup, the servers 
installed, and the data is ready on some data providers and being setup 
on the others.

The bad news is that FOSS4G is coming up fast and everybody is short on 
time, we've added Oracle tests to the mix just recently, and we still 
have to come up with an interesting presentation!

We both feel that we have to reduce the number of tests we're going to 
run in order to have some time to interpret the results and make 
improvements before the upcoming event.  Therefore, we strongly suggest 
to only test one version of the software - the best version available 
(development or otherwise) - to cut down the number of tests to be run 
and presented.

In the GeoServer case, GeoServer 2.0.0 RC2 will be used.  In the 
MapServer case, MapServer 5.6.0 (or the latest available beta) will be 
used. We're not sure what ArcGIS server version will be used, but we're 
under the impression ESRI was going to participate with just one version 
anyways (please confirm).

Cheers
Andrea and Jeff