[Benchmarking] Thoughts on 2010 exercise

[Schlagel, Joel D Mr CIV USA USACE]

An important  question for 2010 is  VM's vs shared OS instance.    For 2009 we tested 2 mapping engines on one OS assuring most differences were related to the applications.  Adding a third  we started to hit version dependencies etc.  If we go to a VM approach - we can easily and more securely accommodate more systems, but then you would be testing the "whole package" application+operating system+tuning of the OS, web server, etc.   Using this approach you would be able to discover for example that the MapGuide team running on XP with IIS was able to make the system perform faster than MapServer on Solaris with Apache for a given set of hardware - but you wouldn't really be able to say why.

Also, while head to head competition on the same hardware makes for great drama .. it might also be interesting to benchmark different hardware combinations looking for most cost effective infrastructure.   Understanding payback of disk speed vs ram vs cpu speed vs processor type would be useful information


-joel





On Nov 25, 2009, at 1:09 PM, Jeff McKenna wrote:

> Also, I wonder if we should set a timeline for preparations for the next exercise.  How about January for contacting potential mapping server teams?  I'd like us to track our contacts this time (list each mapping server, the person who will be contacted, who is doing the contacting, and status).
> 
> Can we start now by brainstorming what mapping servers should be contacted?
> 
> MapServer
> GeoServer
> ESRI ArcGIS Server
> Erdas Apollo
> MapGuide OS
> ...
> 
> 
> Or...am I too early on my thinking?  I just want to give these corporate teams the heads up they need to participate.
> 
> -jeff
> 
> 
> 
> 
> _______________________________________________
> Benchmarking mailing list
> Benchmarking at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/benchmarking