[Benchmarking] Test System Specs

[Andrea Aime]

Daniel Morissette ha scritto:
> Daniel Morissette wrote:
>> FYI I've picked up tonight the server that I had mentioned in a 
>> previous email. Unless it really doesn't want to cooperate then we 
>> should have it online in the next few days. I'd suggest we try that 
>> avenue and if it doesn't work out then we can always look at renting.
>>
> 
> Talking of this server, I suggest we set it up with Ubuntu Hardy 
> (8.04/LTS) 64 bits unless someone has objections. There are two 300GB 
> disks that will be setup in software RAID-1.
> 
> Then we have a choice to make: we can either run everything in the same 
> host, or use OpenVZ to setup two virtual environments (VEs) on the 
> server, one for MapServer and one for GeoServer ... more VEs could 
> easily be created on the machine if MapGuide or other groups join the 
> game later on.
> 
> We have been using OpenVZ for a while and it's really great if you want 
> to avoid possible conflicts between hosted environments. Think of it as 
> a chroot++ where you have root access to a full OS instead of just a 
> user account. I don't know if we absolutely need the VEs in this 
> specific case, but OpenVZ is very light weight and should not impact 
> performance unless two groups run benchmarks at the same time. Well, we 
> may need to bump up the default limits on resources available to each VE 
> since we want to push the limits of the system but that's about it. We 
> could even shutdown all the VEs and keep only one running when we run 
> the final/official benchmarks if anyone has concerns. The benefit is 
> that each VE would be a standalone environment (a complete OS) where 
> each group can install software at will and there is no risk that one 
> group installs stuff that interferes with the other group's software.
> 
> OTOH, if we prefer to have everything (MapServer+GeoServer+...) in the 
> same host/environment then I have no problem with that... we'll just 
> have to be careful with what we install to not interfere with the other 
> group's setup.
> 
> So I'm tempted to go with an OpenVZ setup with one VE for MapServer and 
> one for GeoServer... if anyone has a better idea then please speak up ASAP.

I asked around and a colleague, he's suggesting that techonology behind 
OpenVZ (in particular, its scheduler) has been designed to work in 
heavily multiprocess envirometns and that it might visibly slow down
the single process, heavily multithreaded nature of a Java server.

He also pointed me at this comparison:
http://www.linuxsymposium.org/2008/ols-2008-Proceedings-V1.pdf
(page 63)

His suggestion was to use Xen instead. Makes some sense, the benchmarks
are usually quite CPU bound, and Xen seems to be doing well in those
cases. He also suggested that Xen is leaving the OS quite close
to unaltered, so it should affect the two servers in a more fair
way.

Anyways, I'm no expert in virtualization technology, but I guess
a way to double check would be to run the same load test against
the native OS, as is, and then again in the VM. This would tell us
if the virtualization enviroment is favouring one server or not.
A slowdown is to be expected, the key is that the slowdown
should be the same for all the web gis servers being compared.

Cheers
Andrea