[Benchmarking] Hardware Specs added to Wiki

[Dimitri Monie]

Andrea,

I am a bit surprised by the sizing considerations and especially by the money issues that are raised. I explain.
On one side, each participant is spending days of man-power to get ready for the benchmark, and they will also pay to be at the FOSS4G benchmarking day (travel, accomodation, fees,...). There will also be a marketing effort as the market is looking at the event and should not mis-interprete the results.
On the organizer's side, this benchmark gives visibility and the results of the benchmark are expected to provide them credibility.
The organizers and the participants both expect the benchmark results to be as realistic as possible, or at least as close as possible to the performance digits that customers can produce on their own machines.

If the organizers cannot afford better than a 4-years old workstation-type machine, 2GB ram (it is what I have on my personal laptop) and a 160 GB disk (1 TerraByte disk costs less than 100 Euros today), I am not sure this benchmark still makes sense, as it does not match any realistic environment the usual users are setting up in production.

I am not complaining on the machine, I am confident that our software will perform properly and with acceptable performance on it. My issue is on the relevance of the platforms and on the usability of the test results to the customers expectations.

A server-type machine (4-core) with 8 GB RAM and 1 TB of hard disk looks more suitable to me.

Dimitri Monie

Team Manager & Quality Assurance Engineer
ERDAS, SA
Quai Timmermans 14/01
4000 Liège, Belgium
Phone: +32 4 364 03 64
Cell: +32 475 206 196
Fax: +32 4 253 47 37
Dimitri.Monie at erdas.com
Web: www.erdas.com




-----Message d'origine-----
De : benchmarking-bounces at lists.osgeo.org [mailto:benchmarking-bounces at lists.osgeo.org] De la part de Andrea Aime
Envoyé : mercredi 23 juin 2010 8:45
À : Performance testing of OSGeo and other web service engines.
Objet : Re: [Benchmarking] Hardware Specs added to Wiki

Frank Warmerdam ha scritto:
> On Tue, Jun 22, 2010 at 1:35 PM, Smith, Michael D ERDC-CRREL-NH 
> <michael.smith at usace.army.mil> wrote:
>> All,
>>
>> I've added the hardware specs to the wiki for the machines I've been 
>> able to collect.
> 
> Mike,
> 
> The machines look good from  my point of view, with a good match 
> between the linux and windows system.
> I see the disk space (160GB) is modest so this will be a limiting 
> factor in our data volumes.

Indeed. I don't see us adding the OS, potentially
8 servers with all their dependencies, 20GB or so of shapefiles in native format + various other optimized formats for the best effort
trial) and 100GB of geotiffs + various optimized other formats in 160GB space total.

I guess we seriously need to scale down the amount of data we'll be using and/or give up the idea of doing "best effort" benchmarking as that is the thing that will make most teams roll out their own special formats, one per server

Either that or someone puts the money on the table for larger disks (Mike, is that even an option or the machines have to be left untouched?)

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
_______________________________________________
Benchmarking mailing list
Benchmarking at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/benchmarking