[Benchmarking] Last minute vote: shall we switch to 6600requestsrun?

[Andrea Aime]

Anne-Sophie Collignon ha scritto:
> Hi all,
> 
> Sorry for late answer, I've been busy the all day.
> 
>>From what I see regarding the vote, we don't have the feedback from all
> teams. So it means we don't have a consensus.
> 
> I've used the new jmeter tests today, and I was quite surprised that the
> numbers were lower than the previous Jmeter tests...
> Our developers are unable to explain this behavior by the fact that the
> requests are different in all 3 runs. So, maybe is there something else?
> Has someone else noticed the same?
> 
> In any case, I think it's late to check and change again the Jmeter test
> (unless we could we postpone the foss4g conference ;-) (I'm joking))
> 
> What do you think about the following alternative ?
> 
> The idea is to publish the results of the 3 runs in the final
> presentation, and to provide comments on the possible differences that
> could be observed on the 3 runs results.

Seems sensible, we definitely have to explain what goes on when a test
flips from disk bound to non disk bound.

Maybe we have time to present the three runs for all the servers, maybe
not, we'll see when making the slides. But with 1.5 hours we should be
set.
At the very least we should try to wipe out the file system caches
at the beginning of the full run, otherwise it's difficult to explain
why some servers get fast right away (because maybe the shapefiles were
already in cache) and others don't.
On Linux there is a command to drop the caches, not sure we have
an equivalent for Windows.

As far as I can see only CadCorp, Erdas and Constellation have tried
something for the best effort scenario, while everybody else just
setup the baseline (well, GS might try to setup a best effort case
for raster data, we'll see). Which means we can dedicate a bit more
effort to the baseline part of the presentation :-p

Cheers
Andrea

-- 
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.