[Benchmarking] Last minute vote: shall we switch to 6600requestsrun?

[Anne-Sophie Collignon]

Hi all,

Sorry for late answer, I've been busy the all day.

>From what I see regarding the vote, we don't have the feedback from all
teams. So it means we don't have a consensus.

I've used the new jmeter tests today, and I was quite surprised that the
numbers were lower than the previous Jmeter tests...
Our developers are unable to explain this behavior by the fact that the
requests are different in all 3 runs. So, maybe is there something else?
Has someone else noticed the same?

In any case, I think it's late to check and change again the Jmeter test
(unless we could we postpone the foss4g conference ;-) (I'm joking))

What do you think about the following alternative ?

The idea is to publish the results of the 3 runs in the final
presentation, and to provide comments on the possible differences that
could be observed on the 3 runs results.

Thank you,
Kind regards,

Anne-Sophie


-- 
Anne-Sophie Collignon
Enterprise Solutions - Project Engineer

ERDAS, SA



-----Original Message-----
From: benchmarking-bounces at lists.osgeo.org
[mailto:benchmarking-bounces at lists.osgeo.org] On Behalf Of Andrea Aime
Sent: jeudi 2 septembre 2010 9:21
To: Performance testing of OSGeo and other web service engines.
Subject: Re: [Benchmarking] Last minute vote: shall we switch to
6600requestsrun?

Smith, Michael ERDC-CRREL-NH ha scritto:
> I vote + 0 for the same reason as Andrea. 

Well, finally had the occasion to make a run and I don't
believe we can actually present the results as is.
Not the default for all test runs at least.

What I get at the third set of tests is that regardless
of the number of threads the throughput is more or less
constant around 4 r/s (on my machine, which has
less available memory).

Now imagine that on a graph. Completely flat chat.
Repeat that for all the tests, for all the servers,
all telling us the same thing: you have just one slow
disk baby, the 8 CPUs you have are worthless.
And then hope the audience is not armed with rotten eggs :-)

If we really want to, we could maybe make one run of these,
say the vector 4326 one, to show how the various servers
behave under disk boundness, but we cannot really extend
that to all the runs.

Cheers
Andrea

-- 
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
_______________________________________________
Benchmarking mailing list
Benchmarking at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/benchmarking