[Benchmarking] Last minute vote: shall we switch to6600requestsrun?

[Adrian Custer]

On Fri, 2010-09-03 at 11:06 +0200, Anne-Sophie Collignon wrote:
> Hi Adrian,
> 
> I meant : publish the 3 runs of tests (csv with 2000 request, 1,2,4,8,16,32,64 threads), so not the last scripts provided by Andrea two days ago with the 3 runs is one scripts.
> We keep the "initial" one, but publish the 3 runs (not only the run3, or best one...) is that make sense?
> 
> Thank you,
> 
> Anne-Sophie

Thanks for the clarification.


In response to this proposal, I think it goes against the whole purpose
of the triple run. As I understand it, the goal of the first two runs is
to get the server into a known state and warm it up for the third run.
The third run is then performed to measure the performance. The two
warmup runs do things like load file blocks into memory and allow
run-time compilation to occur.

If the first run were to be published, then we would have to worry about
what folk have been doing *before* the first run. That would be a
reasonable alternative strategy from performing the two warmup runs we
currently use but would need discussion and agreement---it seems more
practical to stick to the two warmup runs.


cheers,
  --adrian