[Benchmarking] Last minute vote: shall we switchto6600requestsrun?

[Anne-Sophie Collignon]

Hi Adrian,

Sorry for late answer, but maybe should we all logs all the runs (1,2,3...) and then based on the meeting we'll have Tuesday evening, we'll decide all together what we publish (only run3 or also run1,run2,run3...)

Also, as I understood, and because of the lack of time to change the test plan, we keep then the "initial" csv scripts, not the latest one provided by Andrea two days ago...

Thank you,

Anne-Sophie




-- 
Anne-Sophie Collignon
Enterprise Solutions - Project Engineer

ERDAS, SA


-----Original Message-----
From: benchmarking-bounces at lists.osgeo.org [mailto:benchmarking-bounces at lists.osgeo.org] On Behalf Of Adrian Custer
Sent: vendredi 3 septembre 2010 11:20
To: Performance testing of OSGeo and other web service engines.
Subject: RE: [Benchmarking] Last minute vote: shall we switchto6600requestsrun?

On Fri, 2010-09-03 at 11:06 +0200, Anne-Sophie Collignon wrote:
> Hi Adrian,
> 
> I meant : publish the 3 runs of tests (csv with 2000 request, 1,2,4,8,16,32,64 threads), so not the last scripts provided by Andrea two days ago with the 3 runs is one scripts.
> We keep the "initial" one, but publish the 3 runs (not only the run3, or best one...) is that make sense?
> 
> Thank you,
> 
> Anne-Sophie

Thanks for the clarification.


In response to this proposal, I think it goes against the whole purpose
of the triple run. As I understand it, the goal of the first two runs is
to get the server into a known state and warm it up for the third run.
The third run is then performed to measure the performance. The two
warmup runs do things like load file blocks into memory and allow
run-time compilation to occur.

If the first run were to be published, then we would have to worry about
what folk have been doing *before* the first run. That would be a
reasonable alternative strategy from performing the two warmup runs we
currently use but would need discussion and agreement---it seems more
practical to stick to the two warmup runs.


cheers,
  --adrian
 

_______________________________________________
Benchmarking mailing list
Benchmarking at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/benchmarking