[Benchmarking] potential caching issue ?

[Anne-Sophie Collignon]

Hi, 

I don't know why, but it seems, my first email is not coming....

I resent, sorry if it comes twice...

 

Thank you,

 

Anne-Sophie

 

From: Anne-Sophie Collignon 
Sent: mercredi 1 septembre 2010 12:34
To: 'benchmarking-bounces at lists.osgeo.org'
Subject: [Benchmarking] potential caching issue ?

 

 

Hi All,

 

We would like to raise a potential issue in the way the Jmeter tests are
designed..

We had the confirmation today that the GetMap BBOX are randomly chosen
(which is good) but once chosen, these are used in the 3 runs.

This means that one single csv is used for the 3 runs.

 

We have concern that some servers can simply cache responses for
subsequent runs, which would produce unrealistic throughput results.

 

We understand the fact that we have 3 runs, indeed the server needs to
be initialized, but server initialization and caching are from our point
of view different topics.

 

We think that caching the responses is not a realistic use-case. It
would mean that we measure the network bandwidth (caching results =
sending back bytes on the wire) rather than the real getmap performances
of the servers. 

 

At this level, we understand it may be too late to change the test (E.g:
to have the BBOX slightly different in all 3 runs, that way, the server
will be initialized but will be unable to cache the data blocks and the
results responses). 

 

What do you think can be done to avoid this scenario? 

Thank you !

Kind Regards,

 

Ps : we are grateful for the time spent by certain persons to provide
the Jmeter tests. Our goal is not to criticize their work, but to ensure
that the produced results are fair...

 

-- 

Anne-Sophie Collignon

Enterprise Solutions - Project Engineer

 

ERDAS, SA

 

 

********CONFIDENTIALITY NOTICE********

This email message, including any attachments, may contain confidential
information exclusively provided for intended recipients or authorized
representatives of the intended recipients.  Any dissemination of this
e-mail by anyone other than an intended recipient or authorized
representatives of the intended recipients is strictly prohibited. If
you are not a named recipient or authorized representatives of the
intended recipients, you are prohibited from any further viewing of the
e-mail or any attachments or from making any use of the e-mail or
attachments. If you believe you have received this e-mail in error,
notify the sender immediately and permanently delete the e-mail, any
attachments, and all copies thereof from any drives or storage media and
destroy any printouts of the e-mail or attachments.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/benchmarking/attachments/20100901/0c8c13f4/attachment.html