HI Andrea,<div><br></div><div>I have been playing around with your jmeter script with csv file. Tested with some WMS engines (GeoServer and a commercial) and it works well.</div><div><br></div><div>It will be a good idea to test with fixed tiles apart from random; I would suggest;</div>
<div><br></div><div>Fixed tile size, Random BBox, Bluemarble Image (better to fix the bbox limit to -178, 178, -88, 88)</div><div><br></div><div>128x128 - smart phone clients</div><div>256x256 - typical tile size (if requested as tiles)</div>
<div>800 x 600 - standard map window for a 1024x768 screen </div><div><br></div><div>Can you generate the three CSV files and post it over here? (it will be nice to have 5000 samples)</div><div><br></div><div><br></div><div>
Thanks</div><div>Ganesh</div><div><br></div><div><br></div><div><br></div><div><br><br><div class="gmail_quote">2009/8/25 Andrea Aime <span dir="ltr"><<a href="mailto:aaime@opengeo.org">aaime@opengeo.org</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im">Hi,<br>
I've been playing a bit with sample JMeter load tests using<br>
using Frank's python script to generate random bboxes to be<br>
painted.<br>
<br></div>
First off, I've fixed a bug in the script (it used to generate<div class="im"><br>
bboxes whose upper corner was outside the requested bounds)<br>
and modified the output a little so that it looks<br>
width;height;x1,y1,x2,y2<br>
This makes it easier to extract the bbox as a single param.<br></div>
Modified script attached.<div><div></div><div class="h5"><br>
<br>
I've then used the python script and some existing JMeter<br>
scripts to put togheter a couple of sample load scripts.<br>
<br>
The first, bluemarble_ecw.jmx can be used to load a server<br>
exposing the bluemarble raster dataset, the second,<br>
tiger_texas.jmx, can be used to test the tiger roads<br>
layer.<br>
<br>
The .csv files driving the requests have been obtained running<br>
the fixed script with params:<br>
wms_request.py -count 2000 -region -180 -90 180 90 -minres 0.0042 -maxres 0.5 > bluemarble.csv<br>
wms_request.py -count 2000 -region -102 30 -94 34 -minres 0.00005 -maxres 0.001 > texas.csv<br>
<br>
(the script is generating random output, so if you run them again<br>
you won't get the same output).<br>
<br>
To run the scripts you need to download JMeter (get the latest<br>
version, 2.3.4), which in turn requires<br>
a Java runtime. I'll be installing it on the test box soon<br>
(that does not require to have its OS reinstalled, right?)<br>
but in the meantime you can also run them on your machine.<br>
<br>
JMeter runs as a GUI, and in that case you can use<br>
Run -> Start to start the tests, Run -> clear to cleanup<br>
the results, and look into "Summary Report" in the tree<br>
to see it working.<br>
<br>
JMeter can also be run headless, but things become more<br>
complicated here.<br>
To run it headless, one has to execute:<br>
jmeter -p jmeter.properties -n -t bluemarble_ecw.jmx -l bluemarble_samples.csv<br>
<br>
and that unfortunately results just in the log of the requests<br>
done, with timing and response.<br>
That file can then be loaded into the JMeter GUI to get a nice<br>
looking summary. To do that:<br>
- generate the .csv file with the above command<br>
- scp to your host if needed<br>
- open SummaryLoader.jmx via the GUI, go into the<br>
"Summary Report" item, browse for the file, done<br>
<br>
Or, we can make up a script that does the summary<br>
for us and outputs it as a text table directly on the<br>
server, without any GUI.<br>
<br>
To get more information about the csv file contents, see here:<br>
<a href="http://jakarta.apache.org/jmeter/usermanual/listeners.html" target="_blank">http://jakarta.apache.org/jmeter/usermanual/listeners.html</a><br>
Some idea on how to make a summarizer script are here:<br>
<a href="http://wiki.apache.org/jakarta-jmeter/LogAnalysis" target="_blank">http://wiki.apache.org/jakarta-jmeter/LogAnalysis</a><br>
<br>
I hope the above is not too confusing. Anyways, anyone<br>
working on running the scripts should get some familiarity with<br>
JMeter (unless we decide to use something else as the runner,<br>
which is doable, provided you find a runner that can be<br>
driven by a csv file so that we never make the same request<br>
twice in the run to avoid possible in memory caching in the<br>
servers tested).<br>
<br>
Cheers<br>
Andrea<br>
</div></div><br>#---------------------------------------------------------------------------<br>
# Results file configuration<br>
#---------------------------------------------------------------------------<br>
<br>
# This section helps determine how result data will be saved.<br>
# The commented out values are the defaults.<br>
<br>
# legitimate values: xml, csv, db. Only xml and csv are currently supported.<br>
jmeter.save.saveservice.output_format=csv<br>
<br>
<br>
# true when field should be saved; false otherwise<br>
<br>
# assertion_results_failure_message only affects CSV output<br>
#jmeter.save.saveservice.assertion_results_failure_message=false<br>
#<br>
#jmeter.save.saveservice.data_type=true<br>
#jmeter.save.saveservice.label=true<br>
#jmeter.save.saveservice.response_code=true<br>
# response_data is not currently supported for CSV output<br>
#jmeter.save.saveservice.response_data=false<br>
# Save ResponseData for failed samples<br>
#jmeter.save.saveservice.response_data.on_error=false<br>
#jmeter.save.saveservice.response_message=true<br>
#jmeter.save.saveservice.successful=true<br>
#jmeter.save.saveservice.thread_name=true<br>
#jmeter.save.saveservice.time=true<br>
#jmeter.save.saveservice.subresults=true<br>
#jmeter.save.saveservice.assertions=true<br>
#jmeter.save.saveservice.latency=true<br>
#jmeter.save.saveservice.samplerData=false<br>
#jmeter.save.saveservice.responseHeaders=false<br>
#jmeter.save.saveservice.requestHeaders=false<br>
#jmeter.save.saveservice.encoding=false<br>
#jmeter.save.saveservice.bytes=true<br>
jmeter.save.saveservice.url=true<br>
#jmeter.save.saveservice.filename=false<br>
#jmeter.save.saveservice.hostname=false<br>
#jmeter.save.saveservice.thread_counts=false<br>
#jmeter.save.saveservice.sample_count=false<br>
<br>
# Timestamp format<br>
# legitimate values: none, ms, or a format suitable for SimpleDateFormat<br>
#jmeter.save.saveservice.timestamp_format=ms<br>
#jmeter.save.saveservice.timestamp_format=MM/dd/yy HH:mm:ss<br>
<br>
# Put the start time stamp in logs instead of the end<br>
sampleresult.timestamp.start=true<br>
<br>
<br>
# legitimate values: none, first, all<br>
#jmeter.save.saveservice.assertion_results=none<br>
<br>
# For use with Comma-separated value (CSV) files or other formats<br>
# where the fields' values are separated by specified delimiters.<br>
# Default:<br>
#jmeter.save.saveservice.default_delimiter=,<br>
# For TAB, since JMeter 2.3 one can use:<br>
#jmeter.save.saveservice.default_delimiter=\t<br>
<br>
#jmeter.save.saveservice.print_field_names=false<br>
<br>
# Optional list of JMeter variable names whose values are to be saved in the result data files.<br>
# Use commas to separate the names. For example:<br>
#sample_variables=SESSION_ID,REFERENCE<br>
# N.B. The current implementation saves the values in XML as attributes,<br>
# so the names must be valid XML names.<br>
# Versions of JMeter after 2.3.2 send the variable to all servers<br>
# to ensure that the correct data is available at the client.<br>
<br>
# Optional xml processing instruction for line 2 of the file:<br>
#jmeter.save.saveservice.xml_pi=<?xml-stylesheet type="text/xsl" href="sample.xsl"?><br>
<br>
<br>_______________________________________________<br>
Benchmarking mailing list<br>
<a href="mailto:Benchmarking@lists.osgeo.org">Benchmarking@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/benchmarking" target="_blank">http://lists.osgeo.org/mailman/listinfo/benchmarking</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Shanmugam Ganeshkumar<br>GSM: +65-81808003<br>
</div>