[Benchmarking] First pair of load scripts and some considerations

[Andrea Aime]

Hi,
I've been playing a bit with sample JMeter load tests using
using Frank's python script to generate random bboxes to be
painted.

All the files I'll be referencing to are contained in this zip:
http://sigma.openplans.org/~aaime/jmeter_scripts_v1.zip
(it was too big to be attached to a mail to the ml, the post
  limit is 40kb)

I've fixed a bug in the script (it used to generate
bboxes whose upper corner was outside the requested bounds)
and modified the output a little so that it looks
width;height;x1,y1,x2,y2
This makes it easier to extract the bbox as a single param.

I've then used the python script and some existing JMeter
scripts to put togheter a couple of sample load scripts.

The first, bluemarble_ecw.jmx can be used to load a server
exposing the bluemarble raster dataset, the second,
tiger_texas.jmx, can be used to test the tiger roads
layer.

The .csv files driving the requests have been obtained running
the fixed script with params:
wms_request.py -count 2000 -region -180 -90 180 90 -minres 0.0042
-maxres 0.5 > bluemarble.csv
wms_request.py -count 2000 -region -102 30 -94 34 -minres 0.00005
-maxres 0.001 > texas.csv

(the script is generating random output, so if you run them again
   you won't get the same output).

To run the scripts you need to download JMeter (get the latest
version, 2.3.4), which in turn requires
a Java runtime. I'll be installing it on the test box soon
(that does not require to have its OS reinstalled, right?)
but in the meantime you can also run them on your machine.

JMeter runs as a GUI, and in that case you can use
Run -> Start to start the tests, Run -> clear to cleanup
the results, and look into "Summary Report" in the tree
to see it working.

JMeter can also be run headless, but things become more
complicated here.
To run it headless, one has to execute:
jmeter -p jmeter.properties -n -t bluemarble_ecw.jmx -l
bluemarble_samples.csv

and that unfortunately results just in the log of the requests
done, with timing and response.
That file can then be loaded into the JMeter GUI to get a nice
looking summary. To do that:
- generate the .csv file with the above command
- scp to your host if needed
- open SummaryLoader.jmx via the GUI, go into the
    "Summary Report" item, browse for the file, done

Or, we can make up a script that does the summary
for us and outputs it as a text table directly on the
server, without any GUI.

To get more information about the csv file contents, see here:
http://jakarta.apache.org/jmeter/usermanual/listeners.html
Some idea on how to make a summarizer script are here:
http://wiki.apache.org/jakarta-jmeter/LogAnalysis

I hope the above is not too confusing. Anyways, anyone
working on running the scripts should get some familiarity with
JMeter (unless we decide to use something else as the runner,
which is doable, provided you find a runner that can be
driven by a csv file so that we never make the same request
twice in the run to avoid possible in memory caching in the
servers tested).

Cheers
Andrea


-- 
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.