[Benchmarking] First pair of load scripts and some considerations

Tue Sep 1 19:29:27 EDT 2009

HI Andrea,
I have been playing around with your jmeter script with csv file. Tested
with some WMS engines (GeoServer and a commercial) and it works well.

It will be a good idea to test with fixed tiles apart from random; I would
suggest;

Fixed tile size, Random BBox, Bluemarble Image (better to fix the bbox limit
to -178, 178, -88, 88)

128x128 - smart phone clients
256x256 - typical tile size (if requested as tiles)
800 x 600 - standard map window for a 1024x768 screen

Can you generate the three CSV files and post it over here? (it will be nice
to have 5000 samples)

Thanks
Ganesh

2009/8/25 Andrea Aime <aaime at opengeo.org>

> Hi,
> I've been playing a bit with sample JMeter load tests using
> using Frank's python script to generate random bboxes to be
> painted.
>
> First off, I've fixed a bug in the script (it used to generate
> bboxes whose upper corner was outside the requested bounds)
> and modified the output a little so that it looks
> width;height;x1,y1,x2,y2
> This makes it easier to extract the bbox as a single param.
> Modified script attached.
>
>
> I've then used the python script and some existing JMeter
> scripts to put togheter a couple of sample load scripts.
>
> The first, bluemarble_ecw.jmx can be used to load a server
> exposing the bluemarble raster dataset, the second,
> tiger_texas.jmx, can be used to test the tiger roads
> layer.
>
> The .csv files driving the requests have been obtained running
> the fixed script with params:
> wms_request.py -count 2000 -region -180 -90 180 90 -minres 0.0042 -maxres
> 0.5 > bluemarble.csv
> wms_request.py -count 2000 -region -102 30 -94 34 -minres 0.00005 -maxres
> 0.001 > texas.csv
>
> (the script is generating random output, so if you run them again
>  you won't get the same output).
>
> To run the scripts you need to download JMeter (get the latest
> version, 2.3.4), which in turn requires
> a Java runtime. I'll be installing it on the test box soon
> (that does not require to have its OS reinstalled, right?)
> but in the meantime you can also run them on your machine.
>
> JMeter runs as a GUI, and in that case you can use
> Run -> Start to start the tests, Run -> clear to cleanup
> the results, and look into "Summary Report" in the tree
> to see it working.
>
> JMeter can also be run headless, but things become more
> complicated here.
> To run it headless, one has to execute:
> jmeter -p jmeter.properties -n -t bluemarble_ecw.jmx -l
> bluemarble_samples.csv
>
> and that unfortunately results just in the log of the requests
> done, with timing and response.
> That file can then be loaded into the JMeter GUI to get a nice
> looking summary. To do that:
> - generate the .csv file with the above command
> - scp to your host if needed
> - open SummaryLoader.jmx via the GUI, go into the
>  "Summary Report" item, browse for the file, done
>
> Or, we can make up a script that does the summary
> for us and outputs it as a text table directly on the
> server, without any GUI.
>
> To get more information about the csv file contents, see here:
> http://jakarta.apache.org/jmeter/usermanual/listeners.html
> Some idea on how to make a summarizer script are here:
> http://wiki.apache.org/jakarta-jmeter/LogAnalysis
>
> I hope the above is not too confusing. Anyways, anyone
> working on running the scripts should get some familiarity with
> JMeter (unless we decide to use something else as the runner,
> which is doable, provided you find a runner that can be
> driven by a csv file so that we never make the same request
> twice in the run to avoid possible in memory caching in the
> servers tested).
>
> Cheers
> Andrea
>
>
> #---------------------------------------------------------------------------
> # Results file configuration
>
> #---------------------------------------------------------------------------
>
> # This section helps determine how result data will be saved.
> # The commented out values are the defaults.
>
> # legitimate values: xml, csv, db.  Only xml and csv are currently
> supported.
> jmeter.save.saveservice.output_format=csv
>
>
> # true when field should be saved; false otherwise
>
> # assertion_results_failure_message only affects CSV output
> #jmeter.save.saveservice.assertion_results_failure_message=false
> #
> #jmeter.save.saveservice.data_type=true
> #jmeter.save.saveservice.label=true
> #jmeter.save.saveservice.response_code=true
> # response_data is not currently supported for CSV output
> #jmeter.save.saveservice.response_data=false
> # Save ResponseData for failed samples
> #jmeter.save.saveservice.response_data.on_error=false
> #jmeter.save.saveservice.response_message=true
> #jmeter.save.saveservice.successful=true
> #jmeter.save.saveservice.thread_name=true
> #jmeter.save.saveservice.time=true
> #jmeter.save.saveservice.subresults=true
> #jmeter.save.saveservice.assertions=true
> #jmeter.save.saveservice.latency=true
> #jmeter.save.saveservice.samplerData=false
> #jmeter.save.saveservice.responseHeaders=false
> #jmeter.save.saveservice.requestHeaders=false
> #jmeter.save.saveservice.encoding=false
> #jmeter.save.saveservice.bytes=true
> jmeter.save.saveservice.url=true
> #jmeter.save.saveservice.filename=false
> #jmeter.save.saveservice.hostname=false
> #jmeter.save.saveservice.thread_counts=false
> #jmeter.save.saveservice.sample_count=false
>
> # Timestamp format
> # legitimate values: none, ms, or a format suitable for SimpleDateFormat
> #jmeter.save.saveservice.timestamp_format=ms
> #jmeter.save.saveservice.timestamp_format=MM/dd/yy HH:mm:ss
>
> # Put the start time stamp in logs instead of the end
> sampleresult.timestamp.start=true
>
>
> # legitimate values: none, first, all
> #jmeter.save.saveservice.assertion_results=none
>
> # For use with Comma-separated value (CSV) files or other formats
> # where the fields' values are separated by specified delimiters.
> # Default:
> #jmeter.save.saveservice.default_delimiter=,
> # For TAB, since JMeter 2.3 one can use:
> #jmeter.save.saveservice.default_delimiter=\t
>
> #jmeter.save.saveservice.print_field_names=false
>
> # Optional list of JMeter variable names whose values are to be saved in
> the result data files.
> # Use commas to separate the names. For example:
> #sample_variables=SESSION_ID,REFERENCE
> # N.B. The current implementation saves the values in XML as attributes,
> # so the names must be valid XML names.
> # Versions of JMeter after 2.3.2 send the variable to all servers
> # to ensure that the correct data is available at the client.
>
> # Optional xml processing instruction for line 2 of the file:
> #jmeter.save.saveservice.xml_pi=<?xml-stylesheet type="text/xsl"
> href="sample.xsl"?>
>
>
> _______________________________________________
> Benchmarking mailing list
> Benchmarking at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/benchmarking
>
>

-- 
Shanmugam Ganeshkumar
GSM: +65-81808003
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/benchmarking/attachments/20090902/b4410d6b/attachment.html