[Benchmarking] JMeter problems on 1cpu benchmarking server.

[Frank Warmerdam]

Andrea Aime wrote:
> Now, the missing bits, imho, is to hack that script so that one does
> not really have to specify the property file and the summarizer, which
> is still to be uploaded, gets called automatically.
> So that by just calling:
> 
> jmeter myscript.jmx
> 
> one gets out the summary results and the full requests log in 
> myscript.jtl or, proabbly simpler, myscript.jmx.jtl

Andrea,

Success!

I have modified /usr/local/bin/jmeter to set the properties file and the
headless mode attribute.   So now I just invoke it like:


[fwarmerdam at localhost msmaps]$ jmeter -n -t bigtiff.xml -l bigtiff.jtl
Created the tree successfully using bigtiff.xml
Starting the test @ Wed Sep 16 14:42:55 EDT 2009 (1253126575221)
Waiting for possible shutdown message on port 4445
Generate Summary Results +   870 in 124.3s =    7.0/s Avg:  1658 Min:    77 
Max: 15996 Err:     0 (0.00%)
Generate Summary Results +   630 in  83.4s =    7.6/s Avg:  3483 Min:    68 
Max: 65824 Err:     0 (0.00%)
Generate Summary Results =  1500 in 190.7s =    7.9/s Avg:  2425 Min:    68 
Max: 65824 Err:     0 (0.00%)
Tidying up ...    @ Wed Sep 16 14:46:06 EDT 2009 (1253126766411)
... end of run

I'll try out the summarizer later today and perhaps update the script
with the remaining steps you suggest.

Now we are getting close to a point where we can start comparing performance!

The above was against the one big bigtiff file using plain cgi (reopening the
file each time).  My ECW with fastcgi performance was poorer:

Generate Summary Results +   452 in 173.2s =    2.6/s Avg:  3114 Min:    70 
Max: 24966 Err:     0 (0.00%)
Generate Summary Results +   607 in 194.3s =    3.1/s Avg:  8606 Min:   232 
Max: 39723 Err:     1 (0.16%)
Generate Summary Results =  1059 in 353.2s =    3.0/s Avg:  6262 Min:    70 
Max: 39723 Err:     1 (0.09%)
Generate Summary Results +   441 in 166.0s =    2.7/s Avg: 10987 Min:   231 
Max: 45339 Err:     0 (0.00%)
Generate Summary Results =  1500 in 481.1s =    3.1/s Avg:  7651 Min:    70 
Max: 45339 Err:     1 (0.07%)

though much much better than without fastcgi (which was circa 0.3 renders
per second!).

I'll try to produce some more final ecw+fastcgi and tiled tiff+fastcgi
results in the next day or so that should be suitable for comparing to
geoserver performance.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent