[Wps-discuss] WPS Benchmarking Session at FOSS4G-Europe

[Fenoy Gerald]

Dear participants,
I would like to keep you updated with the ongoing work made for the WPS Benchmarking which will happen for the first time in Bremen next week during the FOSS4G-Europe which will happen next week.

First of all, I would like to invite you to look at the CPTesting testing suite shared on github [1] and then try it on your Server setup made on the shared infrastructure (so from your OSGeoLive dedicated environment - see here [2] for reference).

To make the long story short about the testing scripts, let say that the aggregate.sh script shared on github is responsible for generating the slides from results produced during the tests (that each team should run by using the run.sh script). 

For the tests no need for explanations, you know them already in details as they were described in the previous mail [4] and they are summarized in the slides too [3].

Each team then can have his own copy of the CPTesting directory on its OSGeoLive environment dedicated its project, then run the tests by using the following commands:

./run.sh ServerURL 'Buffer Union Intersects’ "GetCapabilities" 5000 100 > ../tmp/WPS_BenchmarkingGC.html
./run.sh ServerURL 'Buffer Union Intersects' "DescribeProcess" 5000 100  > ../tmp/WPS_BenchmarkingDP.html
./run.sh ServerURL 'Buffer Union Intersects’ "ExecuteSync" 500 10  > ../tmp/WPS_BenchmarkingES.html
./run.sh ServerURL 'Buffer Union Intersects' "ExecuteAsync" 500 10  > ../tmp/WPS_BenchmarkingEA.html

I have currently run test scripts over available servers: 52° North WPS and ZOO-Project and published the results I was able to get here [3].

A quick look at the results in slides, you may noticed that some issue occurred during testing, I do take in account conformance results here but more practical use of testing scripts.

First issue was about GetCapabilities POST request sent to 52° North Server which was strangely failing.
 
Second issue was the DescribeProcess test number 1 which is using Apache Benchmark to send 5000 request with 100 concurrents to the server. When trying this over the 52° North WPS server I faced issues (the server was not answering anymore for some time) and I was forced to comment this from the DescribeProcess tests. Nevertheless, at it was working for ZOO-Project I kept it traced in the result.

Third issue was about the Execute request tests, I was not yet able to have correct answer to POST requests from 52° North WPS server setup. But I’m pretty sure that fixing requests templates may help to solve this and have results for this tests.

For being able to go further, let say to test also other WPS Server implementations, I would require at least to have the WPS Endpoint filled by each team. 
In such a case, I can update the slides by adding the generated one by running tests.
In other case, you can still run the tests on your way from your OSGeoLive Environment, then inform me where the results were produced. This way, slides can be easily updated. If you prefer to use this way, then please make sure you provide results of tests run on your OSGeoLive environment and not any other server.

I wish that you will provide comments about the current slides and make your own modifications / addition to the slides through github.

Hope to hear back from you,
Best regards,


[1] https://github.com/WPS-Benchmarking/cptesting
[2] http://wiki.osgeo.org/wiki/WPS_benchmark_2014
[3] http://37.187.89.64:2007/testing/OSGeoLive-presentation-master/WPS_BenchmarkAF.html
[4] http://lists.osgeo.org/pipermail/wps-discuss/2014-June/000040.html


Gérald Fenoy
gerald.fenoy at geolabs.fr

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 204 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osgeo.org/pipermail/wps-discuss/attachments/20140710/b3d50c8e/attachment.pgp>