[Wps-discuss] WPS Benchmarking at FOSS4G 2014

[Benjamin Proß]

Hi all,

Are there any news regarding the benchmark procedure? Or are you all 
busy writing comments for the WPS 2.0 candidate standard? :)

Cheers,

Benjamin

Am 21.08.2014 13:00, schrieb Benjamin Proß:
> Hello Gérald, all,
>
> I am glad that you like the idea and I hope the other projects feel 
> the same. The tests you are suggesting make sense, though I am not 
> completely sure about the combinations with the complexdata in-/output 
> as reference/inline xml. Complexdata output could also be requested as 
> raw data output. Maybe we should create a table listing the 
> combinations. I could start this and then we could choose the ones we 
> want. We should also define the process description, so the projects 
> can create the respective processes.
>
> Cheers,
>
> Benjamin
>
> Am 21.08.2014 11:16, schrieb Fenoy Gerald:
>> Hello Benjamin,
>> I think it is a great idea to handle testings of the Execute request 
>> the way you described, by using a simple echo service.
>>
>> By now for synchronous requests, the tests look like the following:
>>
>>     • Test 1: value as JSON
>>     • Test 2: value as GML
>>     • Test 3: value as reference as GML
>>     • Test 4: same as test 3, result as reference
>>     • Test 5: value as reference as GML using POST request
>>     • Test 6: same as test 5, result as reference
>>
>> And for asynchronous request, we have the following:
>>
>>     • Test 1': value as reference as GML
>>     • Test 2': same as test 1, result as reference
>>     • Test 3': value as reference as GML using POST request
>>     • Test 4': same as test 3, result as reference
>>
>> Personally, I think that the Test 1 and Test 2 should become unique 
>> now as there is no need for parsing any value from the service itself 
>> as it will simply return the value without any treatment. So we 
>> should use only the Test 2 for our next run.
>>
>> I also suppose that we should multiply the number of tests as we will 
>> need to ask for each available outputs individually and all the 
>> possible combinaisons of them to also test this cases.
>>
>> So, I would like to propose the following updates for the tests run 
>> for both synchronous and asynchronous requests. We run each 
>> individual test 7 times: first to have only once output at a time (so 
>> 3 tests), then to test all possible combinaisons of 2 outputs names 
>> (3 tests) and finally to have all the output returned (1 test).
>>
>> I hope that you can confirm that I’m right thinking of this updates 
>> to the current tests.
>>
>> I’ve added one column to the table [1] listing all WPS team 
>> participating to write the name of the echo service when it will be 
>> available.
>>
>> Best regards,
>>
>> [1] 
>> http://wiki.osgeo.org/wiki/WPS_benchmark_2014#Available_infrastructure
>>
>> Le 19 août 2014 à 16:03, Benjamin Proß <b.pross at 52north.org> a écrit :
>>
>>> Dear all,
>>>
>>> I see that there are still not all services are deployed on the test 
>>> machine. I hope you guys are still on board.
>>> At 52°North we were thinking a bit about the execute tests.
>>> We do not yet have any results to compare but we think that 
>>> benchmarking with "real" processes that compute something could 
>>> falsify the results, as the computation in most cases is done by a 
>>> underlying library (GeoTools in our case, but this could also be 
>>> exchanged).
>>> So the results would be influenced by how fast the geo-computation 
>>> lib would do its job. Not sure that we want that?!
>>> We also do not test whether the process really did its job (e.g. 
>>> buffer the input geometry). So do we need to use "real" processes?
>>> We would propose to use an echo process that could have three 
>>> inputs, Literal-, Complex- and BBoxData. The process then simply 
>>> returns what it gets (input/output types would have to match).
>>> We would probably also need a asynchronous version that has a delay 
>>> or something.
>>> That way we could test the input-/output handling capabilities (of 
>>> all three data types) of the services and we would still test 
>>> everything regarding execute that is tested right now (i.e. we would 
>>> not loose functionality).
>>> Do you have any views on that?
>>>
>>> Cheers,
>>>
>>> Benjamin
>>>
>>> -- 
>>> Benjamin Proß
>>> Software Engineer
>>> 52°North Geoprocessing Community
>>>
>>> 52°North Initiative for Geospatial Open Source Software GmbH
>>> Martin-Luther-King-Weg 24
>>> Fon: +49-(0)-251–396371-42
>>> Fax: +49-(0)-251–396371-11
>>> b.pross at 52north.org
>>> http://52north.org/
>>>
>>> General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
>>> Local Court Muenster HRB 10849
>>>
>>> _______________________________________________
>>> Wps-discuss mailing list
>>> Wps-discuss at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/wps-discuss
>>
>>
>> Gérald Fenoy
>> http://wiki.osgeo.org/wiki/User:Djay
>>
>
>


-- 
Benjamin Proß
Software Engineer
52°North Geoprocessing Community

52°North Initiative for Geospatial Open Source Software GmbH
Martin-Luther-King-Weg 24
Fon: +49-(0)-251–396371-42
Fax: +49-(0)-251–396371-11
b.pross at 52north.org
http://52north.org/

General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
Local Court Muenster HRB 10849