[Wps-discuss] WPS Benchmarking at FOSS4G 2014

Thu Sep 4 20:58:16 PDT 2014

The WPS 2.0 standard is winding its way past the geoserver-devel list :)

Thus far we have made no progress on benchmark, everyone seems busy with
workshops.

Jody Garnett

On Thu, Sep 4, 2014 at 3:54 AM, Benjamin Proß <b.pross at 52north.org> wrote:

> Hi all,
>
> Are there any news regarding the benchmark procedure? Or are you all busy
> writing comments for the WPS 2.0 candidate standard? :)
>
> Cheers,
>
> Benjamin
>
> Am 21.08.2014 13:00, schrieb Benjamin Proß:
>
>  Hello Gérald, all,
>>
>> I am glad that you like the idea and I hope the other projects feel the
>> same. The tests you are suggesting make sense, though I am not completely
>> sure about the combinations with the complexdata in-/output as
>> reference/inline xml. Complexdata output could also be requested as raw
>> data output. Maybe we should create a table listing the combinations. I
>> could start this and then we could choose the ones we want. We should also
>> define the process description, so the projects can create the respective
>> processes.
>>
>> Cheers,
>>
>> Benjamin
>>
>> Am 21.08.2014 11:16, schrieb Fenoy Gerald:
>>
>>> Hello Benjamin,
>>> I think it is a great idea to handle testings of the Execute request the
>>> way you described, by using a simple echo service.
>>>
>>> By now for synchronous requests, the tests look like the following:
>>>
>>>     • Test 1: value as JSON
>>>     • Test 2: value as GML
>>>     • Test 3: value as reference as GML
>>>     • Test 4: same as test 3, result as reference
>>>     • Test 5: value as reference as GML using POST request
>>>     • Test 6: same as test 5, result as reference
>>>
>>> And for asynchronous request, we have the following:
>>>
>>>     • Test 1': value as reference as GML
>>>     • Test 2': same as test 1, result as reference
>>>     • Test 3': value as reference as GML using POST request
>>>     • Test 4': same as test 3, result as reference
>>>
>>> Personally, I think that the Test 1 and Test 2 should become unique now
>>> as there is no need for parsing any value from the service itself as it
>>> will simply return the value without any treatment. So we should use only
>>> the Test 2 for our next run.
>>>
>>> I also suppose that we should multiply the number of tests as we will
>>> need to ask for each available outputs individually and all the possible
>>> combinaisons of them to also test this cases.
>>>
>>> So, I would like to propose the following updates for the tests run for
>>> both synchronous and asynchronous requests. We run each individual test 7
>>> times: first to have only once output at a time (so 3 tests), then to test
>>> all possible combinaisons of 2 outputs names (3 tests) and finally to have
>>> all the output returned (1 test).
>>>
>>> I hope that you can confirm that I’m right thinking of this updates to
>>> the current tests.
>>>
>>> I’ve added one column to the table [1] listing all WPS team
>>> participating to write the name of the echo service when it will be
>>> available.
>>>
>>> Best regards,
>>>
>>> [1] http://wiki.osgeo.org/wiki/WPS_benchmark_2014#Available_
>>> infrastructure
>>>
>>> Le 19 août 2014 à 16:03, Benjamin Proß <b.pross at 52north.org> a écrit :
>>>
>>>  Dear all,
>>>>
>>>> I see that there are still not all services are deployed on the test
>>>> machine. I hope you guys are still on board.
>>>> At 52°North we were thinking a bit about the execute tests.
>>>> We do not yet have any results to compare but we think that
>>>> benchmarking with "real" processes that compute something could falsify the
>>>> results, as the computation in most cases is done by a underlying library
>>>> (GeoTools in our case, but this could also be exchanged).
>>>> So the results would be influenced by how fast the geo-computation lib
>>>> would do its job. Not sure that we want that?!
>>>> We also do not test whether the process really did its job (e.g. buffer
>>>> the input geometry). So do we need to use "real" processes?
>>>> We would propose to use an echo process that could have three inputs,
>>>> Literal-, Complex- and BBoxData. The process then simply returns what it
>>>> gets (input/output types would have to match).
>>>> We would probably also need a asynchronous version that has a delay or
>>>> something.
>>>> That way we could test the input-/output handling capabilities (of all
>>>> three data types) of the services and we would still test everything
>>>> regarding execute that is tested right now (i.e. we would not loose
>>>> functionality).
>>>> Do you have any views on that?
>>>>
>>>> Cheers,
>>>>
>>>> Benjamin
>>>>
>>>> --
>>>> Benjamin Proß
>>>> Software Engineer
>>>> 52°North Geoprocessing Community
>>>>
>>>> 52°North Initiative for Geospatial Open Source Software GmbH
>>>> Martin-Luther-King-Weg 24
>>>> Fon: +49-(0)-251–396371-42
>>>> Fax: +49-(0)-251–396371-11
>>>> b.pross at 52north.org
>>>> http://52north.org/
>>>>
>>>> General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
>>>> Local Court Muenster HRB 10849
>>>>
>>>> _______________________________________________
>>>> Wps-discuss mailing list
>>>> Wps-discuss at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/wps-discuss
>>>>
>>>
>>>
>>> Gérald Fenoy
>>> http://wiki.osgeo.org/wiki/User:Djay
>>>
>>>
>>
>>
>
> --
> Benjamin Proß
> Software Engineer
> 52°North Geoprocessing Community
>
> 52°North Initiative for Geospatial Open Source Software GmbH
> Martin-Luther-King-Weg 24
> Fon: +49-(0)-251–396371-42
> Fax: +49-(0)-251–396371-11
> b.pross at 52north.org
> http://52north.org/
>
> General Managers: Dr. Albert Remke, Dr. Andreas Wytzisk
> Local Court Muenster HRB 10849
>
> _______________________________________________
> Wps-discuss mailing list
> Wps-discuss at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/wps-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/wps-discuss/attachments/20140904/00b3332d/attachment.html>