[PyWPS-dev] Handle complex input of a process within/without a chain in a same way possible?

Thu Jan 17 08:17:07 PST 2019

 Hi Carsten,

I guess you are right, will follow the way David has mentioned.

Thanks for the get_output function, seems to be the part I need in my use case, written in a more efficient code than mine.

Cheers,
Gunnar

    Am Donnerstag, 17. Januar 2019, 12:23:23 MEZ hat David Huard <huard.david at ouranos.ca> Folgendes geschrieben:  

 Check the get_output fonction in tests/ test_execute.py from the master.

Le jeu. 17 janv. 2019 05:01, Carsten Ehbrecht <ehbrecht at dkrz.de> a écrit :

Hi Gunnar,
In our project we use a workflow-engine to chain WPS processes … or let's say another software that does the extra work to chain the WPS processes (select the output reference for the input). It might be the case that the chaining with a single WPS xml request document works only in the way you have described. 
Cheers,Carsten

On 16 Jan 2019, at 2:27 pm, Gunnar Ströer <gunnar.stroeer at yahoo.de> wrote:
 Hello Everybody,

sorry but I have to ask you once again because I couldn't find any solution for my problem yet. I'll try to describe as best I can.

Chaining: For all process outputs along the chain I have to write the <wps:ResponseForm> part of my XML POST request. Here when I use <wps:RawDataOutput> I can directly access/use the output from first process for the second process and so on. So I have to define each process output as <wps:RawDataOutput>. That works.

In my case I don't want to use <wps:RawDataOutput> because data can be very large and binary too, so I prefer the use of <wps:Output asReference="true">. For example:

<wps:ResponseForm>
    <wps:ResponseDocument lineage="false" storeExecuteResponse="false" status="false">
        <wps:Output asReference="true" mimeType="application/gml-3.1.1" encoding="utf-8">
            <ows:Identifier>out_intersect</ows:Identifier>
        </wps:Output>
    </wps:ResponseDocument>
</wps:ResponseForm>

Then I get the whole response XML document as input for the second process. Now when I read the first output as input for the second process I have to parse the whole XML for the referenced file within the response section. I can not simply use the PyWPS input attribute like >> request.inputs['in_geom_a'][0].file <<.

So my problem is to find a valid function or attribute in the PyWPS library that gives me only the referenced response file instead the whole response XML. Or, in case of, a statement that in PyWPS no way exists for doing that I want to do... yet.

Are there any ideas to solve that problem?

My repository is on GitLab, for example the intersection process: https://gitlab.com/hadlaskard/integration-of-wps-in-local-sdi/blob/master/processes/proc_vect_intersect.py

Best and thanks,
Gunnar
    Am Mittwoch, 19. Dezember 2018, 13:21:29 MEZ hat Suzana Barreto <suzana.barreto at envsys.co.uk> Folgendes geschrieben: 

 Hi Gunner,
Thanks for getting back to me, yes you have understood my request, I too tried to use the above example xml template to chain a request but came to the conclusion that it was failing because the nested <wps:Body> tag was Geoserver specific - perhaps I was incorrect, if you say that you have a working chain with RawData.   My understanding is that the server should seamlessly use the output from one request as input to the next request without any further coding required because it understands that a nested request's output is its input parameter, and that is why I thought the xml request was incorrect.  I too was trying to chain with a ComplexInput that was the result of a ComplexOutput from a WFS request.  I will have to revisit it and see what my errors were, perhaps it was not an error in the xml.  I will keep you posted on my progress, and thanks for the discussion.
Suzana
On Wed, 19 Dec 2018 at 10:32, Gunnar Ströer <gunnar.stroeer at yahoo.de> wrote:

  Hi Suzana,

I agree a process should not know whether it is part of a chain. If you ask for a working XML request my example request above works fine for a simple chaining. My question aims to the python code behind the request, how to handle a chain with PyWPS and without using RawData. Of course it is possible to be wrong with my XML. I understand a chained request as a nested process of single processes, from inside to outside. The most inner request is the first process of your chain (intersection in my case), and the most surrounding process ist your last process (buffer in my case). I've used following URL for understanding:

https://docs.geoserver.org/latest/en/user/services/wps/processes/chaining.html

I hope I understood you correctly.

Best,
Gunnar
    Am Mittwoch, 19. Dezember 2018, 10:43:54 MEZ hat Suzana Barreto <suzana.barreto at envsys.co.uk> Folgendes geschrieben:  

 Hi Gunner,
I have been trying to get chaining to work but have not had any luck yet, my understanding is that a process should not know whether it is part of a chain.  I would be interested in knowing how you have structured your chained request, if you wouldn't mind sharing that information with me. 
All the best,Suzana
On Mon, 17 Dec 2018 at 13:07, Carsten Ehbrecht <ehbrecht at dkrz.de> wrote:

Hi Gunnar,
I haven’t looked into process chaining, but this question was just raised recently:
https://lists.osgeo.org/pipermail/pywps-dev/2018-December/001363.html
Here is in addition a description of how a process chain can be done:http://geoprocessing.info/wpsdoc/Concepts#chaining
I’m not sure how your code and your chain works (the gitlab link is not public). A comment from my side would be, that your “worker” process should not know that it is part of a process chain.
Cheers,Carsten

On 12 Dec 2018, at 6:34 pm, gunnar.stroeer at yahoo.de wrote:
Hi Everybody,

I currently develop some processes for a masterthesis using PyWPS v4 as implementation. Mostly it works fine for single processes, as well with QGIS as client. However I stucked when it comes to chaining.

Normally in single processes I use "in_file = request.inputs['in_geom'][0].file" for read in some complex GML geometry. In cases I wanna use the same process in a chain the code above doesn't work anymore because the input is the same as the output of the prior process. Therefore "in_file = request.inputs['in_geom'][0].file" additionally gets all the XML code of the prior response - and the process crashed.

To handle this I use "import lxml" to parse the response XML looking for the "xlink:href" attribute value with the referenced GML output file of the first process. It works for the process in a chain, only. Whether the process is used in a chain or not I could handle in the process code, like a case distinction. Therefor I have to parse XML all the time again and again... and it makes me feel that I use PyWPS in a wrong way.

I also could use "RawDataOutput", but this is obviously a bad solution for large data inputs/outputs.

Is there any method to handle the described problem without writing too much code just to get know whether the process is used in a chain or not? Or is there something completely wrong what I'm doing?

https://gitlab.com/hadlaskard/integration-of-wps-in-local-sdi/blob/master/processes/proc_buffer.py

Thanks,
Gunnar
_______________________________________________
pywps-dev mailing list
pywps-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/pywps-dev

_______________________________________________
pywps-dev mailing list
pywps-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/pywps-dev

-- 

Dr Suzana Barreto
Software Analyst
Environment Systems
Tel: +44 (0) 1970 626688
www.envsys.co.uk

The information contained in this e-mail is intended only for the use of the person(s) to whom it is addressed and may contain confidential or privileged information. If you have received this e-mail in error please contact the sender and delete the material without copying, distributing or disclosing the content. All reasonable precautions have been taken to ensure that this e-mail is free from any virus, however, the recipient should verify this to be the case. Please 'think before you print'

-- 

Dr Suzana Barreto
Software Analyst
Environment Systems
Tel: +44 (0) 1970 626688
www.envsys.co.uk

The information contained in this e-mail is intended only for the use of the person(s) to whom it is addressed and may contain confidential or privileged information. If you have received this e-mail in error please contact the sender and delete the material without copying, distributing or disclosing the content. All reasonable precautions have been taken to ensure that this e-mail is free from any virus, however, the recipient should verify this to be the case. Please 'think before you print'
  _______________________________________________
pywps-dev mailing list
pywps-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/pywps-dev

_______________________________________________
pywps-dev mailing list
pywps-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/pywps-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20190117/e7b67255/attachment-0001.html>