[PyWPS-dev] Handle complex input of a process within/without a chain in a same way possible?

Thu Jan 17 03:23:10 PST 2019

Check the get_output fonction in tests/ test_execute.py from the master.

Le jeu. 17 janv. 2019 05:01, Carsten Ehbrecht <ehbrecht at dkrz.de> a écrit :

> Hi Gunnar,
>
> In our project we use a workflow-engine to chain WPS processes … or let's
> say another software that does the extra work to chain the WPS processes
> (select the output reference for the input). It might be the case that the
> chaining with a single WPS xml request document works only in the way you
> have described.
>
> Cheers,
> Carsten
>
>
>
> On 16 Jan 2019, at 2:27 pm, Gunnar Ströer <gunnar.stroeer at yahoo.de> wrote:
>
> Hello Everybody,
>
> sorry but I have to ask you once again because I couldn't find any
> solution for my problem yet. I'll try to describe as best I can.
>
> Chaining: For all process outputs along the chain I have to write the
> <wps:ResponseForm> part of my XML POST request. Here when I use
> <wps:RawDataOutput> I can directly access/use the output from first process
> for the second process and so on. So I have to define each process output
> as <wps:RawDataOutput>. That works.
>
> In my case I don't want to use <wps:RawDataOutput> because data can be
> very large and binary too, so I prefer the use of <wps:Output
> asReference="true">. For example:
>
> <wps:ResponseForm>
>     <wps:ResponseDocument lineage="false" storeExecuteResponse="false"
> status="false">
>         <wps:Output asReference="true" mimeType="application/gml-3.1.1"
> encoding="utf-8">
>             <ows:Identifier>out_intersect</ows:Identifier>
>         </wps:Output>
>     </wps:ResponseDocument>
> </wps:ResponseForm>
>
> Then I get the whole response XML document as input for the second
> process. Now when I read the first output as input for the second process I
> have to parse the whole XML for the referenced file within the response
> section. I can not simply use the PyWPS input attribute like >>
> request.inputs['in_geom_a'][0].file <<.
>
> So my problem is to find a valid function or attribute in the PyWPS
> library that gives me only the referenced response file instead the whole
> response XML. Or, in case of, a statement that in PyWPS no way exists for
> doing that I want to do... yet.
>
> Are there any ideas to solve that problem?
>
> My repository is on GitLab, for example the intersection process:
> https://gitlab.com/hadlaskard/integration-of-wps-in-local-sdi/blob/master/processes/proc_vect_intersect.py
>
> Best and thanks,
> Gunnar
> Am Mittwoch, 19. Dezember 2018, 13:21:29 MEZ hat Suzana Barreto <
> suzana.barreto at envsys.co.uk> Folgendes geschrieben:
>
>
>
> Hi Gunner,
>
> Thanks for getting back to me, yes you have understood my request, I too
> tried to use the above example xml template to chain a request but came to
> the conclusion that it was failing because the nested <wps:Body> tag was
> Geoserver specific - perhaps I was incorrect, if you say that you have a
> working chain with RawData.   My understanding is that the server should
> seamlessly use the output from one request as input to the next request
> without any further coding required because it understands that a nested
> request's output is its input parameter, and that is why I thought the xml
> request was incorrect.  I too was trying to chain with a ComplexInput that
> was the result of a ComplexOutput from a WFS request.  I will have to
> revisit it and see what my errors were, perhaps it was not an error in the
> xml.  I will keep you posted on my progress, and thanks for the discussion.
>
> Suzana
>
> On Wed, 19 Dec 2018 at 10:32, Gunnar Ströer <gunnar.stroeer at yahoo.de>
> wrote:
>
> Hi Suzana,
>
> I agree a process should not know whether it is part of a chain. If you
> ask for a working XML request my example request above works fine for a
> simple chaining. My question aims to the python code behind the request,
> how to handle a chain with PyWPS and without using RawData. Of course it is
> possible to be wrong with my XML. I understand a chained request as a
> nested process of single processes, from inside to outside. The most inner
> request is the first process of your chain (intersection in my case), and
> the most surrounding process ist your last process (buffer in my case).
> I've used following URL for understanding:
>
>
> https://docs.geoserver.org/latest/en/user/services/wps/processes/chaining.html
>
> I hope I understood you correctly.
>
> Best,
> Gunnar
> Am Mittwoch, 19. Dezember 2018, 10:43:54 MEZ hat Suzana Barreto <
> suzana.barreto at envsys.co.uk> Folgendes geschrieben:
>
>
> Hi Gunner,
>
> I have been trying to get chaining to work but have not had any luck yet,
> my understanding is that a process should not know whether it is part of a
> chain.  I would be interested in knowing how you have structured your
> chained request, if you wouldn't mind sharing that information with me.
>
> All the best,
> Suzana
>
> On Mon, 17 Dec 2018 at 13:07, Carsten Ehbrecht <ehbrecht at dkrz.de> wrote:
>
> Hi Gunnar,
>
> I haven’t looked into process chaining, but this question was just raised
> recently:
>
> https://lists.osgeo.org/pipermail/pywps-dev/2018-December/001363.html
>
> Here is in addition a description of how a process chain can be done:
> http://geoprocessing.info/wpsdoc/Concepts#chaining
>
> I’m not sure how your code and your chain works (the gitlab link is not
> public). A comment from my side would be, that your “worker” process should
> not know that it is part of a process chain.
>
> Cheers,
> Carsten
>
> On 12 Dec 2018, at 6:34 pm, gunnar.stroeer at yahoo.de wrote:
>
> Hi Everybody,
>
> I currently develop some processes for a masterthesis using PyWPS v4 as
> implementation. Mostly it works fine for single processes, as well with
> QGIS as client. However I stucked when it comes to chaining.
>
> Normally in single processes I use "in_file =
> request.inputs['in_geom'][0].file" for read in some complex GML geometry.
> In cases I wanna use the same process in a chain the code above doesn't
> work anymore because the input is the same as the output of the prior
> process. Therefore "in_file = request.inputs['in_geom'][0].file"
> additionally gets all the XML code of the prior response - and the process
> crashed.
>
> To handle this I use "import lxml" to parse the response XML looking for
> the "xlink:href" attribute value with the referenced GML output file of the
> first process. It works for the process in a chain, only. Whether the
> process is used in a chain or not I could handle in the process code, like
> a case distinction. Therefor I have to parse XML all the time again and
> again... and it makes me feel that I use PyWPS in a wrong way.
>
> I also could use "RawDataOutput", but this is obviously a bad solution for
> large data inputs/outputs.
>
> Is there any method to handle the described problem without writing too
> much code just to get know whether the process is used in a chain or not?
> Or is there something completely wrong what I'm doing?
>
>
> https://gitlab.com/hadlaskard/integration-of-wps-in-local-sdi/blob/master/processes/proc_buffer.py
>
> Thanks,
> Gunnar
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev
>
>
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev
>
>
>
> --
>
>
>
> *Dr Suzana BarretoSoftware AnalystEnvironment Systems*
> Tel: +44 (0) 1970 626688
> www.envsys.co.uk
>
> The information contained in this e-mail is intended only for the use of
> the person(s) to whom it is addressed and may contain confidential or
> privileged information. If you have received this e-mail in error please
> contact the sender and delete the material without copying, distributing or
> disclosing the content. All reasonable precautions have been taken to
> ensure that this e-mail is free from any virus, however, the recipient
> should verify this to be the case. Please 'think before you print'
>
>
>
> --
>
>
>
> *Dr Suzana BarretoSoftware AnalystEnvironment Systems*
> Tel: +44 (0) 1970 626688
> www.envsys.co.uk
>
> The information contained in this e-mail is intended only for the use of
> the person(s) to whom it is addressed and may contain confidential or
> privileged information. If you have received this e-mail in error please
> contact the sender and delete the material without copying, distributing or
> disclosing the content. All reasonable precautions have been taken to
> ensure that this e-mail is free from any virus, however, the recipient
> should verify this to be the case. Please 'think before you print'
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev
>
>
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20190117/af44aa5c/attachment-0001.html>