From francesco.martinelli at ingv.it Tue Dec 5 03:20:54 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Tue, 5 Dec 2023 12:20:54 +0100 Subject: [pygeoapi] Process - how to return a reference? Message-ID: <3cce9469-9335-4f32-af54-9e427e4e4564@ingv.it> Hi there, I am new to this list, and possibly I am asking simple questions, but I found no info around in the web. I would like to publish a "process", and to allow for the result(s) to be returned as reference. I installed and tested pygeoapi, but I didn't find an example/template to request the result as reference, i.e. using the parameter "transmissionMode": "reference". The case I am interested in, is where the output parameter to return as reference is a file (i.e. a generic bytestream). However I also wonder if a process with "outputTransmission": ['value', 'reference'] should also allow for all other output parameter types to be retrieved as reference. Also, having "outputTransmission": ['reference'] (i.e. without the option 'value') would conform to the standard? What I am looking for is: a basic example on the response (including the headers) that the client is expected to receive where an output parameter of an execute request has set the property "transmissionMode": "reference". Even better if it were possible to have a code snippet on how to make it. Also, in api.py, describe_processes(), I found the "outputTransmission" is fixed to only be ['value'], while there is no option to load the process metadata specific value associated to "outputTransmission". I would like to minimize (if not avoiding) the changes in pygeoapi framework, while allowing this option. Thank you for any help, Francesco From ricardo.garcia.silva at gmail.com Wed Dec 6 02:58:44 2023 From: ricardo.garcia.silva at gmail.com (Ricardo Filipe Soares Garcia da) Date: Wed, 6 Dec 2023 10:58:44 +0000 Subject: [pygeoapi] Process - how to return a reference? In-Reply-To: <3cce9469-9335-4f32-af54-9e427e4e4564@ingv.it> References: <3cce9469-9335-4f32-af54-9e427e4e4564@ingv.it> Message-ID: Hi Francesco You are correct in your investigations. Pygeoapi does not currently have support for outputting process execution results by reference. Unfortunately, support for version 1.0.0 of the OGC API - Processes standard is still not 100% in pygeoapi, and the missing bits include return result by reference. I had opened an issue to mention this previously, see: https://github.com/geopython/pygeoapi/issues/1285 Hopefully, this thread will help it gain a bit more traction ;) Best regards Francesco Martinelli via pygeoapi escreveu no dia ter?a, 5/12/2023 ?(s) 11:21: > Hi there, > I am new to this list, and possibly I am asking simple questions, > but I found no info around in the web. > > I would like to publish a "process", > and to allow for the result(s) to be returned as reference. > I installed and tested pygeoapi, > but I didn't find an example/template to request the result as reference, > i.e. using the parameter "transmissionMode": "reference". > > The case I am interested in, is where the output parameter to return as > reference is a file (i.e. a generic bytestream). > However I also wonder if a process with > "outputTransmission": ['value', 'reference'] > should also allow for all other output parameter types to be retrieved > as reference. > Also, having "outputTransmission": ['reference'] (i.e. without the > option 'value') would conform to the standard? > > > What I am looking for is: > a basic example on the response (including the headers) that the client > is expected to receive where an output parameter of an execute request > has set the property "transmissionMode": "reference". > Even better if it were possible to have a code snippet on how to make it. > > Also, in api.py, describe_processes(), I found the "outputTransmission" > is fixed to only be ['value'], > while there is no option to load the process metadata specific value > associated to "outputTransmission". > > I would like to minimize (if not avoiding) the changes in pygeoapi > framework, while allowing this option. > > > Thank you for any help, > Francesco > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > -- ___________________________ ___ __ Ricardo Garcia Silva -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco.martinelli at ingv.it Wed Dec 6 08:29:35 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Wed, 6 Dec 2023 17:29:35 +0100 Subject: [pygeoapi] Process - is it defined how to retrieve single output of a job result ? Message-ID: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> Hi there, I am not sure this is the appropriate mailing list where to ask: if this is inappropriate I apologize, and if possible I ask to advise where should I post my question. Reading the OGC API - PROCESSES -Part 1: CORE I don't find a way (if possible) to retrieve a single output result from a job. Specifically I am looking to paragraph 7.13 If I submit an execute request, and I get back a UUID for the job (i.e. a 'Location' in the header, where the location contains the UUID/jobID), then I can thereafter get the results of the job on the path /jobs/{jobID}/results. However if I had multiple outputs (possibly of sensible size) and I want to get only one, I don't find a way to get only that single output: is it possible? If positive: how to specify this request? Thank you, Francesco From ricardo.garcia.silva at gmail.com Wed Dec 6 09:32:12 2023 From: ricardo.garcia.silva at gmail.com (Ricardo Filipe Soares Garcia da) Date: Wed, 6 Dec 2023 17:32:12 +0000 Subject: [pygeoapi] Process - is it defined how to retrieve single output of a job result ? In-Reply-To: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> References: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> Message-ID: Hi Francesco If I understand the question correctly, then I think table 12 of the OGC API - Processes standard provides you with the answer. The way that results are accessed depends on your original processing request: 1. if you requested sync execution, results ought to be transmitted in a similar way as the one described for async (more on that below) 2. if you requested async execution (or let's say you requested sync, but want to retrieve the results again), then it depends on a number of factors: 2.1. If response type is 'raw' and transmission mode is 'value' and there is a single output, then the result will be available at the link indicated by the 'Location' header in the raw output type (e.g. a geotiff file that was generated as the result of the process execution) 2.2. If response type is 'raw' and transmission mode is 'value' and there are a multiple outputs, then the result will be a multipart/related response with each result being a 'part', and they will be available at the link indicated by the 'Location' header (e.g. multiple geotiffs generated, then response will have all of them concatenated, each will be a 'part' of the message) - This is the same way as an email client would transmit images via email. Browse the Internet for information on the 'multipart/related' media type in order to know more about this 2.3. If response type is 'raw' and transmission mode is 'reference' and there are is either a single output or multiple outputs, then when you visit the resource that the 'Location' header links to you will get back an empty response body, but the response headers shall contain a 'Link' header for each output, with the value of the header being a link for each output where you should be able to get each result in its raw output type (i.e. there will be a 'Link' header with a link for the actual geotiff generated as a process result) 2.4. If response type is 'raw' and transmission mode is mixed (i.e. there are multiple outputs and some of them are to be transmitted by 'value' while others by 'reference') then the result will again be a multipart/related resource and presumably there will be one 'part' for each output that is being transmitted by 'value' and one 'Link' header for each output being transmitted by 'reference' - I think this is my interpretation of the standard, it does not actually say this though 2.5. If the response type is 'document', the response will always be a JSON document that complies with the relevant 'results.yaml' schema (as mentioned in the standard). 2.5.1 For each generated process result that has its transmission mode requested as 'value', the actual result is transmitted in the JSON document. If the result it is a JSON type (int, str, object, etc) then it is shown directly. If the result is some binary data, then it should be encoded as a string, maybe using base64, but there is a provision for specifying the encoding of the data and also the media-type of theunderlying data - you can check the example 2 provided in section 7.13.2 of the standard for more info on this 2.5.2. For each generated process result that has its transmission mode set to 'reference' then the response JSON document will include a link to the resource. I hope this makes it a bit clearer. Please note that, as mentioned in a previous message, pygeoapi does not currently support this form of result retrieval, not yet anyway ;) - it effectively ignores whatever form of output generation has been (thereby always assuming response type 'raw' and transmission mode 'value'). As for where to post these kinds of questions, I'm not sure, but maybe you can open an issue at the standard's github repo? Here: https://github.com/opengeospatial/ogcapi-processes/issues I guess asking them here might also be appropriate. Best regards Francesco Martinelli via pygeoapi escreveu no dia quarta, 6/12/2023 ?(s) 16:29: > Hi there, > I am not sure this is the appropriate mailing list where to ask: > if this is inappropriate I apologize, > and if possible I ask to advise where should I post my question. > > Reading the OGC API - PROCESSES -Part 1: CORE > I don't find a way (if possible) to retrieve a single output result from > a job. > Specifically I am looking to paragraph 7.13 > > If I submit an execute request, and I get back a UUID for the job (i.e. > a 'Location' in the header, where the location contains the UUID/jobID), > then I can thereafter get the results of the job on the path > /jobs/{jobID}/results. > > However if I had multiple outputs (possibly of sensible size) > and I want to get only one, > I don't find a way to get only that single output: is it possible? > If positive: how to specify this request? > > > Thank you, > Francesco > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > -- ___________________________ ___ __ Ricardo Garcia Silva -------------- next part -------------- An HTML attachment was scrubbed... URL: From xbartolone at gmail.com Wed Dec 6 11:58:55 2023 From: xbartolone at gmail.com (Francesco Bartoli) Date: Wed, 6 Dec 2023 20:58:55 +0100 Subject: [pygeoapi] Process - is it defined how to retrieve single output of a job result ? In-Reply-To: References: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> Message-ID: <4a732db1-625a-4035-9995-aeef38909868@Spark> Hi Francesco, First of all, thanks for reaching the pygeoapi community out and point out the lack of compliance for the capabilities of OGC API Processes. I would second the comment from Ricardo and I do believe that part would definitively need some love. The best you can do at the moment is in sparse order: - Open an issue (if it doesn?t exist) to report that our implementation is behind the specification with the details of what is not aligned; - Contribute to the project by forking and sending a pull request to close the issue; - Fund a service provider to do the work for you if it is crucial for your organisation. Hope this helps. Kind regards, Francesco On 6 Dec 2023, 18:32 +0100, Ricardo Filipe Soares Garcia da via pygeoapi , wrote: > Hi Francesco > > If I understand the question correctly, then I think table 12 of the OGC API - Processes standard provides you with the answer. The way that results are accessed depends on your original processing request: > > 1. if you requested sync execution, results ought to be transmitted in a similar way as the one described for async (more on that below) > 2. if you requested async execution (or let's say you requested sync, but want to retrieve the results again), then it depends on a number of factors: > > 2.1. If response type is 'raw' and transmission mode is 'value' and there is a single output, then the result will be available at the link indicated by the 'Location' header in the raw output type (e.g. a geotiff file that was generated as the result of the process execution) > > 2.2. If response type is 'raw' and transmission mode is 'value' and there are a multiple outputs, then the result will be a multipart/related response with each result being a 'part', and they will be available at the link indicated by the 'Location' header (e.g. multiple geotiffs generated, then response will have all of them concatenated, each will be a 'part' of the message) - This is the same way as an email client would transmit images via email. Browse the Internet for information on the 'multipart/related' media type in order to know more about this > > 2.3. If response type is 'raw' and transmission mode is 'reference' and there are is either a single output or multiple outputs, then when you visit the resource that the 'Location' header links to you will get back an empty response body, but the response headers shall contain a 'Link' header for each output, with the value of the header being a link for each output where you should be able to get each result in its raw output type (i.e. there will be a 'Link' header with a link for the actual geotiff generated as a process result) > > 2.4. If response type is 'raw' and transmission mode is mixed (i.e. there are multiple outputs and some of them are to be transmitted by 'value' while others by 'reference') then the result will again be a multipart/related resource and presumably there will be one 'part' for each output that is being transmitted by 'value' and one 'Link' header for each output being transmitted by 'reference' - I think this is my interpretation of the standard, it does not actually say this though > > 2.5. If the response type is 'document', the response will always be a JSON document that complies with the relevant 'results.yaml' schema (as mentioned in the standard). > > 2.5.1 For each generated process result that has its transmission mode requested as 'value', the actual result is transmitted in the JSON document. If the result it is a JSON type (int, str, object, etc) then it is shown directly. If the result is some binary data, then it should be encoded as a string, maybe using base64, but there is a provision for specifying the encoding of the data and also the media-type of theunderlying data - you can check the example 2 provided in section 7.13.2 of the standard for more info on this > > 2.5.2. For each generated process result that has its transmission mode set to 'reference' then the response JSON document will include a link to the resource. > > > I hope this makes it a bit clearer. > > > Please note that, as mentioned in a previous message, pygeoapi does not currently support this form of result retrieval, not yet anyway ;) - it effectively ignores whatever form of output generation has been (thereby always assuming response type 'raw' and transmission mode 'value'). > > As for where to post these kinds of questions, I'm not sure, but maybe you can open an issue at the standard's github repo? Here: > > https://github.com/opengeospatial/ogcapi-processes/issues > > I guess asking them here might also be appropriate. > > > Best regards > > > > > Francesco Martinelli via pygeoapi escreveu no dia quarta, 6/12/2023 ?(s) 16:29: > > > Hi there, > > > I am not sure this is the appropriate mailing list where to ask: > > > if this is inappropriate I apologize, > > > and if possible I ask to advise where should I post my question. > > > > > > Reading the OGC API - PROCESSES -Part 1: CORE > > > I don't find a way (if possible) to retrieve a single output result from > > > a job. > > > Specifically I am looking to paragraph 7.13 > > > > > > If I submit an execute request, and I get back a UUID for the job (i.e. > > > a 'Location' in the header, where the location contains the UUID/jobID), > > > then I can thereafter get the results of the job on the path > > > /jobs/{jobID}/results. > > > > > > However if I had multiple outputs (possibly of sensible size) > > > and I want to get only one, > > > I don't find a way to get only that single output: is it possible? > > > If positive: how to specify this request? > > > > > > > > > Thank you, > > > Francesco > > > _______________________________________________ > > > pygeoapi mailing list > > > pygeoapi at lists.osgeo.org > > > https://lists.osgeo.org/mailman/listinfo/pygeoapi > > > -- > ___________________________ ___ __ > Ricardo Garcia Silva > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco.martinelli at ingv.it Thu Dec 7 00:33:36 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Thu, 7 Dec 2023 09:33:36 +0100 Subject: [pygeoapi] Process - is it defined how to retrieve single output of a job result ? In-Reply-To: <4a732db1-625a-4035-9995-aeef38909868@Spark> References: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> <4a732db1-625a-4035-9995-aeef38909868@Spark> Message-ID: Thank you Francesco and Ricardo. My organization is a research institute, and currently I am investigating on how to use the API, trying to make some proof of concept. I could (and would like to) try helping, but first of all I need to fully understand how it works (or should work). I'll make some more questions hoping they require a short answer to not waste your time. At the moment I am interested in response type 'raw' only. I would like to initially focus on transmission mode 'value', and then would like to consider the 'reference'. The outputs of the test process "hello_world" is one 'object', "echo", with the properties (not defined in the metadata) 'id' and 'value'. The class API, in method execute_process(), *if mime_type == 'application/json'* serializes the output. The mime_type is set by the process "hello_world". Now, if "hello_world" returned an application/json, I would expect it to be a string, therefore not to be serialized. Am I missing something? In any case, is it correct (in this specific case of a single output value) the returned 'body' contains the process output ? If I had the outputs section of the process metadata defined as the following: 'outputs': { 'output_file': { 'title': 'Some title', 'description': 'A generic file', 'schema': { 'type': 'string', 'contentEncoding': 'binary' } } }, should I expect the returned body to be a binary stream encoded base64? If the outputs section were: outputs: { 'test_1': { type: string }, 'test_2': { type: string } } and the actual values from the process output were: test_1 = "abc" test_2 = "123" I understand the mime_type should be set to 'multipart/related', but what about the body? Please, can you write the text of the body I should get from such an execute request. Thank you very much in advance for your time. Hope to be able to contribute shortly. Francesco Il giorno mer 6 dic 2023 alle ore 20:59 Francesco Bartoli < xbartolone at gmail.com> ha scritto: > Hi Francesco, > > First of all, thanks for reaching the pygeoapi community out and point out > the lack of compliance for the capabilities of OGC API Processes. > I would second the comment from Ricardo and I do believe that part would > definitively need some love. > The best you can do at the moment is in sparse order: > - Open an issue (if it doesn?t exist) to report that our implementation is > behind the specification with the details of what is not aligned; > - Contribute to the project by forking and sending a pull request to close > the issue; > - Fund a service provider to do the work for you if it is crucial for your > organisation. > > Hope this helps. > > Kind regards, > Francesco > On 6 Dec 2023, 18:32 +0100, Ricardo Filipe Soares Garcia da via pygeoapi < > pygeoapi at lists.osgeo.org>, wrote: > > Hi Francesco > > If I understand the question correctly, then I think table 12 of the OGC > API - Processes standard provides you with the answer. The way that results > are accessed depends on your original processing request: > > 1. if you requested sync execution, results ought to be transmitted in a > similar way as the one described for async (more on that below) > 2. if you requested async execution (or let's say you requested sync, but > want to retrieve the results again), then it depends on a number of factors: > > 2.1. If response type is 'raw' and transmission mode is 'value' and there > is a single output, then the result will be available at the link indicated > by the 'Location' header in the raw output type (e.g. a geotiff file that > was generated as the result of the process execution) > > 2.2. If response type is 'raw' and transmission mode is 'value' and there > are a multiple outputs, then the result will be a multipart/related > response with each result being a 'part', and they will be available at the > link indicated by the 'Location' header (e.g. multiple geotiffs generated, > then response will have all of them concatenated, each will be a 'part' of > the message) - This is the same way as an email client would transmit > images via email. Browse the Internet for information on the > 'multipart/related' media type in order to know more about this > > 2.3. If response type is 'raw' and transmission mode is 'reference' and > there are is either a single output or multiple outputs, then when you > visit the resource that the 'Location' header links to you will get back an > empty response body, but the response headers shall contain a 'Link' header > for each output, with the value of the header being a link for each output > where you should be able to get each result in its raw output type (i.e. > there will be a 'Link' header with a link for the actual geotiff generated > as a process result) > > 2.4. If response type is 'raw' and transmission mode is mixed (i.e. there > are multiple outputs and some of them are to be transmitted by 'value' > while others by 'reference') then the result will again be a > multipart/related resource and presumably there will be one 'part' for each > output that is being transmitted by 'value' and one 'Link' header for each > output being transmitted by 'reference' - I think this is my interpretation > of the standard, it does not actually say this though > > 2.5. If the response type is 'document', the response will always be a > JSON document that complies with the relevant 'results.yaml' schema (as > mentioned in the standard). > > 2.5.1 For each generated process result that has its transmission mode > requested as 'value', the actual result is transmitted in the JSON > document. If the result it is a JSON type (int, str, object, etc) then it > is shown directly. If the result is some binary data, then it should be > encoded as a string, maybe using base64, but there is a provision for > specifying the encoding of the data and also the media-type of > theunderlying data - you can check the example 2 provided in section 7.13.2 > of the standard for more info on this > > 2.5.2. For each generated process result that has its transmission mode > set to 'reference' then the response JSON document will include a link to > the resource. > > > I hope this makes it a bit clearer. > > > Please note that, as mentioned in a previous message, pygeoapi does not > currently support this form of result retrieval, not yet anyway ;) - it > effectively ignores whatever form of output generation has been (thereby > always assuming response type 'raw' and transmission mode 'value'). > > As for where to post these kinds of questions, I'm not sure, but maybe you > can open an issue at the standard's github repo? Here: > > https://github.com/opengeospatial/ogcapi-processes/issues > > I guess asking them here might also be appropriate. > > > Best regards > > > > Francesco Martinelli via pygeoapi escreveu no > dia quarta, 6/12/2023 ?(s) 16:29: > >> Hi there, >> I am not sure this is the appropriate mailing list where to ask: >> if this is inappropriate I apologize, >> and if possible I ask to advise where should I post my question. >> >> Reading the OGC API - PROCESSES -Part 1: CORE >> I don't find a way (if possible) to retrieve a single output result from >> a job. >> Specifically I am looking to paragraph 7.13 >> >> If I submit an execute request, and I get back a UUID for the job (i.e. >> a 'Location' in the header, where the location contains the UUID/jobID), >> then I can thereafter get the results of the job on the path >> /jobs/{jobID}/results. >> >> However if I had multiple outputs (possibly of sensible size) >> and I want to get only one, >> I don't find a way to get only that single output: is it possible? >> If positive: how to specify this request? >> >> >> Thank you, >> Francesco >> _______________________________________________ >> pygeoapi mailing list >> pygeoapi at lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/pygeoapi >> > > > -- > ___________________________ ___ __ > Ricardo Garcia Silva > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco.martinelli at ingv.it Mon Dec 11 03:25:07 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Mon, 11 Dec 2023 12:25:07 +0100 Subject: [pygeoapi] Question on suspicious bug in API.execute_process() / BaseProcessor.execute() Message-ID: I have a question on the returned parameter from the following method: *BaseProcessor.execute()*, and how it is used from *API.execute_process()*. The inline documentation for *BaseProcessor.execute()* says: "*returns: tuple of MIME type and process response*" What I am interested in is the "*process response*". *BaseProcessor.execute()* is called by *BaseManager.execute_process()* (indirectly through *_execute_handler_sync()* ), which pass back to the caller (*API.execute_process()* ) the "*process response*" unchanged. If the *execute-request* had the parameter *"response": "raw"* or undefined (whch defauls to *"raw"*), then *API.execute_process()*, if *mime_type == 'application/json',* assumes the "*process response*" (i.e. actual variable *response*) is an object, and *converts the object to a string* to be returned as *content*, in all other cases *API.execute_process()* returns the original "*process response*" unchanged (i.e. assuming it is a string). To my understanding *the above is not correct*: if the mime_type = 'application/json' then the "process response" *should be a string* representing an object, *not an object*. Even more: any "*process response*" returned by *BaseProcessor.execute()* should be a *string*, coded according to the *mime_type*, and in case of an *execute-request* with *"response"="raw"*, the "*process response*" should be returned unchanged as "*content*" from *API.execute_process()*. Specifically about Mime application/json: "This file type is a data interchange format that is text-based" (ref. https://mimeapplication.net/json) In any case I would like to have some feedback in case I missed something, before rising the above as a bug issue. I would also propose to specify in the inline documentation for *BaseProcessor.execute()* the types of the returned items into the tuple (string, string): Thank you, Francesco -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco.martinelli at ingv.it Tue Dec 19 05:46:50 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Tue, 19 Dec 2023 14:46:50 +0100 Subject: [pygeoapi] Process - how to request a job result with mimetype='multipart/related' ? In-Reply-To: References: <94120799-a79f-43ee-96aa-3c99014ae1a5@ingv.it> Message-ID: <850dcbd3-c9c7-472c-9182-ac80b9f8d599@ingv.it> Dear all, thank you to Ricardo, I elaborated his answer, but I am still confused. As per ? 6.1: "JSON is the encoding for request and responses." As stated in requirement 17, response default to "raw", but also allows "document". I understand that responses may accept a format parameter: possible values are "json" (the default) and "html". At this point I could guess "raw"<->"json", "document"<->"html". The example next to requirement 43 is about a job result with no format specific parameter, and it returns a response encoded JSON. As far as I understand, for a job with: -) execution: async -) transmission mode: value -) # outputs: * the "body content" will be: -) if the format is missing (i.e. "json") the returned value is encoded JSON (as per the above example) -) if the format is "html" (i.e. "document") the returned value is encoded JSON (mimetype='application/json', as per tables 11 and 12) [There is no example: should it be the same as before?] **Question is**: how is it specified that the response of the above example should be of type "raw" (mimetype='multipart/related')? Thak you, Francesco On 06/12/23 18:32, Ricardo Filipe Soares Garcia da wrote: > Hi Francesco > > If I understand the question correctly, then I think table 12 of the OGC > API - Processes standard provides you with the answer. The way that > results are accessed depends on your original processing request: > > 1. if you requested sync execution, results ought to be transmitted in a > similar way as the one described for async (more on that below) > 2. if you requested async execution (or let's say you requested sync, > but want to retrieve the results again), then it depends on a number of > factors: > > 2.1. If response type is 'raw' and transmission mode is 'value' and > there is a single output, then the result will be available at the link > indicated by the 'Location' header in the raw output type (e.g. a > geotiff file that was generated as the result of the process execution) > > 2.2. If response type is 'raw' and transmission mode is 'value' and > there are a multiple outputs, then the result will be a > multipart/related response with each result being a 'part', and they > will be available at the link indicated by the 'Location' header (e.g. > multiple geotiffs generated, then response will have all of them > concatenated, each will be a 'part' of the message) - This is the same > way as an email client would transmit images via email. Browse the > Internet for information on the 'multipart/related' media type in order > to know more about this > > 2.3. If response type is 'raw' and transmission mode is 'reference' and > there are is either a single output or multiple outputs, then when you > visit the resource that the 'Location' header links to you will get back > an empty response body, but the response headers shall contain a 'Link' > header for each output, with the value of the header being a link for > each output where you should be able to get each result in its raw > output type (i.e. there will be a 'Link' header with a link for the > actual geotiff generated as a process result) > > 2.4. If response type is 'raw' and transmission mode is mixed (i.e. > there are multiple outputs and some of them are to be transmitted by > 'value' while others by 'reference') then the result will again be a > multipart/related resource and presumably there will be one 'part' for > each output that is being transmitted by 'value' and one 'Link' header > for each output being transmitted by 'reference' - I think this is my > interpretation of the standard, it does not actually say this though > > 2.5. If the response type is 'document', the response will always be a > JSON document that complies with the relevant 'results.yaml' schema (as > mentioned in the standard). > > 2.5.1 For each generated process result that has its transmission mode > requested as 'value', the actual result is transmitted in the JSON > document. If the result it is a JSON type (int, str, object, etc) then > it is shown directly. If the result is some binary data, then it should > be encoded as a string, maybe using base64, but there is a provision for > specifying the encoding of the data and also the media-type of > theunderlying data - you can check the example 2 provided in section > 7.13.2 of the standard for more info on this > > 2.5.2. For each generated process result that has its transmission mode > set to 'reference' then the response JSON document will include a link > to the resource. > > > I hope this makes it a bit clearer. > > > Please note that, as mentioned in a previous message, pygeoapi does not > currently support this form of result retrieval, not yet anyway ;) - it > effectively ignores whatever form of output generation has been (thereby > always assuming response type 'raw' and transmission mode 'value'). > > As for where to post these kinds of questions, I'm not sure, but maybe > you can open an issue at the standard's github repo? Here: > > https://github.com/opengeospatial/ogcapi-processes/issues > > > I guess asking them here might also be appropriate. > > > Best regards > > > > Francesco Martinelli via pygeoapi > escreveu no dia quarta, 6/12/2023 > ?(s) 16:29: > > Hi there, > I am not sure this is the appropriate mailing list where to ask: > if this is inappropriate I apologize, > and if possible I ask to advise where should I post my question. > > Reading the OGC API - PROCESSES -Part 1: CORE > I don't find a way (if possible) to retrieve a single output result > from > a job. > Specifically I am looking to paragraph 7.13 > > If I submit an execute request, and I get back a UUID for the job (i.e. > a 'Location' in the header, where the location contains the UUID/jobID), > then I can thereafter get the results of the job on the path > /jobs/{jobID}/results. > > However if I had multiple outputs (possibly of sensible size) > and I want to get only one, > I don't find a way to get only that single output: is it possible? > If positive: how to specify this request? > > > Thank you, > Francesco > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > > > > > -- > ___________________________ ___ __ > Ricardo Garcia Silva From francesco.martinelli at ingv.it Sat Dec 23 06:38:02 2023 From: francesco.martinelli at ingv.it (Francesco Martinelli) Date: Sat, 23 Dec 2023 15:38:02 +0100 Subject: [pygeoapi] About an example class Message-ID: @tomkralidis About examples, I wanted to make a simple class derived from BaseProcessor: just an "echo" of different types of input (i.e. string, numbers, objects, array, bytes). As the class works fine for "simple" types, I would like to extend the possible inputs to include the whole set of inputs described in the standard for "processes". Would such class be of interest? [Personally I wanted to make it as a starting point for my development.] If positive: would it be the "examples" folder (possibly of the new project you suggested today) appropriate for it? I already found some issues, namely (at least) with multiple outputs where "array type" is present, but I am not sure if the problem is my coding (very likely) or the API. I have no experience in "collaborative development", so I am not sure what is the best way and best place to share my code and ask for support/collaboration. I also think that at some point the class could be part of test cases, as my aim is to provide the inputs/outputs as described in the "API STANDARD" document. In this case I am not sure "example" is the right place. However I never worked with "tests", and I would need some help if it would be suggested to include the class in the testbed. Thank you for any advice, Francesco -- Dott. Ing. Francesco Martinelli Istituto Nazionale di Geofisica e Vulcanologia Sezione di Pisa Via Cesare Battisti, 53 - 56125 Pisa Tel. 050 8311964 -------------- next part -------------- An HTML attachment was scrubbed... URL: From justb4 at gmail.com Mon Dec 25 06:16:27 2023 From: justb4 at gmail.com (Just van den Broecke) Date: Mon, 25 Dec 2023 15:16:27 +0100 Subject: [pygeoapi] [vote] PSC: approve budget $100 for pygeoapi demo server Message-ID: Dear pygeoapi PSC-members, Please your vote for allocating $100,- budget from $4000,- (From OSGeo TBD see [1]) in 2024 for the pygeoapi demo server at demo.pygeoapi.io. This covers both single VM-hosting plus domain name + DNS pygeoapi.io. So only PSC-members to vote! My vote: +1 [1] https://github.com/geopython/pygeoapi/wiki/OSGeo-Budget-2024 Best, Just van den Broecke - @justb4 From tomkralidis at gmail.com Tue Dec 26 06:16:47 2023 From: tomkralidis at gmail.com (Tom Kralidis) Date: Tue, 26 Dec 2023 09:16:47 -0500 Subject: [pygeoapi] [vote] PSC: approve budget $100 for pygeoapi demo server In-Reply-To: References: Message-ID: +1. Thanks Just for maintaining this important resource. Thanks ..Tom On Mon, Dec 25, 2023 at 9:16?AM Just van den Broecke via pygeoapi < pygeoapi at lists.osgeo.org> wrote: > Dear pygeoapi PSC-members, > > Please your vote for allocating $100,- budget from $4000,- (From OSGeo > TBD see [1]) in 2024 for the pygeoapi demo server at demo.pygeoapi.io. > > This covers both single VM-hosting plus domain name + DNS pygeoapi.io. > > So only PSC-members to vote! > > My vote: +1 > > [1] https://github.com/geopython/pygeoapi/wiki/OSGeo-Budget-2024 > > Best, > > Just van den Broecke - @justb4 > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomkralidis at gmail.com Tue Dec 26 12:56:53 2023 From: tomkralidis at gmail.com (Tom Kralidis) Date: Tue, 26 Dec 2023 15:56:53 -0500 Subject: [pygeoapi] PSC vote: Approve Travel Cost for OGC/OSGeo/ASF Sprint attendance Message-ID: Dear pygeoapi PSC: This is a motion to approve up to 2 pygeoapi developers to travel to the joint OGC/OSGeo/ASF Sprint in ?vora, Portugal on 26-28 February 2024 [1]. The budget to approve is $3900 USD and will cover travel expenses. Starting with my +1 ..Tom [1] https://developer.ogc.org/sprints/23/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gcpp.kalxas at gmail.com Wed Dec 27 03:53:47 2023 From: gcpp.kalxas at gmail.com (Angelos Tzotsos) Date: Wed, 27 Dec 2023 13:53:47 +0200 Subject: [pygeoapi] PSC vote: Approve Travel Cost for OGC/OSGeo/ASF Sprint attendance In-Reply-To: References: Message-ID: <07fab29b-6f25-4fca-a366-7b0ebdaeb962@gmail.com> +1 Angelos On 12/26/23 22:56, Tom Kralidis via pygeoapi wrote: > Dear pygeoapi PSC: > > This is a motion to approve up to 2 pygeoapi developers to travel to the > joint OGC/OSGeo/ASF Sprint in ?vora, Portugal on 26-28 February 2024 [1]. > > The budget to approve is $3900 USD and will cover travel expenses. > > Starting with my +1 > > ..Tom > > [1] https://developer.ogc.org/sprints/23/ > > > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi -- Angelos Tzotsos, PhD President Open Source Geospatial Foundation http://users.ntua.gr/tzotsos From xbartolone at gmail.com Wed Dec 27 18:11:12 2023 From: xbartolone at gmail.com (Francesco Bartoli) Date: Thu, 28 Dec 2023 03:11:12 +0100 Subject: [pygeoapi] [vote] PSC: approve budget $100 for pygeoapi demo server In-Reply-To: References: Message-ID: +1, thanks Francesco On 26 Dec 2023, 15:17 +0100, Tom Kralidis via pygeoapi , wrote: > +1.? Thanks Just for maintaining this important resource. > > Thanks > > ..Tom > > > On Mon, Dec 25, 2023 at 9:16?AM Just van den Broecke via pygeoapi wrote: > > > Dear pygeoapi PSC-members, > > > > > > Please your vote for allocating $100,- budget from $4000,- (From OSGeo > > > TBD see [1]) in 2024 for the pygeoapi demo server at demo.pygeoapi.io. > > > > > > This covers both single VM-hosting plus domain name + DNS pygeoapi.io. > > > > > > So only PSC-members to vote! > > > > > > My vote: +1 > > > > > > [1] https://github.com/geopython/pygeoapi/wiki/OSGeo-Budget-2024 > > > > > > Best, > > > > > > Just van den Broecke - @justb4 > > > _______________________________________________ > > > pygeoapi mailing list > > > pygeoapi at lists.osgeo.org > > > https://lists.osgeo.org/mailman/listinfo/pygeoapi > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi -------------- next part -------------- An HTML attachment was scrubbed... URL: From genuchten at yahoo.com Thu Dec 28 08:14:05 2023 From: genuchten at yahoo.com (Paul Van Genuchten) Date: Thu, 28 Dec 2023 17:14:05 +0100 Subject: [pygeoapi] [vote] PSC: approve budget $100 for pygeoapi demo server References: <5626DEB7-54AF-405A-89E8-2D33E04CE898.ref@yahoo.com> Message-ID: <5626DEB7-54AF-405A-89E8-2D33E04CE898@yahoo.com> +1 Groet Paul van Genuchten From genuchten at yahoo.com Thu Dec 28 08:13:07 2023 From: genuchten at yahoo.com (Paul Van Genuchten) Date: Thu, 28 Dec 2023 17:13:07 +0100 Subject: [pygeoapi] PSC vote: Approve Travel Cost for OGC/OSGeo/ASF Sprint attendance References: <9FF82D84-070A-4D1C-BE20-32BFCD88B3F6.ref@yahoo.com> Message-ID: <9FF82D84-070A-4D1C-BE20-32BFCD88B3F6@yahoo.com> +1 Groet Paul van Genuchten From gcpp.kalxas at gmail.com Fri Dec 29 01:19:34 2023 From: gcpp.kalxas at gmail.com (Angelos Tzotsos) Date: Fri, 29 Dec 2023 11:19:34 +0200 Subject: [pygeoapi] [vote] PSC: approve budget $100 for pygeoapi demo server In-Reply-To: References: Message-ID: <1fd31f40-07d2-48b1-b59c-9951c0a7a16c@gmail.com> +1 Angelos On 12/25/23 16:16, Just van den Broecke via pygeoapi wrote: > Dear pygeoapi PSC-members, > > Please your vote for allocating $100,- budget from $4000,- (From OSGeo > TBD see [1]) in 2024 for the pygeoapi demo server at demo.pygeoapi.io. > > This covers both single VM-hosting plus domain name + DNS pygeoapi.io. > > So only PSC-members to vote! > > My vote: +1 > > [1] https://github.com/geopython/pygeoapi/wiki/OSGeo-Budget-2024 > > Best, > > Just van den Broecke - @justb4 > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi -- Angelos Tzotsos, PhD President Open Source Geospatial Foundation http://users.ntua.gr/tzotsos From xbartolone at gmail.com Sat Dec 30 07:43:20 2023 From: xbartolone at gmail.com (Francesco Bartoli) Date: Sat, 30 Dec 2023 16:43:20 +0100 Subject: [pygeoapi] Fwd: Re: PSC vote: Approve Travel Cost for OGC/OSGeo/ASF Sprint attendance In-Reply-To: <54cdb64f-91a1-4095-afb9-f6b0399d5b06@Spark> References: <54cdb64f-91a1-4095-afb9-f6b0399d5b06@Spark> Message-ID: By mistake I replied only to Tom, forward my +1 to the list Thanks, Francesco ---------- Forwarded message ---------- From: Francesco Bartoli Date: 27 Dec 2023, 09:20 +0100 To: Tom Kralidis Subject: Re: [pygeoapi] PSC vote: Approve Travel Cost for OGC/OSGeo/ASF Sprint attendance > +1, thanks Tom > > Francesco > On 26 Dec 2023, 21:57 +0100, Tom Kralidis via pygeoapi , wrote: > > Dear pygeoapi PSC: > > > > This is a motion to approve up to 2 pygeoapi developers to travel to the > > joint OGC/OSGeo/ASF Sprint in ?vora, Portugal on 26-28 February 2024 [1]. > > > > The budget to approve is $3900 USD and will cover travel expenses. > > > > Starting with my +1 > > > > ..Tom > > > > [1] https://developer.ogc.org/sprints/23/ > > _______________________________________________ > > pygeoapi mailing list > > pygeoapi at lists.osgeo.org > > https://lists.osgeo.org/mailman/listinfo/pygeoapi -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomkralidis at gmail.com Sun Dec 31 10:44:54 2023 From: tomkralidis at gmail.com (Tom Kralidis) Date: Sun, 31 Dec 2023 13:44:54 -0500 Subject: [pygeoapi] for review: RFC2: Dependency policy In-Reply-To: References: Message-ID: On Tue, Nov 28, 2023 at 10:10?AM Ricardo Filipe Soares Garcia da via pygeoapi wrote: > Thanks for preparing the RFC, I think pygeoapi needs more of this type of > stuff :) > I wish it would be a bit more detailed though. Perhaps I'm just missing > the context from the discussions that were already had about it, but having > just read it, I found the text to be very succinct, and was left wanting to > know more details about the proposal. > > More specifically: > > 1. With regard to the actual dependency management policy, I must admit I > did not really understand some parts of it - perhaps it could be a bit more > verbose, even with one or two examples thrown in there? To elaborate on > this, when the text mentions that: > > "... > master branch dependencies and requirements shall be consistent against a > given operating system baseline: > > - Ubuntu dependencies > - pip3 requirements > - Docker / Docker Compose > ..." > > Excuse my obtuseness, but what does this mean exactly? Is it that the > master branch shall only rely upon those libraries (and respective > versions) which can be found on a given OS' baseline repositories, which I > guess is being proposed to be the current Ubuntu LTS? And then the various > requirements.txt files and Dockerfile would restrict themselves to the same > libraries and versions? - I'm not saying I do not agree with such a policy, > it is just that I did not really understand if it is this or something > else. > > Yes. pip3 requirements (at the time of release at least) shall be available on Ubuntu. Docker and Docker Compose would be built off Ubuntu as well. > > 2. The RFC mentions that "...Careful consideration shall be given when new > dependencies are proposed..." but I would like it to elaborate on this with > points about: > > - What would be some criteria for deciding whether an additional > dependency ought to be added? > Based on personal experience, the idea would be (on top of Ubuntu availability): - health/pulse of a dependency as a project - is the dependency really needed? For example, can the same functionality be achieved with Python primitives or the standard library, even if less elegant? > - What would be the process to follow for when a new dependency is being > considered or proposed? I think this could be documented somewhere. I see > this question as being relevant in the context of pygeoapi providers and > their inclusion as part of the 'core pygeoapi experience' and what would be > a good policy for inclusion, as it seems most dependencies would come from > providers. But there are other cases where it could be beneficial to have > some guidelines - with recent examples being the usage of gunicorn or > pydantic. Just to be clear, I'm not trying to make the case for inclusion > of any dependency - this is just so that the whole process becomes a bit > clearer and easier to adhere to > > This may become a moot point as we evolve to 1.0. The thinking is that pygeoapi proper would be a minimal set of plugins with bare / minimal dependencies, and that a pygeoapi-plugins repository would be a growing repository where pygeoapi plugins would be maintained / developed. To be discussed at our upcoming meetings/sprints. > > 3. The RFC mentions that pygeoapi maintainers would provide a master > branch and additional maintenance branches for each minor and major > release, with relevant security fixes being backported from master. This > seems like a substantial increase in maintenance-related work, but I guess > it may depend on additional details, which the RFC does not provide. Some > questions I think would be worth considering: > > Yes, I think this approach is needed for the long term viability of pygeoapi. It is work, but it is also sustainable and a sign/indicator of a serious project with a long term commitment. > - How many simultaneous stable branches do you propose to have in addition > to master? Does it make sense to keep more than one (which would be the > latest release)? > We would have a stable branch for every major and minor release. We would also need a policy of how may branches are supported. n - 1, for example, would be a suitable approach. There will always be exceptions of course (think a CVE which would need patching and backporting across numerous previous versions). > - The RFC mentions a number of dissemination channels (UbuntuGIS, pip3, > docker, etc) as being officially supported. Do you want to keep all of > them? Could there be other dissemination channels that make sense > considering? I imagine that each of these distribution channels would also > need to be updated whenever a new maintenance version got pushed. > > I think our critical path support is Ubuntu, but yes that doesn't stop us from providing additional channels (see https://docs.pygeoapi.io/en/latest/installation.html for example). > I'm wondering if this last item would not be perhaps more appropriate for > a post-1.0.0 release? Or maybe be the subject of a different RFC? > > +1 to discuss in another thread/meeting/sprint. Note that RFC2 has now been updated and now going for PSC vote. > Anyway, thanks for putting it forth and I hope these help in improving the > overall document. > > Thank you for your valuable feedback Ricardo! ..Tom > Best regards > > > Tom Kralidis via pygeoapi escreveu no dia > ter?a, 28/11/2023 ?(s) 00:09: > >> Hi all: as discussed at the OSGeo Community Sprint earlier this month, >> please find RFC2 [1], for >> review and discussion. If/once we update as needed, I will put to a PSC >> vote for formal approval. >> >> Thanks >> >> ..Tom >> >> [1] https://pygeoapi.io/development/rfc/2 >> _______________________________________________ >> pygeoapi mailing list >> pygeoapi at lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/pygeoapi >> > > > -- > ___________________________ ___ __ > Ricardo Garcia Silva > _______________________________________________ > pygeoapi mailing list > pygeoapi at lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/pygeoapi > -------------- next part -------------- An HTML attachment was scrubbed... URL: