[Zoo-discuss] Handling reference input files for processing Zoo-kernel

Tue May 3 07:39:59 PDT 2016

Hi Gerald,
Thanks again for timely response. 
If you refer to the figure 5 in OCG WPS spec 2.0 (page 25 of 133), there is a link between WPS server and external data source. This made me believe that WPS server fetches the data. 
Can I assume that if the data (referenced by input) is protected by some authentication, it cannot be fetched by zoo-kernel. Is that right? or does zoo-kernel supports some of the authentication configuration to fetch data from protected end points? 
I also believe that zoo-kernel supports HTTP and FTP endpoints for fetching the data. Is this correct?
Information that may of some use to others:I also wanted to know the complete list of keys that are passed to the processor via the HashMap. So I created Java processor to print the input HashMap. The processor takes 2 reference inputs and creates 2 outputs. Each output is string that represents the input Hashmap. 
Here is the output of wps:Result. I could clearly see that file data is available via "value" parameter and file reference is available via 3 parameters: Reference, xlink:href and href
<wps:Outputid="Result1"><wps:Data mimeType="text/plain">[inRequest=true, maxOccurs=1, cache_file=/home/ftep/04-data/wps//20cc9e739115b57d2a6de27d21e8ea37.zca, value=[B at 791a9e33, Order=1, Reference=http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW.tif, xlink:href=http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW.tif, href=http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW.tif, mimeType=application/binary, minOccurs=1, size=2101696]</wps:Data></wps:Output><wps:Outputid="Result2"><wps:Data mimeType="text/plain">[inRequest=true, maxOccurs=1, cache_file=/home/ftep/04-data/wps//720c7826d70201e919f6ea3b25140b36.zca, value=[B at 42886462, Order=2, Reference=http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv, xlink:href=http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv, href=http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv, mimeType=text/csv, minOccurs=1, size=1098]</wps:Data></wps:Output>

Thanks,Rakesh

> Subject: Re: [Zoo-discuss] Handling reference input files for processing Zoo-kernel
> From: gerald.fenoy at geolabs.fr
> Date: Tue, 3 May 2016 13:21:25 +0200
> CC: zoo-discuss at lists.osgeo.org
> To: rakeshuk at outlook.com
> 
> Hi Rakesh,
> thanks a lot for this questions which are all of high quality.
> 
> I cannot recall honestly if it is specified that the processing server should be responsible or not for providing the inputs when they are passed by reference. Nevertheless, you can be sure that in the case of the ZOO-Kernel, it is the case.
> 
> For any value passed by reference, rather than by value, won’t make much difference from the service developper point of view in the sense that it will receive in the value field for the corresponding input the content of the downloaded file (this file can be text file to binary). Obviously, you will also have a reference to the url given in the request in addition to the traditional value field.
> 
> For both the Orfeo Toolbox and SAGA-GIS support they handle the inputs more transparently, meaning that in that specific case we will run an OTB application, or SAGA, respectively, They require mohave access to the file on the filesystem so we simply add in such a case the reference to the local file stored. This require to activate the cache system available in the ZOO-Kernel.
> 
> Anyway, in any case the service developper don’t have to take much care about the way the files are downloaded as he will have access the same value field containing the content of the downloaded file.
> 
> In every language the same thing happen, so for the Java language specifically you will have a hash map for you input containing a hash named value with its value.
> 
> I hope my answer was clear and helped you to better understand the way intros are handled in the ZOO-Kernel.
> 
> Best regards,
> 
> > Le 3 mai 2016 à 12:48, rakesh prithiviraj <rakeshuk at outlook.com> a écrit :
> > 
> > Hi,
> > 
> > After reading OGC WPS spec 2.0.0, I tend to believe that it is the responsibility of the WPS server/ZOO-kernel to fetch the inputs specified in the WPS execute request. This means for inputs passed by reference, ZOO-Kernel will download the input and will provide it to processor. Is this true or is the processor responsible for fetching the inputs (specified by the URL)?
> > 
> > In the Orfeo toolbox example from ZOO project website, the execution request refers to TIF file from http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW.tif. Does BandMath is responsible for retrieving this input or does zoo-kernel retrieves this input and updates the path of the input in "inputs" parameter passed to the processor. (e.g. hashmap inputs for Java)?
> > 
> > http://localhost/cgi-bin/zoo_loader.cgi?request=Execute&service=WPS&version=1.0.0&Identifier=OTB.BandMath&DataInputs=il=Reference@xlink:href=http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW.tif;il=Reference@xlink:href=http://hg.orfeo-toolbox.org/OTB-Data/raw-file/ca154074b282/Examples/verySmallFSATSW_nir.tif;out=float;exp=im1b3*cos%28im1b1%29,im1b2*cos%28im1b1%29,im1b1*cos%28im1b1%29&RawDataOutput=out@mimeType=image/png  
> > 
> > Please advise.
> > 
> > Thanks,
> > Rakesh
> > _______________________________________________
> > Zoo-discuss mailing list
> > Zoo-discuss at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/zoo-discuss
> 
> 
> 
> Gérald Fenoy
> http://wiki.osgeo.org/wiki/User:Djay
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/zoo-discuss/attachments/20160503/e8d592d2/attachment.html>