[Pywps-dev] Guidelines for netCDF file and opendap accesss within pywps

[Jachym Cepicky]

Hi,

yes ComplexInput should work for you - you can pass the url with the data
using "<Reference ... />" element.. see [1] for example

Any Format can have (and has by default) `validator` function, which
return's, whether the input data are valid or no [3]. You can also use
`get_format` function [4] and set the validator there.

Example, how validating function can look can be shapefile or gml
validators [5]

You should probably extend foramts [2] with NetCDF mimetype

But, this will check the file only after it was downloaded to PyWPS - not
the URL. Still. is that sufficient?

Jachym

[1]
https://github.com/geopython/pywps/blob/master/tests/requests/wps_execute_request-responsedocument-1.xml#L24
[2]
https://github.com/geopython/pywps/blob/master/pywps/inout/formats/__init__.py
[3]
https://github.com/geopython/pywps/blob/master/pywps/inout/formats/__init__.py#L42
[4]
https://github.com/geopython/pywps/blob/master/pywps/inout/formats/__init__.py#L215
[5]
https://github.com/geopython/pywps/blob/master/pywps/validator/complexvalidator.py



čt 21. 6. 2018 v 17:15 odesílatel David Huard <huard.david at ouranos.ca>
napsal:

> Hi all,
>
> I'd like to contribute a pull request to better handle netCDF files in
> pywps but I don't know where to start.
>
> We have a number of processes taking netCDF
> <https://www.unidata.ucar.edu/software/netcdf/> files as inputs. For
> those less familiar with the format, netCDF is based on HDF5 and a set of
> conventions <http://cfconventions.org/>. It is the standard data format
> in oceanography and climatology. netCDF files are usually stored on servers
> with support for opendap <https://www.opendap.org/>. This means that
> users can either download the netCDF file and then open it locally, or use
> the opendap protocol to open it remotely. What that means is that you can
> do
>
> from netCDF4 import nc
> ds1 = nc.Dataset("<path to local file>")
> ds2 = nc.Dataset("<link to opendap address>")
>
> and both ds1 and ds2 will behave identically. However ds2 is not
> downloaded locally, but rather read remotely on demand. If a file contains
> a 3D matrix (time, lat, lon), you can read one slice of the matrix without
> downloading it all.
>
> Some of our pywps.Process support both netCDF file and opendap access. We
> define a ComplexInput for the address to an actual netCDF file, and a
> LiteralInput for the opendap address.
>
> My question is whether there would be a clean way for pywps to support
> both modes with one ComplexInput? Internally, pywps would check if the
> address supports opendap (just check if nc.Dataset(url) works), and if not,
> would download the file locally to the server.
>
> In both cases, we could do
>
> ds = nc.Dataset(requests.inputs['resource'][0].file)
>
> I'm willing to put the time to do it, I just don't know where to start.
>
> Thanks,
>
> David
>
>
>
>
>
>
>
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev



-- 
Jachym Cepicky
e-mail: jachym.cepicky gmail com
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/pgp/JachymCepicky.pgp
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20180622/c7e0e304/attachment.html>