[Pywps-dev] Guidelines for netCDF file and opendap accesss within pywps

[David Huard]

Hi all,

I'd like to contribute a pull request to better handle netCDF files in
pywps but I don't know where to start.

We have a number of processes taking netCDF
<https://www.unidata.ucar.edu/software/netcdf/> files as inputs. For those
less familiar with the format, netCDF is based on HDF5 and a set of
conventions <http://cfconventions.org/>. It is the standard data format in
oceanography and climatology. netCDF files are usually stored on servers
with support for opendap <https://www.opendap.org/>. This means that users
can either download the netCDF file and then open it locally, or use the
opendap protocol to open it remotely. What that means is that you can do

from netCDF4 import nc
ds1 = nc.Dataset("<path to local file>")
ds2 = nc.Dataset("<link to opendap address>")

and both ds1 and ds2 will behave identically. However ds2 is not downloaded
locally, but rather read remotely on demand. If a file contains a 3D matrix
(time, lat, lon), you can read one slice of the matrix without downloading
it all.

Some of our pywps.Process support both netCDF file and opendap access. We
define a ComplexInput for the address to an actual netCDF file, and a
LiteralInput for the opendap address.

My question is whether there would be a clean way for pywps to support both
modes with one ComplexInput? Internally, pywps would check if the address
supports opendap (just check if nc.Dataset(url) works), and if not, would
download the file locally to the server.

In both cases, we could do

ds = nc.Dataset(requests.inputs['resource'][0].file)

I'm willing to put the time to do it, I just don't know where to start.

Thanks,

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20180621/2e9379ab/attachment.html>