[PyWPS-dev] wps raster processing pipeline (chaining)

[Idan Miara]

Hi Jachym,

Thanks for your reply!
I was able to make a working first single pywps service for a different
task - cropping a DEM raster to a given cutline, make a color relief out of
it and then return the result as a CZML. I think it would be good to use
these individually, thus I though about chaining.
I wasn't able to figure out how the chaining works, if not for this
particular task I think it would be useful for other tasks I need to
develop. I will publish it as open source very soon and ask some followup
questions regarding improving it.

Kind regards,
Idan

On Mon, 17 Feb 2020, 20:54 Jachym Cepicky, <jachym.cepicky at gmail.com> wrote:

> Hi Idan,
>
> sorry for late response,
>
> do you really need to chain processes? Wouldn't it be enough, just to
> write single process, maybe consisting out of several python modules,
> which would apply the three steps you described?
>
> I would say, storing raster data to harddrive is not bad idea, if the
> harddrive is SSD, then it's usually significantly faster
>
> and saving the data on the server can be used as "cache".
>
> Anyway, I'm not completely sure, where the biggest issue of yours
> currently is: do you know, how to implement the tasks in python, but
> you are not sure, how to wrap it as an PyWPS Process? Are you
> completely unaware, how to implement this in Python at all? Everything
> is clear, you have your 3 processes, you just need to figure out, how
> to chain them?
>
> Jachym
>
> ne 2. 2. 2020 v 11:50 odesílatel Idan Miara <idan at miara.com> napsal:
> >
> > Hi All,
> >
> > I'm new to pywps :)
> > I'm trying to find a way to make an efficient raster processing pipeline.
> > For instance, with Geoserver wps I was able to chain wps requests to
> make the following chain:
> > 1. clipping a DEM raster from a geoserver layer.
> > 2. apply an SLD style
> > 3. publish the result on geoserver.
> >
> > I wanted to know how can I do something similar with pywps and possibly
> reuse gdal tools (such as gdal_translate, gdal_transform or gdaldem via
> their python interfaces)
> > Some points to consider:
> > 1. I couldn't find a way to chain requests on pywps.
> > 2. As raster data can be heavy, ideally intermediate rasters won't be
> saved/loaded to disk, but kept in memory and deleted once the result is
> ready (maybe use gdal memory raster?)
> > 3. I want to make use of cache in a way that multiple requests for the
> same input raster(s) blocks will make use of cache (maybe gdal internal
> cache could be good).
> > 4. Is there a best practice for serving the result raster(s) and
> possibly use them as cache in case the same request is submitted again to
> the service?
> >
> > Any pointers would be appreciated!
> > Kind regards,
> > Idan
> >
> > _______________________________________________
> > pywps-dev mailing list
> > pywps-dev at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/pywps-dev
>
>
>
> --
> Jachym Cepicky
> e-mail: jachym.cepicky gmail com
> URL: http://les-ejk.cz
> GPG: http://les-ejk.cz/pgp/JachymCepicky.pgp
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pywps-dev/attachments/20200217/abb1f703/attachment.html>