[PyWPS-dev] wps raster processing pipeline (chaining)

[Jachym Cepicky]

Hi Idan,

sorry for late response,

do you really need to chain processes? Wouldn't it be enough, just to
write single process, maybe consisting out of several python modules,
which would apply the three steps you described?

I would say, storing raster data to harddrive is not bad idea, if the
harddrive is SSD, then it's usually significantly faster

and saving the data on the server can be used as "cache".

Anyway, I'm not completely sure, where the biggest issue of yours
currently is: do you know, how to implement the tasks in python, but
you are not sure, how to wrap it as an PyWPS Process? Are you
completely unaware, how to implement this in Python at all? Everything
is clear, you have your 3 processes, you just need to figure out, how
to chain them?

Jachym

ne 2. 2. 2020 v 11:50 odesílatel Idan Miara <idan at miara.com> napsal:
>
> Hi All,
>
> I'm new to pywps :)
> I'm trying to find a way to make an efficient raster processing pipeline.
> For instance, with Geoserver wps I was able to chain wps requests to make the following chain:
> 1. clipping a DEM raster from a geoserver layer.
> 2. apply an SLD style
> 3. publish the result on geoserver.
>
> I wanted to know how can I do something similar with pywps and possibly reuse gdal tools (such as gdal_translate, gdal_transform or gdaldem via their python interfaces)
> Some points to consider:
> 1. I couldn't find a way to chain requests on pywps.
> 2. As raster data can be heavy, ideally intermediate rasters won't be saved/loaded to disk, but kept in memory and deleted once the result is ready (maybe use gdal memory raster?)
> 3. I want to make use of cache in a way that multiple requests for the same input raster(s) blocks will make use of cache (maybe gdal internal cache could be good).
> 4. Is there a best practice for serving the result raster(s) and possibly use them as cache in case the same request is submitted again to the service?
>
> Any pointers would be appreciated!
> Kind regards,
> Idan
>
> _______________________________________________
> pywps-dev mailing list
> pywps-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pywps-dev



-- 
Jachym Cepicky
e-mail: jachym.cepicky gmail com
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/pgp/JachymCepicky.pgp