[Zoo-discuss] Introduction - Rémi Cresson

[Rémi Cresson]

Hello,

My name is Rémi Cresson, I work in the field of remote sensing image 
processing. I am currently with french National Research Institute of 
Science and Technology for Environment and Agriculture (Irstea) to 
develop a geospatial data infrastructure for the GEOSUD project. I work 
mainly with c++ libraries (GDAL, OTB, ITK, GEOS, ...) for image 
processing pipelines development. A goal of the project is to use 
supercomputers for remote sensing data processing, binded with web services.
We tried Zoo because it was the only open source WPS server which has a 
friendly binding with OTB applications. We used it, and found that 
Gerald did a nice work (at least a very solid proof of concept). 
However, we needed to launch our programs from a HPC architecture (i.e. 
not programs hosted by the WPS server as Zoo kernel do with OTB 
applications libraries). Indeed, we wanted to make use of our 
supercomputer's front node job scheduler in order to delegate resources 
management, job submitting, ... That leads us to implement a new Zoo 
service, which use ssh/sftp on the WPS server side and DRMAA on the HPC 
front node side. This new Zoo service basically transfert files between 
WPS server and HPC front node, construct a command line and submit a 
job, then transfert back files. We make it working with OTB applications 
only, using the same .zcfg files (but with another service name). We 
think the approach can be generalized to other existing processing 
libraries (Grass, ...). In addition, it can for sure be extended to 
parallel applications (MPI), as DRMAA can be used to dispatch process on 
a given number of nodes.
I want to contribute to zoo-project with this service, and I might be 
interested to be PSC member of Zoo-project in a near future, if however 
this is an option.

Thank you for taking time to read this, I look forward to responses !

Cheers,

Rémi Cresson - Research Engineer, Irstea
Maison de la Télédétection
500 Rue de jean François Breton
34090 Montpellier