<div dir="ltr"><div><div><div>Hi Remi,<br><br></div>That sounds fantastic, I would be interested in trying out your code on some HPC services!<br><br></div>Cheers,<br><br></div>Tim<br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 7, 2015 at 1:33 AM, Rémi Cresson <span dir="ltr"><<a href="mailto:remi.cresson@teledetection.fr" target="_blank">remi.cresson@teledetection.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
Hello Nicolas and Gerald,<br>
<br>
Should I share the code through the Zoo-Project trac first? The code
is not really clean yet, I mean I have to improve a lot of things.<br>
<br>
By the way there are issues related to code and contribution want to
discuss about:<br>
-the service require an autonomous application (i.e. completely
independent of the zoo-kernel) on the supercomputer side (for job
submitting) and this application must be compiled on the
supercomputer. One way to cope with this problem may be to put this
application separately in the third parties?<br>
-I don't yet know the way to catch environment parameters (which are
in the main.cfg) <br>
-I have to understand how to properly activate the service at
compile time (e.g. the use of #ifdef USE_MYSERVICE in
zoo_service_loader.c ... where is defined the USE_MYSERVICE
variable?) for now, I just don't use #ifdef...<br>
<br>
Thanks for your replies,<br>
<br>
Best<span class="HOEnZb"><font color="#888888"><br>
<br>
Rémi</font></span><div><div class="h5"><br>
<br>
<div>On 06/08/2015 07:36, nicolas bozon
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hello Rémy,</div>
<div><br>
</div>
<div>Welcome to this list and thank you for such feedback and
contribution.</div>
<div>It is very interesting to learn how Irstea is using and
extending ZOO-Project for the needs of the Geosud SDI.</div>
<div>As Gerald mentioned, please feel to share your services
source code through the ZOO-Project trac directly.</div>
<div>You are of course most welcome as new committer! Please
accept the committers guidelines on this list first so we can
proceed.</div>
<div>IMHO, you would also be a great and valuable addition to
the ZOO-Project PSC. This will first take a request from the
PSC Chair and then vote by other PSC members.</div>
<div><br>
</div>
<div>Best regards,</div>
<div><br>
</div>
<div>Nick</div>
<div><br>
</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2015-08-05 15:02 GMT+02:00 Fenoy
Gerald <span dir="ltr"><<a href="mailto:gerald.fenoy@geolabs.fr" target="_blank">gerald.fenoy@geolabs.fr</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Welcome
aboard Rémi !<br>
<br>
I am glad to hear that you are using the ZOO-Project and
its OTB support even if it was not your original goal as
you need to run the service remotely from the WPS server
rather than running them directly on the same server.<br>
<br>
The work and the implementation you made looks very
interesting and is a real asset to the ZOO-Project giving
it the capability to be linked to a HPC infrastructure. So
we welcome your contribution.<br>
<br>
Anybody can contribute code to the ZOO_Project, obviously
if you need write access to the SVN then it will require
that you are part of the ZOO-Project developers [1]. You
should be member of the ZOO-Project PSC only in case you
want to deal with the project organisation, plan and so
on.<br>
<br>
Prior to ask the PSC for a vote, we should make sure that
you agree with the committer guideline [2], once
confirmed, I can then forward the request to the PSC.<br>
<br>
Hope to hear back from you,<br>
Best regards,<br>
<br>
<br>
[1] <a href="http://zoo-project.org/docs/contribute/code.html#for-registered-developers" rel="noreferrer" target="_blank">http://zoo-project.org/docs/contribute/code.html#for-registered-developers</a><br>
[2] <a href="http://zoo-project.org/docs/contribute/dev.html" rel="noreferrer" target="_blank">zoo-project.org/docs/contribute/dev.html</a><br>
<div>
<div><br>
> Le 4 août 2015 à 15:21, Rémi Cresson <<a href="mailto:remi.cresson@teledetection.fr" target="_blank">remi.cresson@teledetection.fr</a>>
a écrit :<br>
><br>
> Hello,<br>
><br>
> My name is Rémi Cresson, I work in the field of
remote sensing image processing. I am currently with
french National Research Institute of Science and
Technology for Environment and Agriculture (Irstea) to
develop a geospatial data infrastructure for the
GEOSUD project. I work mainly with c++ libraries
(GDAL, OTB, ITK, GEOS, ...) for image processing
pipelines development. A goal of the project is to use
supercomputers for remote sensing data processing,
binded with web services.<br>
> We tried Zoo because it was the only open source
WPS server which has a friendly binding with OTB
applications. We used it, and found that Gerald did a
nice work (at least a very solid proof of concept).
However, we needed to launch our programs from a HPC
architecture (i.e. not programs hosted by the WPS
server as Zoo kernel do with OTB applications
libraries). Indeed, we wanted to make use of our
supercomputer's front node job scheduler in order to
delegate resources management, job submitting, ...
That leads us to implement a new Zoo service, which
use ssh/sftp on the WPS server side and DRMAA on the
HPC front node side. This new Zoo service basically
transfert files between WPS server and HPC front node,
construct a command line and submit a job, then
transfert back files. We make it working with OTB
applications only, using the same .zcfg files (but
with another service name). We think the approach can
be generalized to other existing processing libraries
(Grass, ...). In addition, it can for sure be extended
to parallel applications (MPI), as DRMAA can be used
to dispatch process on a given number of nodes.<br>
> I want to contribute to zoo-project with this
service, and I might be interested to be PSC member of
Zoo-project in a near future, if however this is an
option.<br>
><br>
> Thank you for taking time to read this, I look
forward to responses !<br>
><br>
> Cheers,<br>
><br>
> Rémi Cresson - Research Engineer, Irstea<br>
> Maison de la Télédétection<br>
> 500 Rue de jean François Breton<br>
> 34090 Montpellier<br>
><br>
> _______________________________________________<br>
> Zoo-discuss mailing list<br>
> <a href="mailto:Zoo-discuss@lists.osgeo.org" target="_blank">Zoo-discuss@lists.osgeo.org</a><br>
> <a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss" rel="noreferrer" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss</a><br>
<br>
<br>
<br>
</div>
</div>
Gérald Fenoy<br>
<a href="http://wiki.osgeo.org/wiki/User:Djay" rel="noreferrer" target="_blank">http://wiki.osgeo.org/wiki/User:Djay</a><br>
<div>
<div><br>
_______________________________________________<br>
Zoo-discuss mailing list<br>
<a href="mailto:Zoo-discuss@lists.osgeo.org" target="_blank">Zoo-discuss@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss" rel="noreferrer" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss</a></div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Zoo-discuss mailing list<br>
<a href="mailto:Zoo-discuss@lists.osgeo.org">Zoo-discuss@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss" rel="noreferrer" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/zoo-discuss</a><br></blockquote></div><br></div>