[Qgis-developer] QGIS Processing Framework

[Martin Dobias]

Hi Soeren

thanks for your further input.

On Fri, Apr 29, 2011 at 10:45 AM, Soeren Gebbert
<soerengebbert at googlemail.com> wrote:
>> - WPS servers are IMHO not really widespread yet(?). Surely you can
>> install your own on a local computer but I don't think that an average
>> user is able to manage it with few clicks. And not everyone has access
>> to a fast server in basement.
>
> IMHO installing a WPS server with different back-ends is no magic. The
> web server, the WPS server and the back-ends can be bundled and
> installed as easy as QGIS with all its plugins and libraries and third
> party dependencies. This is technically no problem and IMHO the
> 52North guys are able to provide such an out of the box solution.

I agree that technically there are no serious problems. But still
adding further layers of software (wps client + wps server with a
backend) between user and the underlying analysis library increases
the risk of problems with distribution of the software and
configuration (e.g. should such bundled WPS server run as a service
all the time or only start it on request? should the server be
available only for the user who starts it or for everyone?).


>> The reasons above make me believe that we shouldn't depend exclusively
>> on WPS. In my vision WPS should be simply another provider of
>> processing modules next to SAGA, OTB or various other smaller plugins.
>> WPS itself simply cannot deliver the rich user interface that we would
>> like to provide.
>
> This is a good point. Maybe you can use the WPS specification as the
> base technology and extend it with additional features which will not
> break compatibility. Then you will use an OGC standard and you will be
> able to integrate WPS processes out of the box in the much more
> powerful framework with the possibility of chaining any kind of
> processes?

I think we are converging to an API very similar to WPS interface. It
makes sense to use the WPS specification to match the data types and
the protocol - so that building a WPS client and/or WPS server on top
of it should be simple.


> You will still need wrapper for each software you want to integrate to
> wrap there interface into the abstract QGIS process framework. So the
> design of a really powerful abstract process framework providing
> interaction and stuff is a lot of work and needs plenty of knowledge
> about all the software which might be integrated as processes in QGIS.

As I explained in one of the previous mails, the framework in my
opinion should consist of two concepts:
1. API for execution of modules (WPS-like): describe module, set
parameters, execute module, get results. No interactivity at this
stage (apart from progress/log information)
2. API for rich user interface for modules: auto-generation of dialogs
for modules, facilities for overriding default behavior

It needs to be designed properly (not too generic and not too
constrained) but I think this is no rocket science.

> To get an idea of the amount of work: Several very intelligent people
> have designed the WPS specification and they needed several years to
> version 1.0.0.

Just a side note: standards always take years to get designed and
approved. The time and number of people involved in the creation of a
specification does not necessarily correlate with its overall
usefulness and adoption by the industry. (I am speaking in general,
not related to WPS!)


> My main concern is the manpower which is needed to implement AND
> maintain such a framework as well as the wrapper and binding for other
> software packages. For example the GRASS GIS integration in QGIS. If
> there is nobody in the GRASS GIS or QGIS community who is willingly to
> take responsibility and time to maintain, update and extend it, it
> will be unusable in the future or it will depend on an obsolete or
> buggy GRASS GIS version.

You are right that it is easy to run out of manpower for maintenance.
The framework itself will not require a lot of maintenance once
implemented, a bigger problem might be with outdated adapters for
libraries. Sometimes a new soul arrives willing to fix some bugs,
sometimes the code is left for extinction.


> I am absolutely not against a cool abstract process framework in QGIS,
> but i would like to bring some IMHO important considerations into
> discussion.

We are indeed happy that you have joined the discussion!

Regards
Martin