[Qgis-developer] QGIS Processing Framework

[Martin Dobias]

Hi Julien!

On Wed, Apr 27, 2011 at 8:24 PM, Julien Malik <julien.malik at c-s.fr> wrote:
> We should be very careful about not being limited by "too much genericity".
>
> The SAGA and GRASS modules have each their own design/parameter types/inner
> but also outer logic.
> On the OTB side, we have some liberty since we are currently working on
> them, but we have some plans to integrate them in other tools, possibly
> benefiting from their additional possibilities (the pipeline capabilities of
> Monteverdi for example).
> We have some ideas for making the GUI as dynamic as possible (and it is with
> their Qt/Qgis implementation that we plan to have the most full-featured
> GUI) but I guess other libs will have capabilities that we don't have.
>
> Trying to fit everyone in the same framework will be hard : either each
> library (SAGA/GRASS/OTB) will only use a subset of the provided
> functionnalities, either we will end up taking the common factor of all libs
> and everyone will feel limited. I'm worried than we fall into one of these
> two pitfalls. Please do not impose to much constraints !

The point is actually not to provide every possible parameter type in
the framework. IMO we should have a base "parameter" type with several
common types such as integer, string, vector layer, enumeration. Such
types are used widely and QGIS would come with default edit widgets
for them.

Then, each library might require some additional parameter types that
might also require special editing widgets. The plugin implementing
support for such library will provide these types and editing widgets,
however they will stay opaque for QGIS. So in the result QGIS will
just know there is a parameter and it will know what widget to show in
the form, nothing more. Of course such custom types would not be
usable across various libraries, but that fine.

I don't know if anyone of you is experienced with web development, but
I see a nice parallel between models and views in web frameworks such
and Django and the processing framework here:
- in Django you define models consisting of common and/or custom
fields (integer, text, list etc) - here we talk about modules and
their parameters. Django then provides functionality like
loading/saving models from/to database, here we would provide running
of the modules.
- in Django you get a default html form for a model, but you can
customize it and provide further checking etc. Here QGIS would provide
default GUI for a module based on the description, with the
possibility for the developer to improve the basic behavior using Qt
signals and slots for a better user experience.

Therefore I tend to think about the whole procesing framework in two levels:
1. backend - similar to WPS: an interface for description of modules,
querying the modules and running them
2. gui - built on top of the backend, enhancing the usability and
experience for the user:

In the GUI part there are endless possibilities how to improve the
experience: the developer might want to add a preview for a module
that does some raster processing. Or the developer might add
interactive rectangle selection from map canvas for choosing a region
of interest for the analysis.

> [3]
> You seem to consider parameter validation only in an independent way
>
> From our experience it is insufficient (for example, absolute default values
> is not enough).
> How we plan to solve this (trying to keep it simple also...), is to give our
> modules 3 main entry points :
> - description of the parameters
> - a global method to update all the parameters, called as soon as one
> parameter is changed
> - an execution method
>
> Let's imagine a simple orthorectification application with the following
> params :
> - input/output image
> - the output CRS
> - output origin, size, and spacing
> We want the origin, size and spacing to be auto-computed each time the input
> or CRS changes
> If CRS changes from UTM to WGS84, we expect the outputorigin units/display
> to also change from meter to degree, for example.

I completely agree that many times the dependencies between the
parameters are far too complex to be modelled in a declarative way. I
hope my answer above clarifies this: you as a developer of the
orthorectification module would be allowed to watch for changes in the
form (using Qt signals) and update other parameters - or do whatever
you want.

> [5]
> The model we imagined for the parameter list will be a tree more than a flat
> list.
> A group of parameters is just one node of the tree, containing a parameters
> list, where each parameter can also be a group of parameters.

Well, I'm a bit afraid of trees of parameters. Would that be really
necessary for OTB?


>> Formats and format conversion: for import and export of map layers the
>> framework should support any layer loaded (or loadable) by QGIS. In
>> case the library uses a different format for input/output, it should
>> take care of import/export.
>
> I'm not sure this can be handled in the framework, but more in the different
> implementations.
> The input images to our modules will be OTB specific data objects, and the
> transformation from QGis raster (= filename) to OTB raster would be done the
> OTB-QGis plugins implementation (you can check the current Smoothing
> application code).

Probably I was not very clear that statement - but I have actually the
same opinion as you do: the format conversions should be handled
within the plugin implemented support for a given library. The
framework should not care about that at all.

Regards
Martin