[Qgis-developer] QGIS Processing Framework

[Martin Dobias]

Hi Camilo

On Tue, Apr 26, 2011 at 9:46 PM, Camilo Polymeris <cpolymeris at gmail.com> wrote:
> Hello everyone,
> I was accepted as a student in GSoC to implement a "SAGA interface for QGIS".

First of all congratulations to you and to other accepted students for
the Summer of Code programme!

> There are also proyects that try to interface QGIS with GRASS and the
> Orfeo Toolbox (OTB). Paolo has informed me that the creation of a
> common framework for those systems was discussed and approved during
> HF. A very interesting idea, IMO, that completely changes the way I
> would have to work on my project.

Great that you like the idea of generic framework for processing tools
in QGIS. Helping us to bring this framework from ideas into code would
be another great benefit of your project.


> While I didn't participate in those discussions and don't know what
> conclusions were drawn, as a brainstorming exercise I am writing a
> short description of what a very simple API to that framework could
> look like:
>
> https://github.com/polymeris/qgis/wiki/QGIS-Processing-Framework
>
> Would that more or less fit what you have in mind? Please comment/criticize.

The discussion was not going into implementation details, we were
mostly collecting the ideas how the things should work.

Framework implementation: as you write in the proposal, ideally it
should be implemented in C++ with Python bindings. I would suggest you
to start the implementation in Python and skip C++ coding at least at
the beginning. My experience is that Python is great for this kind of
prototyping (no compiling, shorter code etc). If you will end up with
sufficient amount of time at the end of the summer, you can move that
into C++ and add Python bindings. The idea is not to waste time with
writing and rewriting various c++ classes when the API is not clear
yet. Consider this as a suggestion, not a command how to do it :-)

Forms/Parameters: I would start with with generic input fields (line
edit / spin box) for integers, floats, strings that can be constrained
with a range (numbers) or regular expressions (strings) - easily
achieved with QValidator classes. Then some custom widgets can be
added for better handling of some types: layer selector, enumeration
etc. The framework may be extensible so that if a library often uses a
particular custom type, the plugin integrating the library could add a
custom widget for it.

Advanced parameters: some modules may allow users set lots of
parameters. Although this is handy, many times the default values work
well and only few basic parameters are changed regularly. It should be
possible to mark some parameters as advanced and they would be hidden
by default, they would show up after clicking a checkbox or a button.
For example, when creating a vector buffer, the distance would be a
basic parameter while the number of segments per quadrant would be
advanced parameter.

Categorization: currently we are starting to face some trouble with
organization of plugins and the "modules" they provide. We thought
about creating fixed categories as in GRASS (e.g. Raster, Vector,
Database) where the modules would be placed. We also considered custom
categories and subcategories. The general problem with categories is
that many modules do not suit them well if they are not coming from a
well-organized library such as grass, saga or otb.
The conclusion that I recall from the discussion at hackfest is that
tagging is probably the only way to go: each module could have several
tags assigned to it, then the list of modules would be a tree with top
level items containing tags and modules would be shown inside these
tags. Bigger libraries requiring deeper hierarchy might come with
hierarchical tags like "SAGA/Grid - Tools". The names of tags will be
left for developers' creativity.
This list of processing modules should later get some advanced
functionality like searching, favourite modules or even attaching most
used modules on a toolbar, but that is something for the future, we
are not expecting you to do that during the summer :-) A simple tree
listing tags and modules assigned to them will suffice.

Formats and format conversion: for import and export of map layers the
framework should support any layer loaded (or loadable) by QGIS. In
case the library uses a different format for input/output, it should
take care of import/export.

Running modules from scripts: you probably have that in mind but let's
keep it explicit: the framework should provide API that would allow
user to run modules from python console or from a plugin, QGIS would
basically act just as a default GUI using that API. This will finally
allow users to run the processing modules in batches - poor man's
pipelining :-)

Feedback: many modules are able to return some feedback - either in
the form of a progress to indicate how long the process will take - or
to return some messages that will be shown to user. Support for
cancelling a running module would be helpful too, but not really
necessary right now.



Regards
Martin