[Qgis-developer] QGIS Processing Framework

[Julien Malik]

> Looks very good, and in principle similar to what I was doing (or
> trying to do) with SAGA, see github repo. But, from what I can tell,
> is very OTB-specific. I am not sure how we can integrate this into a
> common framework.

We will find a way...


> I feel in general QGIS should handle the GUI part.
> That makes the framework usable in non-GUI contexts and only dependant
> on QtCore (the non-QGIS side, I mean). In my test code I used a
> QFormLayout, populated with Widgets according to the type of the
> parameters alone. (screenshot linked from the "QGIS Processing
> Framework" page)

I should look into this QFormLayout...
I'm still a Qt newbie, and in no way a GUI specialist so input on the 
good classes to use is warmly welcome.


> I imagined the particular implementations using the existing Plugin
> mechanism and thus having access to QgisInterface anyway, if really
> necessary.

I'd like this framework to keep some of the freedom of the current QGis 
plugin system, where you can almost do what you want.
I feel this is how anything will become integrable in Qgis.


>> Trying to fit everyone in the same framework will be hard : either each
>> library (SAGA/GRASS/OTB) will only use a subset of the provided
>> functionnalities, either we will end up taking the common factor of all libs
>> and everyone will feel limited. I'm worried than we fall into one of these
>> two pitfalls. Please do not impose to much constraints !
>>
>> Reactions very welcome on this point !
> IMO, it is this genericity that would enable the different
> implementations to talk to each other... inter-library pipelining
> being possible if the parameter (including I/O) types match.
>
> Compare the list of parameters for OTB and SAGA:
> https://github.com/polymeris/qgis/wiki/List-of-parameters
> I think a common ground can be found for most cases, while retaining
> the ability to specify implementation-specific types.
> That allows consistency in the user interface and easier scripting (no
> need to worry about how each implementation handles each data type).


 From my point of view, the Qgis common ground is all about :
- getting raster/vector layers as input, and producing raster/vector 
layer as output.
- having a Qt based interface using widgets which are as similar as 
possible when they represent similar concepts, which fit Qgis UI guidelines
- having Python interfaces

The rest seems implementation-specific to me.
Do we really want to add a bunch of other "Qgis data types" ?

To say it differently : Forget SAGA! Forget OTB! Think QGis!
It would be nice to have some inputs from the GRASS gurus for example, a 
software I don't know much.
I guess this is again another complete set of parameter types.



>> [2]
>> Chaining modules from different libraries seems a little ambitious to me,
>> but this would be awesome !
>> As you said Camilo, I think we can leave pipelining out as a first step.
>> We are not really focusing on that part now neither, though the current
>> implementation should allow this.
> I think chaining modules is possible if input/ouput formats are well
> defined. At least in a crude "execute module A, convert data, execute
> module B" way.

Yes !
At least I see that (easily?) possible for raster/vector layers I/O.



>> [3]
>> You seem to consider parameter validation only in an independent way
>>
>>  From our experience it is insufficient (for example, absolute default values
>> is not enough).
>> How we plan to solve this (trying to keep it simple also...), is to give our
>> modules 3 main entry points :
>> - description of the parameters
>> - a global method to update all the parameters, called as soon as one
>> parameter is changed
>> - an execution method
>> Let's imagine a simple orthorectification application with the following
>> params :
>> - input/output image
>> - the output CRS
>> - output origin, size, and spacing
>> We want the origin, size and spacing to be auto-computed each time the input
>> or CRS changes
>> If CRS changes from UTM to WGS84, we expect the outputorigin units/display
>> to also change from meter to degree, for example.
> Yes, very important point. One possibility would be to use signals to
> connect the framework to each parameter's validator (to change
> constraints) and to the module (to add/remove parameters, as per [4])
>
>> [4] (related to [3])
>> Our first working-and-usefull application in this new framework we're
>> building shows also dynamic parameters list in the GUI (see the screenshot
>> on the Wiki).
>> It is a smoothing application with 3 modes: Median, Gaussian, Anisotropic
>> Filtering.
>> Each mode has associated params. Median and Gaussian needs a radius,
>> Anisotropic Filtering needs a time step and a number of iterations.
>> So below the combobox, the widget displayed depend on the combobox value.
>>
>> [5]
>> The model we imagined for the parameter list will be a tree more than a flat
>> list.
>> A group of parameters is just one node of the tree, containing a parameters
>> list, where each parameter can also be a group of parameters.
>>
> Ok. A generic list type could be defined as parameter type. Perhaps
> implemented with QList<Parameter>  (and add that to the QVariant union,
> of course). Not to be confused with list-of-something parameters.
>
>> We would also want checkable parameters.
>> mandatory parameters should have a value.
>> non-mandatory parameters would have a checkbox to be activated.
> That shouldn't be too hard. I think it is ok to have the validator
> handle that, accepting empty values if the parameter is optional.
> It is a minor detail, but from a UI point of view, I wouldn't use
> checkboxes, because it demands extra clicks from the end-user, but
> rather use some icon and/or a bold typeface to identifiy mandatory
> parameters. This sort of thing would have to be checked with people
> more versed in UI design, though.

Yes. As said before I'm no GUI specialist, so opinion from the QGis GUI 
masters will be very valuable here.

> Agreed. Actually, from the framework's point of view, events and
> progress indicators and such can just be output parameters. And the
> same mechanism that is used to update input parameters (for
> validation, [3]) can be used to report back.
>
> Well. Thanks, Julien, for your comments. You have a very clear picture
> of the issues and limitations involved in this project and I really
> appreciate the time you took in sharing it with us.

No, be sure I don't have a clear picture in mind yet, I only have a wish 
list of what I'd like to do with the OTB modules in Qgis ;)

> Also, I think in general we agree. Except, as I said before, I would
> move as much of the GUI-related code from the implementation side to
> the QGIS side to make it more-scripting friendly and consistent.
>
> Of course, I am only starting with this and everything I said above
> may very well be wrong or irrelevant. Please correct/criticize.

I completely disagree ;)
You're giving very insightful comments here, thank you !

Best Regards,
Julien