[Qgis-developer] Processing NG/V2 - brainstorming

[Matthias Kuhn]

Hi Richard,

On 12/06/2016 11:56 AM, Richard Duivenvoorde wrote:
> On 05-12-16 10:03, Nyall Dawson wrote:
>> Hi all,
> 
> Hi Nyall et al. Good thinking, just wondering... what is NG/V2? :-)

Next Generation, Version 2

Add to that:

X / 2.0 / reloaded ;)

> 
>> So here we go... a bunch of random ideas on future processing
>> enhancements:
>>
>> 1. Rework native algorithms to avoid layer input/outputs
> 
> I have seen some people creating huge FME workbenches, and what I see is
> that these models are almost never 'linear'.. that is: never work in
> 'efficient pipes'. But often more like: we grab some data from here,
> from there, combine, do something on it, calculate something from the
> whole, reproject and rewrite some attributes and output it to ... a db.
> 
> I mean to say: some kind of intermediate disk-format is needed anyway?

Yes, that's true. Sometimes. The idea is to skip it where possible and
to apply it where not possible. Transparently.

> What I have heard from 'the others' is that they use some binary
> (spatially indexed?) file format? Would that be an option for us?
> Looking to the python world where you can 'pickle' arbitrary objects to
> disk. Is there something like that available in cpp?

I think the order of precedence would be (always depending on the
involved algorithms)

1. Manual user configuration
2. Iterator
3. GeoPackage
4. Shapefile

>> 3. Porting components of processing to core
>>
>> There's demand (from eg QField) to reuse parts of processing outside
>> of PyQGIS.
>>
>> I think good candidates for porting to core would be:
>> - parameters
>> - inputs
>> - the algorithm base class
> 
> This all means a stronger api isn't it? To be implemented in whatever
> language you prefer? That is always good isn't it :-)

Exactly what I was thinking :-)

Matthias