[QGIS-Developer] FME to QGIS functionality mapping

[Denis Rouzaud]

Hi all,

We are currently evaluating work to improve QGIS processing to a closer
experience to FME.
There is already some information here from Nyall.
Our memories are a bit loose, is there any other work done on requirements
or a wishlist?

Is there any work already started so we don't hijack?

Thanks a lot,
Denis

Le mar. 7 avr. 2020 à 12:22, Nyall Dawson <nyall.dawson at gmail.com> a écrit :

> On Tue, 7 Apr 2020 at 17:35, Andreas Neumann <a.neumann at carto.net> wrote:
> >
> > Hi Nyall,
> >
> > Thank you for sharing this interesting document.
> >
> > I have questions and remarks:
> >
> > 1. One fundamental difference between FME and QGIS modeler, as far as I
> > understand it, is the fact that FME data sources are much more tightly
> > coupled with the model than in QGIS modeler. This has the advantage,
> > that the data readers, transformers and writers always know what
> > geometry types and attributes they have. The disadvantage is that it is
> > harder to exchange data sources with sources that have other attributes.
> > In my experience the FME approach is what most users prefer. They like
> > the fact that they can always see at each transformer what attributes
> > are available at this stage. Would you see an improvement - perhaps
> > another modeler mode - where also QGIS modeler would be more tightly
> > coupled with the data sources and that the model and algorithms would
> > always know at any step what attributes are available at this stage? Or
> > is this a fundamental difference between the two approaches that
> > couldn't be bridged?
>
> I think it's totally possible -- we just need to know how each
> algorithm transforms its inputs into an output data set in advance
> (i.e. the crs, wkb type, and fields).  API which permits this was
> recently added, but it's not yet hooked into the modeler.  There's
> still considerable work to do for that, but it's far from impossible!
>
> > 2. One aspect where FME shines is the ability to graphically debug
> > models. One can at every step in the model get the view (map and table)
> > of the intermediate step. One can also see at each branch in the model
> > how many features take this route and how many the other route. I know
> > this doesn't have anything to do with the transformer list you shared,
> > but this an aspect that I think, many users like about FME.
>
> A recent improvement is that after running a model in the designer,
> you can now see the actual values of inputs and outputs which were
> used for each child algorithm in the model. It's a partial step
> towards the FME approach. To get the rest of the way we'd need:
> 1. to ensure that every algorithm has numeric outputs for the number
> of features processed
> 2. ideally to allow the model to run on a background thread inside the
> designer, so that you'd see these input/output values populate
> instantly as the model runs (would also allow us to color-code child
> algorithms by state, e.g. completed/running/pending/skipped).
>
> 1. is pretty straightforward, just a lot of grunt work. 2. is more
> complex (and I suspect only achievable in builds based on qt 5.10+
> (since we'd need to invoke the child algorithm prepare steps on the
> main thread)... but its probably time to bump the minimum anyway).
>
> Anyway, definitely achievable in the near future.
>
> > I agree that there would a lot of "low hanging" fruits and I'm looking
> > forward to more feature parity between the two. And to the discussion
> > around it.
>
> Me too! When 3.14 is released the model capabilities in QGIS will be
> much much improved vs previous releases. Just the fact that **every**
> parameter for child algorithms (including input and output layers) can
> now be defined using qgis expressions opens up a WHOLE new world of
> possibilities alone...
>
> Nyall
>
>
>
> >
> > Greetings,
> >
> > Andreas
> >
> > Am 07.04.20 um 03:27 schrieb Nyall Dawson:
> > > Hi list,
> > >
> > > Just wanted to give a heads up on a document I've recently completed,
> > > which maps existing FME functionality across to the equivalent QGIS
> > > Processing or Expression equivalent:
> > >
> > >
> https://github.com/nyalldawson/qgis-processing-hitlist/blob/master/fme.md
> > >
> > > Since FME transformers are both algorithm-like and expression-like
> > > functionality, the document splits the QGIS equivalents into columns
> > > for both algorithms and expressions. Wherever the transformer
> > > functionality doesn't apply, a "N/A" is entered in the corresponding
> > > column (e.g. the "StringCaseChanger" transformer doesn't make sense
> > > the implement as a QGIS Processing algorithm -- it's equivalent is
> > > instead the expression functions "lower", "upper" and "title". And
> > > conversely the "DuplicateFilter" transformer maps to the Processing
> > > algorithms "Delete Duplicate Geometries" and "Delete duplicates by
> > > attribute", but doesn't make sense to have an expression function
> > > equivalent!)
> > >
> > > There's also Transformers in FME which really don't need an equivalent
> > > in QGIS -- e.g. those relating to styling features.
> > >
> > > Overall we have a good level of coverage of the most important
> > > functionality. There's a LOT of low-hanging fruit which could easily
> > > be addressed in small PRs (e.g. things like the
> > > "CharacterCodeExtractor" transformer would be a trivial task to
> > > implement as a new QGIS expression function ... a GREAT task for
> > > someone getting started with QGIS development!).
> > >
> > > (and just for completeness - QGIS offers hundreds of processing
> > > algorithms and expression functionality which **doesn't** have
> > > equivalents in FME. I suspect the reverse document would have just as
> > > many "missing" cells!)
> > >
> > > Feedback welcome! At some stage I think it would be nice to move this
> > > document across to the QGIS repo, and possibly pull into the QGIS
> > > documentation itself (along with similar documents for Arc/...
> > > functionality!)
> > >
> > > Nyall
> > > _______________________________________________
> > > QGIS-Developer mailing list
> > > QGIS-Developer at lists.osgeo.org
> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > _______________________________________________
> > QGIS-Developer mailing list
> > QGIS-Developer at lists.osgeo.org
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20230612/d669fec0/attachment.htm>