[QGIS-Developer] FME to QGIS functionality mapping

Mon Jun 12 01:32:37 PDT 2023

Hi there,
That's good new, especially given the new policy of SAFE software on
licence costs.

On my view, here is the wishlist :

- as stated before, be aware of the datasource structure in a model, with
the ability to refresh it, or read it from a config ressources (file,
database etc..)
- have the magic graphical field autobinder widget. It is a massive
productivity gain having this. Autobind / show un-mapped fields with a
color indicator , replace the match by an expression is 90% of what I used
- have an internal model that is less strict than pure GIS ressources. If
you want to be able to parse JSON / GIS / OSM tags / multigeométry
objectfs, we probably need an internal abstraction that is less constrained
than current WKT + typed table we have currently. I'm not sure if plans are
to go this way, but this is a major feature of FME (except multigeometries
that are not handled)
- Be able to store a cache of object to debug transformers.
- A very strong, clear and readable log. Currently most modeler errors are
just not giving userful informations.
- A graphical tool to drag and drop elements of the modele and enable /
disable them.

All the best !

Le lun. 12 juin 2023 à 10:17, Denis Rouzaud via QGIS-Developer <
qgis-developer at lists.osgeo.org> a écrit :

> Hi all,
>
> We are currently evaluating work to improve QGIS processing to a closer
> experience to FME.
> There is already some information here from Nyall.
> Our memories are a bit loose, is there any other work done on requirements
> or a wishlist?
>
> Is there any work already started so we don't hijack?
>
> Thanks a lot,
> Denis
>
> Le mar. 7 avr. 2020 à 12:22, Nyall Dawson <nyall.dawson at gmail.com> a
> écrit :
>
>> On Tue, 7 Apr 2020 at 17:35, Andreas Neumann <a.neumann at carto.net> wrote:
>> >
>> > Hi Nyall,
>> >
>> > Thank you for sharing this interesting document.
>> >
>> > I have questions and remarks:
>> >
>> > 1. One fundamental difference between FME and QGIS modeler, as far as I
>> > understand it, is the fact that FME data sources are much more tightly
>> > coupled with the model than in QGIS modeler. This has the advantage,
>> > that the data readers, transformers and writers always know what
>> > geometry types and attributes they have. The disadvantage is that it is
>> > harder to exchange data sources with sources that have other attributes.
>> > In my experience the FME approach is what most users prefer. They like
>> > the fact that they can always see at each transformer what attributes
>> > are available at this stage. Would you see an improvement - perhaps
>> > another modeler mode - where also QGIS modeler would be more tightly
>> > coupled with the data sources and that the model and algorithms would
>> > always know at any step what attributes are available at this stage? Or
>> > is this a fundamental difference between the two approaches that
>> > couldn't be bridged?
>>
>> I think it's totally possible -- we just need to know how each
>> algorithm transforms its inputs into an output data set in advance
>> (i.e. the crs, wkb type, and fields).  API which permits this was
>> recently added, but it's not yet hooked into the modeler.  There's
>> still considerable work to do for that, but it's far from impossible!
>>
>> > 2. One aspect where FME shines is the ability to graphically debug
>> > models. One can at every step in the model get the view (map and table)
>> > of the intermediate step. One can also see at each branch in the model
>> > how many features take this route and how many the other route. I know
>> > this doesn't have anything to do with the transformer list you shared,
>> > but this an aspect that I think, many users like about FME.
>>
>> A recent improvement is that after running a model in the designer,
>> you can now see the actual values of inputs and outputs which were
>> used for each child algorithm in the model. It's a partial step
>> towards the FME approach. To get the rest of the way we'd need:
>> 1. to ensure that every algorithm has numeric outputs for the number
>> of features processed
>> 2. ideally to allow the model to run on a background thread inside the
>> designer, so that you'd see these input/output values populate
>> instantly as the model runs (would also allow us to color-code child
>> algorithms by state, e.g. completed/running/pending/skipped).
>>
>> 1. is pretty straightforward, just a lot of grunt work. 2. is more
>> complex (and I suspect only achievable in builds based on qt 5.10+
>> (since we'd need to invoke the child algorithm prepare steps on the
>> main thread)... but its probably time to bump the minimum anyway).
>>
>> Anyway, definitely achievable in the near future.
>>
>> > I agree that there would a lot of "low hanging" fruits and I'm looking
>> > forward to more feature parity between the two. And to the discussion
>> > around it.
>>
>> Me too! When 3.14 is released the model capabilities in QGIS will be
>> much much improved vs previous releases. Just the fact that **every**
>> parameter for child algorithms (including input and output layers) can
>> now be defined using qgis expressions opens up a WHOLE new world of
>> possibilities alone...
>>
>> Nyall
>>
>>
>>
>> >
>> > Greetings,
>> >
>> > Andreas
>> >
>> > Am 07.04.20 um 03:27 schrieb Nyall Dawson:
>> > > Hi list,
>> > >
>> > > Just wanted to give a heads up on a document I've recently completed,
>> > > which maps existing FME functionality across to the equivalent QGIS
>> > > Processing or Expression equivalent:
>> > >
>> > >
>> https://github.com/nyalldawson/qgis-processing-hitlist/blob/master/fme.md
>> > >
>> > > Since FME transformers are both algorithm-like and expression-like
>> > > functionality, the document splits the QGIS equivalents into columns
>> > > for both algorithms and expressions. Wherever the transformer
>> > > functionality doesn't apply, a "N/A" is entered in the corresponding
>> > > column (e.g. the "StringCaseChanger" transformer doesn't make sense
>> > > the implement as a QGIS Processing algorithm -- it's equivalent is
>> > > instead the expression functions "lower", "upper" and "title". And
>> > > conversely the "DuplicateFilter" transformer maps to the Processing
>> > > algorithms "Delete Duplicate Geometries" and "Delete duplicates by
>> > > attribute", but doesn't make sense to have an expression function
>> > > equivalent!)
>> > >
>> > > There's also Transformers in FME which really don't need an equivalent
>> > > in QGIS -- e.g. those relating to styling features.
>> > >
>> > > Overall we have a good level of coverage of the most important
>> > > functionality. There's a LOT of low-hanging fruit which could easily
>> > > be addressed in small PRs (e.g. things like the
>> > > "CharacterCodeExtractor" transformer would be a trivial task to
>> > > implement as a new QGIS expression function ... a GREAT task for
>> > > someone getting started with QGIS development!).
>> > >
>> > > (and just for completeness - QGIS offers hundreds of processing
>> > > algorithms and expression functionality which **doesn't** have
>> > > equivalents in FME. I suspect the reverse document would have just as
>> > > many "missing" cells!)
>> > >
>> > > Feedback welcome! At some stage I think it would be nice to move this
>> > > document across to the QGIS repo, and possibly pull into the QGIS
>> > > documentation itself (along with similar documents for Arc/...
>> > > functionality!)
>> > >
>> > > Nyall
>> > > _______________________________________________
>> > > QGIS-Developer mailing list
>> > > QGIS-Developer at lists.osgeo.org
>> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20230612/0af7eed0/attachment-0001.htm>