[QGIS-Developer] Thoughts on using processing models for ETL tasks

[Régis Haubourg]

Hi John,
Thanks for sharing your feedback.

I must say I agree with your points, and have some others concerning
database i/o options or a graphical version of refactor field alg.

All this would very nice fit into a QEP so that we can centralize thoughts
and requirements in a more collaborative way than redmine issues.

We should even go a bit further and lead a dedicated group of interest -
like the one Tim is leading for metadata.

However, this is a big task and we'll need a way to leverage ressources,
time and money for leading this.

Regards

Régis

2018-05-28 12:11 GMT+02:00 johnrobot <johnrobot at gmail.com>:

> Hi
> I have started looking at using the processing model builder in QGIS 3.0.3
> for ETL tasks. A few initial thoughts:
>
> - I would like to be able to resize resize model components, for example to
> be able to see the full name of a component.
> - Add support for copy/paste of components. If I have spent a few minutes
> or
> more on configuring a algorithm and need an additional copy of it in the
> model, copy/paste of the component would save me time.
> - Add support for selecting multiple model components by drawing a box.
> Also, Ctrl + A should select all components.
> - Improved support for undo/redo.
> - Add support for grouping components. Different parts of the model might
> might have different focus, such as transforming geometries, fixing
> attributes or doing calculations. I think it could help usability if I
> could
> group (in a visual box?) or colour the components accordingly.
> - In the tool "Fix geometries", I can´t specify the type of errors to look
> for and fix. That would help me understand the errors better.
> - I would like be able to add a new algorithm to the model without having
> to
> connect it to the other components directly. I would like to be able to add
> an algorith for later consideration.
> - Continuing on the previous suggestion, I would like to be able to connect
> an unconnected algorithm (already added to the model) using drag and drop.
> - I would like to see the number of features flowing through each
> connection. That would help me understand the flow and potential errors
> better.
> - When creating a new model, I often need to run a model quite a few times
> to test it. Each time, I have to select input layers manually when starting
> the model. Can I make QGIS remember the files I picked the last time?
> - If possible, I would like to see OpenCL/CUDA support for algorithms. I
> expect some of my tasks to be rather heavy (hours to days) and using the
> GPU
> could potentially speed up the processing a lot.
> - For some jobs, I want to zip the model output directly in the model. In
> these cases, I might also want to add external files, such as PDF
> documentation, to the zip archive. Support for that would be very useful in
> corporate environments such as ours.
>
> Should I open a separate ticket for each issue?
>
> Regards,
>
> Magnus
>
>
>
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-
> f4099106.html
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20180528/c6251473/attachment.html>