[Qgis-user] Using Processing for nightly ETL jobs

[Bernhard Ströbl]

Hello everybody,

I did some experiments in the last couple days and your help was very valuable. I succeeded in creating two models that - when run within QGIS or from the console - download data of different types and write it into a PostgreSQL database. For automation purposes I want to run my models on a Linux server. I am wondering how I would configure the database access. The algorythm "Export to PostgreSQL" takes the connection name as input. The connection name is the name of a pg-connection configured using the GUI (and backed by a saved authentication in this case). There will be no GUI (and probably no graphic server either) on the server machine. Does anybody know if creating a profile with the connection (without authentication) and placing that as default profile in the home directory of the user running the model will do the trick? Or would I need a different approach?
I cannot try it out because currently I still have no access to the server.

Any help appreciated
Bernhard


> Germán Carrillo <carrillo.german at gmail.com> hat am 22.05.2024 23:30 CEST geschrieben:
> 
>  
> Hi All,
> 
> have a look at:
> 
> https://github.com/gacarrillor/AppendFeaturesToLayer
> 
> It has ready-to-use models to perform transform and upsert (update or
> insert) operations.
> 
> You can run it via standalone processing (Python) scripts or via QGIS
> Process.
> 
> Regards,
> 
> Germán
> 
> 
> El mié, 22 may 2024 a las 12:20, Etienne Trimaille via QGIS-User (<
> qgis-user at lists.osgeo.org>) escribió:
> 
> > Hi,
> >
> > https://docs.qgis.org/3.34/en/docs/user_manual/processing/standalone.html
> >
> > Le mer. 22 mai 2024 à 18:52, NOSPAM via QGIS-User <
> > qgis-user at lists.osgeo.org> a écrit :
> >
> >> Dear List,
> >>
> >> has anybody already used QGIS' processing framework for performing
> >> nightly ETL jobs?
> >>
> >> I need to have nightly jobs run that
> >> 1) download data (e.g. from a WFS server),
> >> 2) perform some tasks like renaming fields or the like
> >> 3) and finally load them into a PostGIS database.
> >>
> >> My idea would be to design the process using the modeller. How can I
> >> have the thus created model run non interactively?
> >>
> >> Any ideas are welcome
> >>
> >> Bernhard
> >> _______________________________________________
> >> QGIS-User mailing list
> >> QGIS-User at lists.osgeo.org
> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >>
> > _______________________________________________
> > QGIS-User mailing list
> > QGIS-User at lists.osgeo.org
> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> >
> 
> 
> -- 
> -----------
>    |\__
> (:>__)(
>    |/
> Soluciones Geoinformáticas Libres
> http://geotux.tuxfamily.org/
> https://twitter.com/GeoTux2 <http://twitter.com/GeoTux2>