[QGIS-Developer] How to deal with QGIS plugins which install additional packages

Ethan Snyder Ethan.Snyder at rve.com
Mon Oct 28 06:10:48 PDT 2024 

I have a possible idea for this problem. Since QGIS relies heavily on Python, it would be beneficial to integrate pip (and conda for conda builds) into QGIS. Maybe add a pip/conda section in the Plugin Manager so that people can easily install extra python packages? Now with this system, a plugin can be written which depends on a python package. And with the plugin would add metadata like the qgis-plugin-dev-tools toml file to specify library dependencies/requirements. In the plugins repo, the people reviewing the plugin would vet the list of required python packages to make sure it’s not requiring anything malicious (this replaces the need to maintain a list of “acceptable” packages). When a user goes to install a plugin that has python dependencies, they will be notified (aside from that information being presented in the plugin info) about additional dependencies which QGIS will automatically install for the user (if the user accepts).

-Ethan

From: Joona Laine <joona.p.laine at gmail.com>
Sent: Wednesday, October 23, 2024 8:10 AM
To: Matthias Kuhn <matthias at opengis.ch>
Cc: John Stevenson - BGS <jostev at bgs.ac.uk>; info at opengis.it; qgis-developer <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages


Qgis-plugin-dev-tools approach solves this problem by vendoring the packages and rewriting <https://github.com/nlsfi/qgis-plugin-dev-tools/blob/2df5c099c9c86700e0d323c67243902f1df46fce/src/qgis_plugin_dev_tools/build/rewrite_imports.py#L10> the imports so that "import module.x.y" imports are rewritten in a vendored format: "import something._vendor.module.x.y". Thus multiple plugins can have different versions of packages since they all import their own vendored versions.


Joona

ke 23. lokak. 2024 klo 14.58 Matthias Kuhn <matthias at opengis.ch<mailto:matthias at opengis.ch>> kirjoitti:
Hi,

This approach will work fine within limitations, as soon as multiple plugins ship the same library things become risky as there is no isolation between libraries.
For python libraries, this may be caused by singletons being used and for native libraries (as in this example), it's easy to cause crashes by multiple versions of the same library exporting the same symbols being loaded in parallel.
That being said: it will work fine in many cases, but I wouldn't promote this as "best practice". After all, python invented virtualenvs for good reasons -- each process will always run one environment (potentially composed of multiple cascading virtual envs, but never multiple "parallel" envs).

Cheers
Matthias

On Wed, Oct 23, 2024 at 1:31 PM John Stevenson - BGS via QGIS-Developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>> wrote:
Hi,

Mergin Maps plugin also packages the dependencies (including the geodiff binary) into the plugin itself.  I’m not sure how it handles cross-platform differences, though.

Plugin:
https://plugins.qgis.org/plugins/Mergin/#plugin-details

GitHub Actions code:

https://github.com/MerginMaps/qgis-plugin/blob/ef0b2502ddb4bcbc1670b0d82832e93b658c18b2/.github/workflows/packages.yml#L116

Cheers,
John
From: QGIS-Developer <qgis-developer-bounces at lists.osgeo.org<mailto:qgis-developer-bounces at lists.osgeo.org>> On Behalf Of Joona Laine via QGIS-Developer
Sent: 23 October 2024 10:58
To: info at opengis.it<mailto:info at opengis.it>
Cc: qgis-developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>>
Subject: Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages


One alternative way of managing the dependencies is to package the non-binary runtime dependencies (including licenses) with the plugin. This also tackles the problem with different versions of the same requirements between multiple plugins. There is a tool for that https://github.com/nlsfi/qgis-plugin-dev-tools which also has many more useful features for developing QGIS plugins.


One example of plugins using this tool is pickLayer (https://plugins.qgis.org/plugins/pickLayer/) which bundles https://github.com/GispoCoding/qgis_plugin_tools with it.


What do you think about this approach?


Regards,

Joona

ke 23. lokak. 2024 klo 12.01 Info O.GIS via QGIS-Developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>> kirjoitti:
I also did a similar thing in qgis2web plugin.
I explained to the user that he can install qtwebengine to get the latest features and to do so he will have to click on a button that indicates that an installation will start.
Here is the screen:


Could it be okay?

The code:

try:
        if system == 'Windows':
            pip_exec = os.path.join(sysconfig.get_path("scripts"), "pip3")
            env = os.environ.copy()
            if full_proxy_url:
                env['http_proxy'] = full_proxy_url
                env['https_proxy'] = full_proxy_url
            subprocess.check_call([pip_exec, "install", "--upgrade", "PyQtWebEngine==5.15.6"], env=env)
        elif system == 'Linux':
            subprocess.check_call(["sudo", "apt-get", "install", "python3-pyqt5.qtwebengine"])
        elif system == 'Darwin':  # macOS
            subprocess.check_call(["brew", "install", "pyqt5"])


Andrea Ordonselli
O.GIS - opengis.it<http://opengis.it>

Da "QGIS-Developer" qgis-developer-bounces at lists.osgeo.org<mailto:qgis-developer-bounces at lists.osgeo.org>
A "Matthias Kuhn" matthias at opengis.ch<mailto:matthias at opengis.ch>
Cc "Thomas B via QGIS-Developer" qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>
Data Wed, 23 Oct 2024 16:16:43 +1000
Oggetto Re: [QGIS-Developer] How to deal with QGIS plugins which install additional packages


On Wed, 23 Oct 2024, 4:07 pm Matthias Kuhn, <matthias at opengis.ch<mailto:matthias at opengis.ch>> wrote:
On Wed, Oct 23, 2024 at 2:49 AM Nyall Dawson via QGIS-Developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>> wrote:

On Wed, 23 Oct 2024, 9:20 am Greg Troxel via QGIS-Developer, <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>> wrote:
Thomas B via QGIS-Developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>> writes:

> Dear QGIS-Developers,
>
> Are there any guidelines from the QGIS project regarding whether a QGIS
> plugin is allowed to autonomously install required packages using PIP or
> similar tools without manual installation by the user?
>
> While this might seem convenient, I see it as a potential security risk,
> especially if the user is not explicitly informed about what is happening
> in the background.

Agreed this is not ok.  I think a plugin downloading anything to be
executed or interpreted should be entirely prohibited.

+1 . This practice should lead to a plugin being removed from the repositories.

(Possibly we could do something on the code side too, eg by monkey patching over subprocess/etc and explicitly blocking execution of sip, with a developer-friendly exception stating this policy. It'd be easy for someone motivated to circumvent, but could at least be used to advise plugin developers that this is not acceptable practice...)

We've tried to come up with a more transparent approach with support for requirements.txt (see https://github.com/opengisch/qpip). It is using pip but with a frontend which informs the user and lets him confirm an eventual installation.
Is this approach generally acceptable?

Well, I definitely trust yourself/OpenGIS significantly more then other random plugin developers 👍

I would personally feel safest if this was something officially endorsed, with an explicit allow list of acceptable packages.



Nyall



Matthias


Nyall

_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
DISCLAIMER: This message and any documents attached may contain confidential information and are intended only for the individual(s) named. If you are not the intended recipient, or the employee or agent authorized to received for the intended recipient, you should not disseminate, distribute or copy this e-mail and any attached documents. If you have received this e-mail in error, please immediately notify the sender at Remington & Vernick Engineers by replying to this e-mail and delete the original e-mail and any reply e-mail messages from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20241028/72ed4543/attachment-0001.htm>

More information about the QGIS-Developer mailing list