[QGIS-Developer] How to deal with QGIS plugins which install additional packages

Matthias Kuhn matthias at opengis.ch
Tue Oct 29 23:22:46 PDT 2024 

Hi Ethan,

The qpip tool does more or less what you are describing
https://github.com/opengisch/qpip
We are looking for more testing at the moment, if we get good feedback for
it, a core adoption can be discussed.

Bests
Matthias

On Mon, Oct 28, 2024 at 2:11 PM Ethan Snyder via QGIS-Developer <
qgis-developer at lists.osgeo.org> wrote:

> I have a possible idea for this problem. Since QGIS relies heavily on
> Python, it would be beneficial to integrate pip (and conda for conda
> builds) into QGIS. Maybe add a pip/conda section in the Plugin Manager so
> that people can easily install extra python packages? Now with this system,
> a plugin can be written which depends on a python package. And with the
> plugin would add metadata like the qgis-plugin-dev-tools toml file to
> specify library dependencies/requirements. In the plugins repo, the people
> reviewing the plugin would vet the list of required python packages to make
> sure it’s not requiring anything malicious (this replaces the need to
> maintain a list of “acceptable” packages). When a user goes to install a
> plugin that has python dependencies, they will be notified (aside from that
> information being presented in the plugin info) about additional
> dependencies which QGIS will automatically install for the user (if the
> user accepts).
>
>
>
> -Ethan
>
>
>
> *From:* Joona Laine <joona.p.laine at gmail.com>
> *Sent:* Wednesday, October 23, 2024 8:10 AM
> *To:* Matthias Kuhn <matthias at opengis.ch>
> *Cc:* John Stevenson - BGS <jostev at bgs.ac.uk>; info at opengis.it;
> qgis-developer <qgis-developer at lists.osgeo.org>
> *Subject:* Re: [QGIS-Developer] How to deal with QGIS plugins which
> install additional packages
>
>
>
> Qgis-plugin-dev-tools approach solves this problem by vendoring the
> packages and rewriting
> <https://github.com/nlsfi/qgis-plugin-dev-tools/blob/2df5c099c9c86700e0d323c67243902f1df46fce/src/qgis_plugin_dev_tools/build/rewrite_imports.py#L10>the
> imports so that "import module.x.y" imports are rewritten in a vendored
> format: "import something._vendor.module.x.y". Thus multiple plugins can
> have different versions of packages since they all import their own
> vendored versions.
>
>
>
> Joona
>
>
>
> ke 23. lokak. 2024 klo 14.58 Matthias Kuhn <matthias at opengis.ch>
> kirjoitti:
>
> Hi,
>
>
>
> This approach will work fine within limitations, as soon as multiple
> plugins ship the same library things become risky as there is no isolation
> between libraries.
>
> For python libraries, this may be caused by singletons being used and for
> native libraries (as in this example), it's easy to cause crashes by
> multiple versions of the same library exporting the same symbols being
> loaded in parallel.
>
> That being said: it will work fine in many cases, but I wouldn't promote
> this as "best practice". After all, python invented virtualenvs for good
> reasons -- each process will always run one environment (potentially
> composed of multiple cascading virtual envs, but never multiple "parallel"
> envs).
>
>
>
> Cheers
>
> Matthias
>
>
>
> On Wed, Oct 23, 2024 at 1:31 PM John Stevenson - BGS via QGIS-Developer <
> qgis-developer at lists.osgeo.org> wrote:
>
> Hi,
>
> Mergin Maps plugin also packages the dependencies (including the geodiff
> binary) into the plugin itself.  I’m not sure how it handles cross-platform
> differences, though.
>
>
> Plugin:
>
> https://plugins.qgis.org/plugins/Mergin/#plugin-details
>
>
>
> GitHub Actions code:
>
>
>
>
> https://github.com/MerginMaps/qgis-plugin/blob/ef0b2502ddb4bcbc1670b0d82832e93b658c18b2/.github/workflows/packages.yml#L116
>
>
> Cheers,
> John
>
> *From:* QGIS-Developer <qgis-developer-bounces at lists.osgeo.org> *On
> Behalf Of *Joona Laine via QGIS-Developer
> *Sent:* 23 October 2024 10:58
> *To:* info at opengis.it
> *Cc:* qgis-developer <qgis-developer at lists.osgeo.org>
> *Subject:* Re: [QGIS-Developer] How to deal with QGIS plugins which
> install additional packages
>
>
>
> One alternative way of managing the dependencies is to package the
> non-binary runtime dependencies (including licenses) with the plugin. This
> also tackles the problem with different versions of the same requirements
> between multiple plugins. There is a tool for that
> https://github.com/nlsfi/qgis-plugin-dev-tools which also has many more
> useful features for developing QGIS plugins.
>
>
>
> One example of plugins using this tool is pickLayer (
> https://plugins.qgis.org/plugins/pickLayer/) which bundles
> https://github.com/GispoCoding/qgis_plugin_tools with it.
>
>
>
> What do you think about this approach?
>
>
>
> Regards,
>
> Joona
>
>
>
> ke 23. lokak. 2024 klo 12.01 Info O.GIS via QGIS-Developer <
> qgis-developer at lists.osgeo.org> kirjoitti:
>
> I also did a similar thing in qgis2web plugin.
>
> I explained to the user that he can install qtwebengine to get the latest
> features and to do so he will have to click on a button that indicates that
> an installation will start.
>
> Here is the screen:
>
>
>
>
>
> Could it be okay?
>
>
>
> The code:
>
>
>
> *try:*
>
> *        if system == 'Windows':*
>
> *            pip_exec = os.path.join(sysconfig.get_path("scripts"),
> "pip3")*
>
> *            env = os.environ.copy()*
>
> *            if full_proxy_url:*
>
> *                env['http_proxy'] = full_proxy_url*
>
> *                env['https_proxy'] = full_proxy_url*
>
> *            subprocess.check_call([pip_exec, "install", "--upgrade",
> "PyQtWebEngine==5.15.6"], env=env)*
>
> *        elif system == 'Linux':*
>
> *            subprocess.check_call(["sudo", "apt-get", "install",
> "python3-pyqt5.qtwebengine"])*
>
> *        elif system == 'Darwin':  # macOS*
>
> *            subprocess.check_call(["brew", "install", "pyqt5"])*
>
>
>
>
>
> *Andrea Ordonselli*
>
> *O.GIS - opengis.it <http://opengis.it>*
>
>
>
> Da "QGIS-Developer" qgis-developer-bounces at lists.osgeo.org
>
> A "Matthias Kuhn" matthias at opengis.ch
>
> Cc "Thomas B via QGIS-Developer" qgis-developer at lists.osgeo.org
>
> Data Wed, 23 Oct 2024 16:16:43 +1000
>
> Oggetto Re: [QGIS-Developer] How to deal with QGIS plugins which install
> additional packages
>
>
>
>
>
> On Wed, 23 Oct 2024, 4:07 pm Matthias Kuhn, <matthias at opengis.ch> wrote:
>
> On Wed, Oct 23, 2024 at 2:49 AM Nyall Dawson via QGIS-Developer <
> qgis-developer at lists.osgeo.org> wrote:
>
>
>
> On Wed, 23 Oct 2024, 9:20 am Greg Troxel via QGIS-Developer, <
> qgis-developer at lists.osgeo.org> wrote:
>
> Thomas B via QGIS-Developer <qgis-developer at lists.osgeo.org> writes:
>
> > Dear QGIS-Developers,
> >
> > Are there any guidelines from the QGIS project regarding whether a QGIS
> > plugin is allowed to autonomously install required packages using PIP or
> > similar tools without manual installation by the user?
> >
> > While this might seem convenient, I see it as a potential security risk,
> > especially if the user is not explicitly informed about what is happening
> > in the background.
>
> Agreed this is not ok.  I think a plugin downloading anything to be
> executed or interpreted should be entirely prohibited.
>
>
>
> +1 . This practice should lead to a plugin being removed from the
> repositories.
>
>
>
> (Possibly we could do something on the code side too, eg by monkey
> patching over subprocess/etc and explicitly blocking execution of sip, with
> a developer-friendly exception stating this policy. It'd be easy for
> someone motivated to circumvent, but could at least be used to advise
> plugin developers that this is not acceptable practice...)
>
>
>
> We've tried to come up with a more transparent approach with support for
> requirements.txt (see https://github.com/opengisch/qpip). It is using pip
> but with a frontend which informs the user and lets him confirm an eventual
> installation.
>
> Is this approach generally acceptable?
>
>
>
> Well, I definitely trust yourself/OpenGIS significantly more then other
> random plugin developers 👍
>
>
>
> I would personally feel safest if this was something officially endorsed,
> with an explicit allow list of acceptable packages.
>
>
>
>
>
>
>
> Nyall
>
>
>
>
>
>
>
> Matthias
>
>
>
>
>
> Nyall
>
>
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> This email and any attachments are intended solely for the use of the
> named recipients. If you are not the intended recipient you must not use,
> disclose, copy or distribute this email or any of its attachments and
> should notify the sender immediately and delete this email from your
> system. UK Research and Innovation (UKRI) has taken every reasonable
> precaution to minimise risk of this email or any attachments containing
> viruses or malware but the recipient should carry out its own virus and
> malware checks before opening the attachments. UKRI does not accept any
> liability for any losses or damages which the recipient may sustain due to
> presence of any viruses.
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
> DISCLAIMER: This message and any documents attached may contain
> confidential information and are intended only for the individual(s) named.
> If you are not the intended recipient, or the employee or agent authorized
> to received for the intended recipient, you should not disseminate,
> distribute or copy this e-mail and any attached documents. If you have
> received this e-mail in error, please immediately notify the sender at
> Remington & Vernick Engineers by replying to this e-mail and delete the
> original e-mail and any reply e-mail messages from your system. E-mail
> transmission cannot be guaranteed to be secure or error-free as information
> could be intercepted, corrupted, lost, destroyed, arrive late or
> incomplete, or contain viruses. The sender therefore does not accept
> liability for any errors or omissions in the contents of this message,
> which arise as a result of e-mail transmission. If verification is required
> please request a hard-copy version. Thank you.
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20241030/003e307c/attachment-0001.htm>

More information about the QGIS-Developer mailing list