[QGIS-Developer] QGIS plugin depends on pypi package

[John Lindsay]

Hi Julien,

Thank you for your detailed answer:

"If you're interested, let me know. It could motivate me to write a tutorial detailing this method."

Yes, I would be very interested in that tutorial if you managed to find the time.

Regards,

John



Prof. John Lindsay

Dept. of Geography, Environment & Geomatics,

The University of Guelph,

Room 122, Hutt Building,

E-mail: jlindsay at uoguelph.ca

Phone: (519) 824-4120 ext. 56074

Web: https://geg.uoguelph.ca/faculty/lindsay-john

Research page: https://jblindsay.github.io/ghrg/index.html

WhiteboxTools GIS project: https://www.whiteboxgeo.com/

Twitter: https://twitter.com/whiteboxgeo

________________________________
From: QGIS-Developer <qgis-developer-bounces at lists.osgeo.org> on behalf of Julien Moura via QGIS-Developer <qgis-developer at lists.osgeo.org>
Sent: Friday, January 19, 2024 3:49 PM
To: qgis-developer at lists.osgeo.org <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] QGIS plugin depends on pypi package


CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to IThelp at uoguelph.ca.


Hello John,


It's a recurring issue/question since QGIS does not support pip dependencies and that PyQGIS plugins are not following the official Python packaging standards from PEPs and related formats (typically metadata.txt instead of setup.cfg, pyproject.toml). You will find some unsuccessful QEPs about this. For example: https://github.com/qgis/QGIS-Enhancement-Proposals/issues/202.


Shortly, here come 3 workarounds:

  *   basic: run pip install in a rough subcommand. Not so elegant but still efficient.
  *   re-wheeling: implement a minimalist (or complete) 3rd party packages managers, miming pip through QgsNetwork tooling. Best example I know until today: ENMap-Box plugin (see attached screenshot or try it yourself).
  *   YAML addicts: run your pip install during your plugin's packaging job in CI and include external dependencies in a subfolder before deploying it to your favorite plugins repository. Then, manage install loading from your plugin with try/except ImportError and 'site' module. This is the way I prefer and use for years now. Main downside: for dependencies requiring a specific compilation on an operating system, you need to include various outputs. See Roof Classify<https://github.com/DINFO-UniFI/RoofClassify> (GitHub Workflow), Thyrsis<https://gitlab.com/Oslandia/qgis/thyrsis> (GitLab CI), QSoccer<https://gitlab.com/Oslandia/qgis/QSoccer/> (GitLab CI), QDuckDB<https://gitlab.com/Oslandia/qgis/qduckdb> (GitLab CI)... If you're interested, let me know. It could motivate me to write a tutorial detailing this method.
  *   bravehearts: working on packaging your dependencies into osgeo4w; but they probably won't be included in default QGIS packages (.msi)

Just a quick advice: if you do some pip install within your plugin, during the packaging or during the run, make it under a custom folder related only to your plugin and use site to load extra dependencies in the QGIS Python environment to avoid issues.


Good luck!

Regards

Julien


Le 19/01/2024 à 19:16, John Lindsay via QGIS-Developer a écrit :

Agreed, when I 'print(sys.executable)' from within the QGIS Python Console (on my MacBook Pro), I see:


/Applications/QGIS.app/Contents/MacOS/QGIS


But from my system terminal, I see:


/Library/Frameworks/Python.framework/Versions/3.11/bin/python3

So QGIS, at least on a Mac, does seem to have it's own version of Python installed. And there is not clear way, again on a Mac at least, to pip install to the QGIS Python such that a plugin will be able to see a package. Now, this said, on my Linux machine, installing my Whitebox Workflows plugin did not run into an error at all and so I'm left wondering if this is perhaps, as Greg suggested, a particular binary distribution issue related to QGIS on MacOS. Could be.

Regards,

John


Prof. John Lindsay

Dept. of Geography, Environment & Geomatics,

The University of Guelph,

Room 122, Hutt Building,

E-mail: jlindsay at uoguelph.ca<mailto:jlindsay at uoguelph.ca>

Phone: (519) 824-4120 ext. 56074

Web: https://geg.uoguelph.ca/faculty/lindsay-john

Research page: https://jblindsay.github.io/ghrg/index.html

WhiteboxTools GIS project: https://www.whiteboxgeo.com/

Twitter: https://twitter.com/whiteboxgeo

________________________________
From: David Strip <qgis-dev at stripfamily.net><mailto:qgis-dev at stripfamily.net>
Sent: Friday, January 19, 2024 1:10 PM
To: Greg Troxel <gdt at lexort.com><mailto:gdt at lexort.com>; John Lindsay <jlindsay at uoguelph.ca><mailto:jlindsay at uoguelph.ca>
Cc: John Lindsay via QGIS-Developer <qgis-developer at lists.osgeo.org><mailto:qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] QGIS plugin depends on pypi package

CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to IThelp at uoguelph.ca<mailto:IThelp at uoguelph.ca>.

On 1/19/2024 10:59 AM, Greg Troxel via QGIS-Developer wrote:

When you say "qgis seems to use its own version", I wonder if that is in
the context of particular binary distribution.  And if there really is a
full extra copy of libs, or if it's about a per-user site-packages
directory.

The OS4Geo binary distribution contains a Python39 directory which is a full python installation including python.exe, a Lib directory which in turn contains a site-packages directory. As least in my case, this is installed system-wide, not per-user



_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20240119/d6f2ad86/attachment-0001.htm>