[Qgis-user] Troubleshooting Python Script - QGIS library Import on Recent Versions on Mac OS X

[Greg Troxel]

"Odom, Johnnie via QGIS-User" <qgis-user at lists.osgeo.org> writes:

> I am having problems using QGIS in a Python script. This script previously
> worked. My goal is to dynamically build maps. I am using Mac OS X (M2
> processor) and have access to both QGIS 3.44 (installed via MacPorts) and
> 4.2 (installed from the QGIS website.)
>
> I read the QGIS documentation on developing apps using QGIS and a few other
> articles and then gradually began writing code. In the end I had a script
> that could create a project, import layes in various formats, and run some
> plugin operations against them. My code worked for several months and then
> a few months ago it stopped working. I suspect an update to my Mac or to
> the MacPorts QGIS was the cause, but I cannot get back to a good state.

This is general advice not necessarily related to your situation but
this is all complicated and it's reasonably likely your installation
situation is not exactly what you think.  Figuring that out and reducing
chaos is likely to help.

With that:

  - Check if you have any environment variables that ask to set load
    paths for anything.  Run printenv and be able to explain every
    single setting, why it's there and what it does.   Research and
    reduce to what you intend.

  - Understand how many installations of python you have and where they
    are.  On up-to-date macOS, I see /usr/bin/python3 which is 3.9.6.
    (Which means that up-to-date macOS is not up to date, but apple
    thinks differently!)   I also have /opt/pkg/bin/python3.13 which is
    3.13.14.

  - In your case, you say macports, so presumably that's /opt/something,
    and you can see how many versions you have.

    My advice would be to remove all packages, and all Applications,
    that you don't still intend to have.  After an extra backup of
    course.

  - Within macports, there should be some sort of "pkg_admin check"
    command that says to check the installed files against the manifest
    of what should be installed.  Figure this out (ask in macports
    lists) and run it and fix anything that's amiss.

  - Also, macports should be in a consistent state.  If you are
    upgrading some programs and not others, you may be inconsistent.
    Ask about this in macports land.  With pkgsrc if you update some
    things but not others you can get into trouble, and I'd expec the
    same.

  [I use qgis on NetBSD not macOS.]

  - Next, I'd search for all the qgis binaries/apps and make sure that
    only the two that you think are installed are there.

  - Pay particular attention to the version from the qgis website and
    see that is bundled with it.   In a packaging system like macports
    or pkgsrc, the various dependencies will be separate packages.

  - Then, for each, I'd find the binary and use the mac equivalent of
    objdump and ldd to see which python version, at which path, is being
    loaded.

At this point you should have a much clearer understanding of how your
setup is supposed to work.

> My issue, as far as I can tell, is that the main QGIS library either does
> not import or crashes when I import it depending upon the QGIS installation.

Yes, but why it does that is highly unclear.

> Using python 3.12, 3.13, or 3.14 if I try to import from
> /Applications/MacPorts/QGIS3.app/Contents/Resources/python/ (for version
> 3.44) then the program stops running after the "import qgis" statement but
> it does not indicate that it has stopped -- it just never reaches the next
> statement in the programming (i.e. print("I have imported qgis")).

It does not make sense that you can do this from 3 python versions.
Those versions are all presumably provided by macports and surely
macports  selected one at qgis build time.   It only makes sense to be
using the version of python that qgis was built with.  On my system
(using the 4.2 work-in-progress version):

  $ ldd /usr/pkg/bin/qgis|egrep python
          -lqgispython.4.2.1 => /usr/pkg/lib/libqgispython.so.4.2.1
          -lpython3.13.1.0 => /usr/pkg/lib/libpython3.13.so.1.0
  $ ldd /usr/pkg/lib/libqgispython.so.4.2.1|egrep pytho
  /usr/pkg/lib/libqgispython.so.4.2.1:
          -lpython3.13.1.0 => /usr/pkg/lib/libpython3.13.so.1.0

> If I switch to version 4.2 and import from
> /Applications/QGIS-final-4_2_0.app/Contents/Resources/python3.12/site-packages/
> with the same code, then I get a crash with the stack trace ending as
> follows:
>
>   File
> "/Applications/QGIS-final-4_2_0.app/Contents/Resources/python3.12/site-packages/qgis/core/__init__.py",
> line 29, in <module>
>     from qgis._core import *
> ImportError:
> dlopen(/Applications/QGIS-final-4_2_0.app/Contents/Resources/python3.12/site-packages/qgis/_core.so,
> 0x0002): Library not loaded: @rpath/libpython3.12.dylib
>   Referenced from: <75DA0087-BE20-3D08-B088-B5B4500EE33B>
> /Applications/QGIS-final-4_2_0.app/Contents/Resources/python3.12/site-packages/qgis/_core.so
>   Reason: no LC_RPATH's found

It seems you have to set a PYTHONPATH because those aren't hooked into
the python implementation's defaults.  That looks like python3.12 and
thus I think you should be running python3.12 to start with. Not just
any of the n you might have, but the one that goes with the python
library that those modules are linked with.  I have an old LTR somehow
(because I don't really use qgis on this mac) and it has

  /Applications/QGIS-LTR.app/Contents/MacOS/bin/python3.9

The error message you got indicateds that trying to load
libpython3.12.dylib from the _core.so module failed because it couldn't
be found.  I can't tell if that's a packaging bug, or a result of you
having started a different python, or something else.

I hope this helps  you towards figuring this out.