[Qgis-user] [Caution - External w/link] Re: Troubleshooting Python Script - QGIS library Import on Recent Versions on Mac OS X
Odom, Johnnie
JOdom at ecsdfl.us
Wed Sep 23 09:44:48 PDT 2026
Greg,
Thank you for all of your advice and help. I've been intermittently trying
to troubleshoot ever since. I'm writing for anyone who comes across this
thread later, and to throw out a few additional possibilities for group
discussion.
First, the advice to clean up Python and MacPorts did not solve my specific
problem but it was very helpful in clarifying the situation and also
getting some other projects straight. Over time I had "Python creep" on my
system, and bringing down the installed Pythons to two -- the
system-installed Python 3.9.x (which one should *never* remove for OS X
reasons) and a copy of 3.12.x (which is what QGIS 4.2.0 and 4.2.2 are
compiled with on the Mac) was helpful in verifying all my other python work.
On the Mac, the equivalent of ldd is "otool -L".
Without explaining the rabbit holes I have been down the last several
weeks, I can simply say that the dylib situation on OS X is complicated.
There is a lot of advice on fixing dynamic library problems if you are
writing an application and compiling it yourself. A lot of this comes down
to using name_install_tool to replace or add rpaths. It is not easy to tell
OS X to look in additional places for dynamic libraries, in part because
System Integrity Protection (SIP) strips out some of the environmental
variables you need.
Turning SIP off will not solve the problem I am having by itself, and is a
bad idea in any case.
Using name_install_tool is not useful with a prepackaged QGIS library
because you have to re-sign the executable after you use it, and I do not
have the code signing certificate for prepackaged QGIS.
If _core.so could find the path to the dynamic libraries that are already
in the app bundle, all would be well. But that appears tricky on the Mac.
I have developed a hypothesis about what the wider issue that is going on.
I may not be right, but here it is for discussion.
1. Somewhere I read that if you wanted to use the qgis python libraries in
an external program on OS X, you need to compile it yourself via MacPorts.
So I did that. And it worked, until it didn't. This was with QGIS 3.4.x
2. My thinking about why it stopped working is that I ran a MacPorts update
that either changed QGIS or my installed Python or both and broke it.
3. I could not re-create a working config using MacPorts and the 3.4.x
releases.
4. So I changed over to 4.x, which of course fixes a lot of the previous
problems with QGIS on the Mac because it signs the app (among other
things). MacPorts does not have 4.x so I use either the official binaries
or the homebrew install (removing MacPorts entirely to avoid conflicts),
which appear to be the same package.
5. Maybe if I built 4.x from source it would work, but right now vcpkg
fails trying to compile py-rpds deep in the first stage and that's a
side-quest rabbit hole that I go down and then leave for a bit. (
https://github.com/qgis/QGIS/blob/master/INSTALL.md#52-building-with-vcpkg )
All of which is to say, the Python libraries are all "correct" for the
machine that compiled it. And of course we can run commands from the
built-in Python console. It just won't work when calling from different
Python from a machine that the QGIS app has been copied to.
As a test, I can run the beginning of my existing script (see #1) in the
Python console within QGIS right up until I reference data files outside
QGIS, at which point it breaks -- maybe I can get around that.
But I don't want to run things in the console, I want to run them from an
external script. So I try that using the path to the python3.12 that lives
inside the QGIS app. That gives me:
ERROR 1: PROJ: proj_identify: Cannot find proj.db
which appears to be a conda error. So perhaps the build process for
official binaries uses conda? (I am aware of conda but have never used it).
Okay, that's another rabbit hole ....
... what if we use python -m venv to create a copy of the correct
environment?
/Applications/QGIS-final-4_2_2.app/Contents/MacOS/python -m venv /tmp/venv2
Error: Command '['/tmp/venv2/bin/python3.12', '-m', 'ensurepip',
'--upgrade', '--default-pip']' returned non-zero exit status 1.
Okay, it was worth a try.
Going back down the path of using the QGIS python environment directly, can
we add additional packages we might need in our program by evoking pip? No,
that doesn't seem to work either.
Having done all of that, I am back to my central requirements:
I want to write external Python scripts / programs on my Mac in an IDE of
my choice that I can do significant work with and which can leverage
various Python libraries not already included with QGIS.
Based on all the above, do you all think it is more fruitful for me to
continue to try to build QGIS 4.2.x from source, to attempt to work with
the python already inside the app, or pursue some other avenue?
Thank You,
Johnnie
On Thu, Aug 20, 2026 at 12:56 PM Greg Troxel via QGIS-User <
qgis-user at lists.osgeo.org> wrote:
> "Odom, Johnnie via QGIS-User" <qgis-user at lists.osgeo.org> writes:
>
> > I am having problems using QGIS in a Python script. This script
> previously
> > worked. My goal is to dynamically build maps. I am using Mac OS X (M2
> > processor) and have access to both QGIS 3.44 (installed via MacPorts) and
> > 4.2 (installed from the QGIS website.)
> >
> > I read the QGIS documentation on developing apps using QGIS and a few
> other
> > articles and then gradually began writing code. In the end I had a script
> > that could create a project, import layes in various formats, and run
> some
> > plugin operations against them. My code worked for several months and
> then
> > a few months ago it stopped working. I suspect an update to my Mac or to
> > the MacPorts QGIS was the cause, but I cannot get back to a good state.
>
> This is general advice not necessarily related to your situation but
> this is all complicated and it's reasonably likely your installation
> situation is not exactly what you think. Figuring that out and reducing
> chaos is likely to help.
>
> With that:
>
> - Check if you have any environment variables that ask to set load
> paths for anything. Run printenv and be able to explain every
> single setting, why it's there and what it does. Research and
> reduce to what you intend.
>
> - Understand how many installations of python you have and where they
> are. On up-to-date macOS, I see /usr/bin/python3 which is 3.9.6.
> (Which means that up-to-date macOS is not up to date, but apple
> thinks differently!) I also have /opt/pkg/bin/python3.13 which is
> 3.13.14.
>
> - In your case, you say macports, so presumably that's /opt/something,
> and you can see how many versions you have.
>
> My advice would be to remove all packages, and all Applications,
> that you don't still intend to have. After an extra backup of
> course.
>
> - Within macports, there should be some sort of "pkg_admin check"
> command that says to check the installed files against the manifest
> of what should be installed. Figure this out (ask in macports
> lists) and run it and fix anything that's amiss.
>
> - Also, macports should be in a consistent state. If you are
> upgrading some programs and not others, you may be inconsistent.
> Ask about this in macports land. With pkgsrc if you update some
> things but not others you can get into trouble, and I'd expec the
> same.
>
> [I use qgis on NetBSD not macOS.]
>
> - Next, I'd search for all the qgis binaries/apps and make sure that
> only the two that you think are installed are there.
>
> - Pay particular attention to the version from the qgis website and
> see that is bundled with it. In a packaging system like macports
> or pkgsrc, the various dependencies will be separate packages.
>
> - Then, for each, I'd find the binary and use the mac equivalent of
> objdump and ldd to see which python version, at which path, is being
> loaded.
>
> At this point you should have a much clearer understanding of how your
> setup is supposed to work.
>
> > My issue, as far as I can tell, is that the main QGIS library either does
> > not import or crashes when I import it depending upon the QGIS
> installation.
>
> Yes, but why it does that is highly unclear.
>
> > Using python 3.12, 3.13, or 3.14 if I try to import from
> > /Applications/MacPorts/QGIS3.app/Contents/Resources/python/ (for version
> > 3.44) then the program stops running after the "import qgis" statement
> but
> > it does not indicate that it has stopped -- it just never reaches the
> next
> > statement in the programming (i.e. print("I have imported qgis")).
>
> It does not make sense that you can do this from 3 python versions.
> Those versions are all presumably provided by macports and surely
> macports selected one at qgis build time. It only makes sense to be
> using the version of python that qgis was built with. On my system
> (using the 4.2 work-in-progress version):
>
> $ ldd /usr/pkg/bin/qgis|egrep python
> -lqgispython.4.2.1 => /usr/pkg/lib/libqgispython.so.4.2.1
> -lpython3.13.1.0 => /usr/pkg/lib/libpython3.13.so.1.0
> $ ldd /usr/pkg/lib/libqgispython.so.4.2.1|egrep pytho
> /usr/pkg/lib/libqgispython.so.4.2.1:
> -lpython3.13.1.0 => /usr/pkg/lib/libpython3.13.so.1.0
>
> > If I switch to version 4.2 and import from
> > /Applications/QGIS-final-4_2_0.app/Contents/Resources/
> python3.12/site-packages/
> > with the same code, then I get a crash with the stack trace ending as
> > follows:
> >
> > File
> > "/Applications/QGIS-final-4_2_0.app/Contents/Resources/
> python3.12/site-packages/qgis/core/__init__.py",
> > line 29, in <module>
> > from qgis._core import *
> > ImportError:
> > dlopen(/Applications/QGIS-final-4_2_0.app/Contents/
> Resources/python3.12/site-packages/qgis/_core.so,
> > 0x0002): Library not loaded: @rpath/libpython3.12.dylib
> > Referenced from: <75DA0087-BE20-3D08-B088-B5B4500EE33B>
> > /Applications/QGIS-final-4_2_0.app/Contents/Resources/
> python3.12/site-packages/qgis/_core.so
> > Reason: no LC_RPATH's found
>
> It seems you have to set a PYTHONPATH because those aren't hooked into
> the python implementation's defaults. That looks like python3.12 and
> thus I think you should be running python3.12 to start with. Not just
> any of the n you might have, but the one that goes with the python
> library that those modules are linked with. I have an old LTR somehow
> (because I don't really use qgis on this mac) and it has
>
> /Applications/QGIS-LTR.app/Contents/MacOS/bin/python3.9
>
> The error message you got indicateds that trying to load
> libpython3.12.dylib from the _core.so module failed because it couldn't
> be found. I can't tell if that's a packaging bug, or a result of you
> having started a different python, or something else.
>
> I hope this helps you towards figuring this out.
> _______________________________________________
> QGIS-User mailing list
> QGIS-User at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20260923/9cac3853/attachment-0001.htm>
More information about the QGIS-User
mailing list