[Qgis-user] Problem loading several provider libs from external script

[Nils Olaf de Reus]

Hi everyone,

When trying to use the qgis libraries from an external script (that
is, a script running not within the context of the embedded console of
QGIS), I get only a severely limited selection of available data
providers. A number of libraries that load fine from inside the
context of QGIS' embedded python console, fail to load when initQgis
is used from a standalone external script.

I have a Linux 2.6.32-5-amd64 x86_64 GNU/Linux platform, running
Debian Squeeze and the 1.8 release of QGIS for Squeeze from the
official repository at http://qgis.org/debian. To test if the problem
described below still existed with a newer version of QGIS, I created
a clean Debian Squeeze chroot environment, and installed the latest
nightly build. The problem still existed with the nightly, same as in
the 1.8 release.


The provider lib files are present in the usual plugin directory:

nils at fennek:~$ ls /usr/lib/qgis/plugins
libcoordinatecaptureplugin.so  libmemoryprovider.so
libdelimitedtextplugin.so      libmssqlprovider.so
libdelimitedtextprovider.so    libofflineeditingplugin.so
libdiagramoverlay.so           libogrprovider.so
libdxf2shpconverterplugin.so   liboracleplugin.so
libevis.so                     libosmprovider.so
libgdalprovider.so             libpostgresprovider.so
libgeorefplugin.so             librasterterrainplugin.so
libgpsimporterplugin.so        libroadgraphplugin.so
libgpxprovider.so              libspatialiteprovider.so
libgrassplugin.so              libspatialqueryplugin.so
libgrassprovider.so            libspitplugin.so
libgrassrasterprovider.so      libwfsprovider.so
libheatmapplugin.so            libwmsprovider.so
libinterpolationplugin.so      libzonalstatisticsplugin.so


My external script goes through the usual steps, shown below in an
interactive session for easy replication:

nils at fennek:~$ python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from qgis.core import *
>>> app = QgsApplication([u'/usr'], False)
>>> print app.showSettings()
Application state:
Prefix:		/usr
Plugin Path:		/usr/lib/qgis/plugins
Package Data Path:	/usr/share/qgis
Active Theme Name:	
Active Theme Path:	:/images/themes//
Default Theme Path:	:/images/themes/default/
SVG Search Paths:	/usr/share/qgis/svg/
		/home/nils/.qgis//svg/
User DB Path:	/usr/share/qgis/resources/qgis.db

>>> app.initQgis()

Now if I check which providers are there, I only get three..

>>> for provider in QgsProviderRegistry.instance().providerList():
...     print provider
...
delimitedtext
gdal
gpx


..while that exact same statement inside Qgis' embedded Python Console
yields all thirteen providers:

>>> for provider in QgsProviderRegistry.instance().providerList():
...     print provider
...
WFS
delimitedtext
gdal
gpx
grass
grassraster
memory
mssql
ogr
osm
postgres
spatialite
wms


Sofar, I've traced it to the point where during initQgis(), the
creation of the QgsProviderRegistry will reject those libraries that
return False on the test if QLibrary's load() method will succesfully
load them. This too can be shown from the interactive python console.
First in the external console:

>>> from PyQt4 import QtCore
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libdelimitedtextprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libgdalprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libgpxprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libogrprovider.so').load()
False
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libspatialiteprovider.so').load()
False
>>>

..well, I think you see the pattern. Outside of those three
('delimitedtext', 'gpx' and 'gdal'), QLibrary doesn't seem able to
load any of the other provider libraries.

Now my first guess when QLibrary can't load a library would be that
something is wrong with that library. Except of course that when these
same commands are issued inside the embedded console under Qgis, these
same libs load fine:

>>> from PyQt4 import QtCore
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libdelimitedtextprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libgdalprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libgpxprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libogrprovider.so').load()
True
>>> QtCore.QLibrary(u'/usr/lib/qgis/plugins/libspatialiteprovider.so').load()
True
>>>


I've tried making my sys.path identical between the embedded and
external python sessions. No effect.

I didn't see any obvious systematic problems with the libraries that
didn't load (I focussed on libmemoryprovider, because that one is
relatively simple), and neither in the ldd nor readelf outputs did
anything stand out as an obvious red flag. (I say 'obvious red flag'
because I lack the expertise to properly understand even half the
things that come out of readelf, and can only look for a limited
number of 'telltale signs')

So.. what am I missing? Is there some other library that Qgis preloads
that makes the loading of the other dynamically linked providers work?
And how can I get that to also happen from a regular python script not
inside the embedded python console?


Kind regards,
Nils