[Qgis-developer] Changes to python/utils.py

[Martin Dobias]

Hi Larry

and sorry for getting back to you so late.

On Wed, May 30, 2012 at 6:07 PM, Larry Shaffer <larrys at dakotacarto.com> wrote:
> A little background... I have spent some effort trying to get my
> plugins to fully unload/load as many times as possible within QGIS
> (via Plugin Reloader) before having to restart the app. Inevitably any
> complex plugins crash (usually QGIS, too) because something is not
> being removed from sys.modules during the unload of the plugin, using
> the current utils.py module tracking method.

Hmm... are you aware of any reproducible crash? I would be interested
to see where it comes from. The unloader and import hook with
dependency tracking is not bullet-proof, but it should cover the
common cases quite well. You have probably seen that there is also
explicit logic for unloading resource data from pyrcc4-generated files
- without that reloaded plugins tend to crash due to dangling pointer
to resources - and that cannot be handled by a generic reloader.


> I now see your point on hard vs. live reloading of plugins (delete
> from sys.modules vs. reload). Yes, avoid live reloading of plugins =
> good thing, keep sanity. :^)  Still, I think there is something to
> gain from reloader.py's more extensive dependency tracking
> methodology.
>
> In utils.py, the import wrapper takes a conservative approach, by only
> including package-dependent plugin modules. I think reloader.py's
> approach, with its global parent, creates a more complete dependency
> graph. In my preliminary tests, all modules were reloaded (including
> those with relative imports or sys.path augmentation for embedded 3rd
> party packages).
>
> If reloader.py's dependency graphing approach was used for utils.py's
> import wrapper, for cleaner plugin unloads, then at least two things
> would have to happen beyond what utils.py does now:
>
> 1) On deletion from sys.modules ( _unloadPluginModules() ), important
> modules would need to be skipped (qgis, PyQt4, utils, console, etc.).
> I had to do this with the reloader.py implementation I am currently
> using, which is for live reloading when testing normal
> modules/packages, not plugins.

There could be a module that does all the mandatory imports, so that
would create another branch in the dependency graph, not allowing
those modules to be unloaded, right?


> 2) A cross-reference check of some kind to make sure the module to be
> removed is not already in another plugin's dependency graph. The
> developer could then be notified that a clean reload was not possible,
> and why. Since this is a plugin-testing reload, a developer is
> probably ok with turning off other plugins. Normal plugin unloads
> (Plugin Manager, Installer, etc.) could silently ignore the incomplete
> unload, as they do now.

Or eventually it could try to reload the other plugins that are
dependent of the shared python modules.

> So, it looks like the best thing to do now is to make the changes and
> test, test, test. I think the new Python exception output, with frame
> locals, should come first, though.

Yeah, the new python exception output would be very helpful. But the
reloader might be interesting as well :-)

Regards
Martin