[Qgis-developer] Changes to python/utils.py

[Larry Shaffer]

On Wed, May 30, 2012 at 12:04 AM, Martin Dobias <wonder.sk at gmail.com> wrote:
> On Wed, May 30, 2012 at 1:44 AM, Larry Shaffer <larrys at dakotacarto.com> wrote:
>>[...]
>> By 'live' I'm guessing you are referring to the 'automatic detection
>> of module changes,' i.e. monitor.py . I don't think that live
>> reloading would be that useful for QGIS plugin development. I'm mostly
>> looking at this module's functionality as a more complete basic plugin
>> reloader. Live reloading and __reload__() functions are secondary, and
>> will probably cause more problems than they solve.
>
> By live reloading I meant the reloading you proposed with reloader
> module, without monitoring of files. I thought that your main reason
> for introducing the reloader was the ability to keep plugin state
> while reloading (with __reload__() method) - if not, where is the
> added value of it?

Ok, I see. After re-reading my original post, I made it seem that the
extra feature of the proposed change, the __reload__() method, was
more my main point. Sorry about that.

A little background... I have spent some effort trying to get my
plugins to fully unload/load as many times as possible within QGIS
(via Plugin Reloader) before having to restart the app. Inevitably any
complex plugins crash (usually QGIS, too) because something is not
being removed from sys.modules during the unload of the plugin, using
the current utils.py module tracking method.

Additionally, one of the main features of my Plugin Editor plugin is
that it runs inside of QGIS. This can also be a drawback, if the
plugin you are working on requires many unloads/loads, precipitating a
restart of QGIS, especially if some resource isn't being reloaded. So,
I've been looking for a solution robust enough to be able to
unload/load most conventional plugins unlimited times - as a
development tool.

I now see your point on hard vs. live reloading of plugins (delete
from sys.modules vs. reload). Yes, avoid live reloading of plugins =
good thing, keep sanity. :^)  Still, I think there is something to
gain from reloader.py's more extensive dependency tracking
methodology.

In utils.py, the import wrapper takes a conservative approach, by only
including package-dependent plugin modules. I think reloader.py's
approach, with its global parent, creates a more complete dependency
graph. In my preliminary tests, all modules were reloaded (including
those with relative imports or sys.path augmentation for embedded 3rd
party packages).

If reloader.py's dependency graphing approach was used for utils.py's
import wrapper, for cleaner plugin unloads, then at least two things
would have to happen beyond what utils.py does now:

1) On deletion from sys.modules ( _unloadPluginModules() ), important
modules would need to be skipped (qgis, PyQt4, utils, console, etc.).
I had to do this with the reloader.py implementation I am currently
using, which is for live reloading when testing normal
modules/packages, not plugins.

2) A cross-reference check of some kind to make sure the module to be
removed is not already in another plugin's dependency graph. The
developer could then be notified that a clean reload was not possible,
and why. Since this is a plugin-testing reload, a developer is
probably ok with turning off other plugins. Normal plugin unloads
(Plugin Manager, Installer, etc.) could silently ignore the incomplete
unload, as they do now.

So, it looks like the best thing to do now is to make the changes and
test, test, test. I think the new Python exception output, with frame
locals, should come first, though.

Larry