bpy.ops.preferences.addon_enable() only checks for changes in __init__.py; Problem for multi-file add-ons #66924

New Issue

Iemand Iets · 2019-07-14T17:24:29+02:00

Iemand Iets commented

2019-07-14 17:24:29 +02:00

Symptoms

When you're developing an add-on existing out of multiple files on a disk, you want to reload your add-on many times to include the changes you made.
This works fine when you make a modification in __init__.py and call bpy.ops.preferences.addon_enable(module='your_module'), as you will see: "module changed on disk: 'path/to/file.py' reloading...".

However, if you make a change in another module, e.g. your_module_props.py, and call addon_enable(), nothing is triggered. You will only see {'FINISHED'}.
If you afterwards make a change to __init__.py, the previous changes you made to your_module_props.py are seen by Blender this time when you call addon_enable().

Likely culprit

I believe the culprit is here: https://developer.blender.org/diffusion/B/browse/master/release/scripts/modules/addon_utils.py$325

mtime_orig = getattr(mod, "__time__", 0)
mtime_new = os.path.getmtime(mod.__file__)  # only checks __init__.py
if mtime_orig != mtime_new:

These lines of code only check if the main file (__init__.py) has changed.

Possible solutions

One of the following:

Extend the check to check if any (Python) file of an add-on has changed.
Remove the check if files changed, and always trigger the reload.

Importance

Any medium/large add-on that follows proper code conventions to prevent spaghetti code will use multiple files. This limited check to only __init__.py severely slows down app development, as we cannot do a quick reload. Instead we have to make a minimal change to __init__.py to trigger the reload, or restart Blender every time we have made a code modification.

# Symptoms When you're developing an add-on existing out of multiple files on a disk, you want to reload your add-on many times to include the changes you made. This works fine when you make a modification in `__init__.py` and call `bpy.ops.preferences.addon_enable(module='your_module')`, as you will see: `"module changed on disk: 'path/to/file.py' reloading..."`. However, if you make a change in another module, e.g. `your_module_props.py`, and call `addon_enable()`, nothing is triggered. You will only see `{'FINISHED'}`. If you afterwards make a change to `__init__.py`, the previous changes you made to `your_module_props.py` are seen by Blender this time when you call `addon_enable()`. # Likely culprit I believe the culprit is here: https://developer.blender.org/diffusion/B/browse/master/release/scripts/modules/addon_utils.py$325 ``` mtime_orig = getattr(mod, "__time__", 0) mtime_new = os.path.getmtime(mod.__file__) # only checks __init__.py if mtime_orig != mtime_new: ``` These lines of code only check if the main file (`__init__.py`) has changed. # Possible solutions One of the following: - Extend the check to check if any (Python) file of an add-on has changed. - Remove the check if files changed, and always trigger the reload. # Importance Any medium/large add-on that follows proper code conventions to prevent spaghetti code will use multiple files. This limited check to only `__init__.py` severely slows down app development, as we cannot do a quick reload. Instead we have to make a minimal change to `__init__.py` to trigger the reload, or restart Blender every time we have made a code modification.

Iemand Iets commented

2019-07-14 17:24:29 +02:00

Added subscriber: @NumesSanguis-3

Iemand Iets commented

2019-07-14 18:37:49 +02:00

To test if the check if mtime_orig != mtime_new: is indeed the problem, I made an operator (simplified version of the original addon_enable(): developer.blender.org/diffusion/B/browse/master/release/scripts/modules/addon_utils.py$327) with that removed:

class OBJECT_OT_reload_module(bpy.types.Operator):
    bl_idname = "object.reload_module"
    bl_label = "Reload specified module"
    bl_options = {'REGISTER'}

    def execute(self, context):
        print("Hello")

        import importlib
        import sys

        module_name = "your_module"
        mod = sys.modules.get(module_name)
        mod.__addon_enabled__ = False
        try:
            importlib.reload(mod)
        except Exception as ex:
            # handle_error(ex)
            print(f"Failed: {ex}")
            del sys.modules[module_name]
            return None

        mod.register()
        # * OK loaded successfully! *
        mod.__addon_enabled__ = True

        return {'FINISHED'}

and indeed, the add-on properly reloads now.
So I would suggest solution 2 in the main post.

To test if the check `if mtime_orig != mtime_new:` is indeed the problem, I made an operator (simplified version of the original `addon_enable()`: developer.blender.org/diffusion/B/browse/master/release/scripts/modules/addon_utils.py$327) with that removed: ``` class OBJECT_OT_reload_module(bpy.types.Operator): bl_idname = "object.reload_module" bl_label = "Reload specified module" bl_options = {'REGISTER'} def execute(self, context): print("Hello") import importlib import sys module_name = "your_module" mod = sys.modules.get(module_name) mod.__addon_enabled__ = False try: importlib.reload(mod) except Exception as ex: # handle_error(ex) print(f"Failed: {ex}") del sys.modules[module_name] return None mod.register() # * OK loaded successfully! * mod.__addon_enabled__ = True return {'FINISHED'} ``` and indeed, the add-on properly reloads now. So I would suggest solution 2 in the main post.

Sybren A. Stüvel commented

2019-07-16 13:54:11 +02:00

Added subscriber: @dr.sybren

Sybren A. Stüvel commented

2019-07-16 13:54:11 +02:00

Changed status from 'Open' to: 'Archived'

Sybren A. Stüvel closed this issue

2019-07-16 13:54:11 +02:00

Sybren A. Stüvel self-assigned this 2019-07-16 13:54:11 +02:00

Sybren A. Stüvel commented

2019-07-16 13:54:11 +02:00

Fortunately, this is not as big an issue as you describe. The code you linked is only used when enabling an add-on; the issue you describe seems to assume the way to reload an add-on is to disable and enable it.

Reloading your add-on can be done via the bpy.ops.script.reload() operator which was bound to F8 in Blender 2.79; it's now available with the F3 menu (search for 'reload scripts') and you can create your own keyboard binding if you want. Multi-file addons need reloading support, which is quite easy to do; see this slide and the following slides or watch my Blender Conference 2016 workshop on the subject.

Fortunately, this is not as big an issue as you describe. The [code you linked](https://developer.blender.org/diffusion/B/browse/master/release/scripts/modules/addon_utils.py$325) is only used when enabling an add-on; the issue you describe seems to assume the way to reload an add-on is to disable and enable it. Reloading your add-on can be done via the `bpy.ops.script.reload()` operator which was bound to F8 in Blender 2.79; it's now available with the F3 menu (search for 'reload scripts') and you can create your own keyboard binding if you want. Multi-file addons need reloading support, which is quite easy to do; see [this slide and the following slides](https:*stuvel.eu/files/bconf2016/#/23) or watch my [Blender Conference 2016 workshop](https:*youtu.be/mYrPqrFY7mA?t=731) on the subject.

Iemand Iets commented

2019-07-21 16:06:11 +02:00

Thank you @dr.sybren , bpy.ops.script.reload() indeed reloads all add-ons. I had if "bpy" in locals(): check to reload add-ons, but that wasn't triggered by my code, because __init__.py didn't change.

While luckily indeed a smaller issue than I assumed, I think it's nota solved one.

When asked at blender.chat #python channel: https:*blender.chat/channel/python?msg=fGEyJtjNZqfpApxQ5 , none of the multiple people who responded pointed me to bpy.ops.script.reload() instead of bpy.ops.preferences.addon_enable(module='your_module') I would assume the most knowledgeable Python programmers are active here, and therefore it seems bpy.ops.script.reload() is not common knowledge. This is more a documentation issue, so I created an issue about this here: https:*developer.blender.org/T67387
bpy.ops.script.reload() reloads ALL add-ons. If one of these add-ons is producing a error / warning, it will pollute the output/logs for the add-on you're actually try to develop. Also, if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process. Possible solutions I could think of:
- Allow for an argument to bpy.ops.script.reload(module_name=None), and only reload all when either None or no argument is passed.
- Make bpy.ops.preferences.addon_enable(module='your_module') intended for reloading specific add-ons by remove / extend the check if only __init__.py file changed.

Thank you @dr.sybren , `bpy.ops.script.reload()` indeed reloads all add-ons. I had `if "bpy" in locals():` check to reload add-ons, but that wasn't triggered by my code, because `__init__.py` didn't change. While luckily indeed a smaller issue than I assumed, I think it's **not**a solved one. - When asked at blender.chat #python channel: https:*blender.chat/channel/python?msg=fGEyJtjNZqfpApxQ5 , none of the multiple people who responded pointed me to `bpy.ops.script.reload()` instead of `bpy.ops.preferences.addon_enable(module='your_module')` I would assume the most knowledgeable Python programmers are active here, and therefore it seems `bpy.ops.script.reload()` is not common knowledge. This is more a documentation issue, so I created an issue about this here: https:*developer.blender.org/T67387 - `bpy.ops.script.reload()` reloads ALL add-ons. If one of these add-ons is producing a error / warning, it will pollute the output/logs for the add-on you're actually try to develop. Also, if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process. Possible solutions I could think of: - Allow for an argument to `bpy.ops.script.reload(module_name=None)`, and only reload all when either None or no argument is passed. - Make `bpy.ops.preferences.addon_enable(module='your_module')` intended for reloading specific add-ons by remove / extend the check if only `__init__.py` file changed.

Campbell Barton commented

2019-07-21 16:29:48 +02:00

Added subscriber: @ideasman42

Campbell Barton commented

2019-07-21 16:29:48 +02:00

For development, you can:

import importlib
importlib.reload(your_addon_name)

Checking the date on the init file isn't meant to be a comprehensive check of the entire package.

For development, you can: ``` import importlib importlib.reload(your_addon_name) ``` Checking the date on the init file isn't meant to be a comprehensive check of the entire package.

Sybren A. Stüvel commented

2019-07-22 11:24:35 +02:00

In #66924#729370, @NumesSanguis-3 wrote:
I would assume the most knowledgeable Python programmers are active here

Not all of them, and not all the time. Just search the Python API documentation for "reload script" and you'll find the bpy.ops.script.reload() operator. Granted, it's not documented very well.

if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process.

Well, if you keep multiple non-relevant add-ons enabled while you're developing, I don't think this is an issue with reloading. I think it's an issue between keyboard & chair.

In #66924#729380, @ideasman42 wrote:
For development, you can:
import importlib
importlib.reload(your_addon_name)

You could even add this code to an operator in your add-on and bind it to a menu item or button, then it's literally a one-click reload for only your add-on.

> In #66924#729370, @NumesSanguis-3 wrote: > I would assume the most knowledgeable Python programmers are active here Not all of them, and not all the time. Just search the Python API documentation for "reload script" and you'll find the `bpy.ops.script.reload()` operator. Granted, it's not documented very well. > if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process. Well, if you keep multiple non-relevant add-ons enabled while you're developing, I don't think this is an issue with reloading. I think it's an issue between keyboard & chair. > In #66924#729380, @ideasman42 wrote: > For development, you can: > > ``` > import importlib > importlib.reload(your_addon_name) > ``` You could even add this code to an operator in your add-on and bind it to a menu item or button, then it's literally a one-click reload for only your add-on.

gecko man commented

2019-07-22 21:02:47 +02:00

Added subscriber: @geckoman

gecko man commented

2019-07-22 21:02:47 +02:00

I have created a workaround to this problem.
Example init.py:

import importlib, sys # required

# reloads class' parent module and returns updated class
def reload_class(c):
	mod = sys.modules.get(c.__module__)
	importlib.reload(mod)
	return mod.__dict__[c.__name__]

# imports to be updated
from . somefile import someclass
from . import somemodule

someclass = reload_class(someclass) # reload imported class
importlib.reload(somemodule) # reload imported module

...

someclass and somemodule will stay up-to-date when reloading addon with bpy.ops.preferences.addon_enable() and with bpy.ops.script.reload().

I have created a workaround to this problem. Example __init__.py: ```lang=py3 import importlib, sys # required # reloads class' parent module and returns updated class def reload_class(c): mod = sys.modules.get(c.__module__) importlib.reload(mod) return mod.__dict__[c.__name__] # imports to be updated from . somefile import someclass from . import somemodule someclass = reload_class(someclass) # reload imported class importlib.reload(somemodule) # reload imported module ... ``` `someclass` and `somemodule` will stay up-to-date when reloading addon with `bpy.ops.preferences.addon_enable()` and with `bpy.ops.script.reload()`.

Sybren A. Stüvel commented

2019-07-22 23:16:33 +02:00

Don't use the from X import Y syntax to import individual classes, that'll make reloading more cumbersome (as you've seen). If you just import (sub)modules you avoid the need for that reload_class() function.

Don't use the `from X import Y` syntax to import individual classes, that'll make reloading more cumbersome (as you've seen). If you just import (sub)modules you avoid the need for that `reload_class()` function.

Iemand Iets commented

2019-07-25 16:14:57 +02:00

In #66924#730083, @dr.sybren wrote:

In #66924#729370, @NumesSanguis-3 wrote:
I would assume the most knowledgeable Python programmers are active here

Not all of them, and not all the time. Just search the Python API documentation for "reload script" and you'll find the bpy.ops.script.reload() operator. Granted, it's not documented very well.

That's good advice. I've gotten so used to just Googling things and usually finding the answer on stackoverflow, I forgot about direct searching.
For some reason Google doesn't index the Blender documentation, which would probably trouble more newcomers to Blender.

if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process.

Well, if you keep multiple non-relevant add-ons enabled while you're developing, I don't think this is an issue with reloading. I think it's an issue between keyboard & chair.

While in most case you'd be right, there are people who develop add-ons, that rely on / add functionality to add-ons of others. It is not always possible to merge this into 2 add-ons.
You could say, you could help speed-up that add-on, but some add-on creators are satisfied with their solution of whatever reason they have to not accept your contribution.
Also, it's possible that the add-on you rely on is good, but only requires a long initialization time (e.g. setup TCP connections to programs outside Blender, not so weird if you want to visualize data from a Robot).

The situation at the moment is:

A function that can reload single add-ons, you're not supposed to use (bpy.ops.preferences.addon_enable(module='your_module'))
A function that can only reload all add-ons (bpy.ops.script.reload())

Is it a huge problem? Not really. However, to me it feels like poor design that you can't just reload 1 add-on. Even if in most cases reloading all add-ons won't be much of a slowdown.
Would you say it makes sense if there is a function that can reload a (list of) specific add-ons? Given it's not too much effort to implement.

p.s. Could someone point me to the reload script?

There is no link from the documentation to the code.
Searching importlib doesn't find me anything on https://developer.blender.org/diffusion/B/browse/master/
Googling (Startpage) site:https://developer.blender.org/ importlib doesn't show up anything either.

p.p.s
Asked for referencing the code by making the function names on the documentation page clickable: https://developer.blender.org/T67674

In #66924#729380, @ideasman42 wrote:
For development, you can:
import importlib
importlib.reload(your_addon_name)

You could even add this code to an operator in your add-on and bind it to a menu item or button, then it's literally a one-click reload for only your add-on.

Yeah, I did this. Is slightly less smooth if you're bugs are in the registering of the modules, which breaks the unloading, although that could have been because of me using addon_enable :')

> In #66924#730083, @dr.sybren wrote: >> In #66924#729370, @NumesSanguis-3 wrote: >> I would assume the most knowledgeable Python programmers are active here > > Not all of them, and not all the time. Just search the Python API documentation for "reload script" and you'll find the `bpy.ops.script.reload()` operator. Granted, it's not documented very well. That's good advice. I've gotten so used to just Googling things and usually finding the answer on stackoverflow, I forgot about direct searching. For some reason Google doesn't index the Blender documentation, which would probably trouble more newcomers to Blender. >> if (multiple) non-relevant add-on(s) are slow at initializing, it will slow down the development process. > > Well, if you keep multiple non-relevant add-ons enabled while you're developing, I don't think this is an issue with reloading. I think it's an issue between keyboard & chair. While in most case you'd be right, there are people who develop add-ons, that rely on / add functionality to add-ons of others. It is not always possible to merge this into 2 add-ons. You could say, you could help speed-up that add-on, but some add-on creators are satisfied with their solution of whatever reason they have to not accept your contribution. Also, it's possible that the add-on you rely on is good, but only requires a long initialization time (e.g. setup TCP connections to programs outside Blender, not so weird if you want to visualize data from a Robot). The situation at the moment is: - A function that can reload single add-ons, you're not supposed to use (`bpy.ops.preferences.addon_enable(module='your_module')`) - A function that can only reload all add-ons (`bpy.ops.script.reload()`) Is it a huge problem? Not really. However, to me it feels like poor design that you can't just reload 1 add-on. Even if in most cases reloading all add-ons won't be much of a slowdown. Would you say it makes sense if there is a function that can reload a (list of) specific add-ons? Given it's not too much effort to implement. p.s. Could someone point me to the reload script? - There is no link from the documentation to the code. - Searching `importlib` doesn't find me anything on https://developer.blender.org/diffusion/B/browse/master/ - Googling (Startpage) `site:https://developer.blender.org/ importlib` doesn't show up anything either. p.p.s Asked for referencing the code by making the function names on the documentation page clickable: https://developer.blender.org/T67674 >> In #66924#729380, @ideasman42 wrote: >> For development, you can: >> >> ``` >> import importlib >> importlib.reload(your_addon_name) >> ``` > > You could even add this code to an operator in your add-on and bind it to a menu item or button, then it's literally a one-click reload for only your add-on. Yeah, I did this. Is slightly less smooth if you're bugs are in the registering of the modules, which breaks the unloading, although that could have been because of me using `addon_enable` :')

blender-admin commented

2019-08-23 23:10:41 +02:00

This issue was referenced by blender/blender@ee4ec69b28

This issue was referenced by blender/blender@ee4ec69b28047629a1c153af356757a8fac5cee9

Antonio Vazquez commented

2019-08-23 23:14:20 +02:00

Changed status from 'Archived' to: 'Resolved'