Removing Addon with compiled modules(e.g. cython) on Windows will cause PermissionError: [WinError 5] Access is denied #77837

New Issue

MACHIN3 · 2020-06-13T17:36:16+02:00

MACHIN3 commented

2020-06-13 17:36:16 +02:00

System Information
Operating system: Linux-4.15.0-101-generic-x86_64-with-debian-buster-sid 64 Bits
Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 435.21

Blender Version
Broken: version: 2.83.0, branch: master, commit date: 2020-06-03 14:38, hash: 211b6c29f7

Short description of error
Any addon with complied modules can't be properly removed on Windows using bpy.ops.preferences.addon_remove()

This error will pop up:

Error: Traceback (most recent call last):
  File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\scripts\startup\bl_operators\userpref.py", line 775, in execute
    shutil.rmtree(path)
  File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 395, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 400, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 398, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\x\\AppData\\Roaming\\Blender Foundation\\Blender\\2.83\\scripts\\addons\\RemoveAddonException\

Exact steps for others to reproduce the error

install the RemoveAddonException addon from the zip (tiny addon to test that has a single operator called "MACHIN3: Printer", which prints using a custom print function in utils/print.py[d])
optionally confirm its working by running MACHIN3: Printer from the operator search menu, then take look at the terminal
remove the addon from the addon prefs
RemoveAddonException.zip

Thoughts

Addon removal is done in /opt/blender 2.83/2.83/scripts/startup/bl_operators/userpref.py, specifically in class PREFERENCES_OT_addon_remove().

in line 776, it's doing this:
shutil.rmtree(path)

shutil.rmtree() takes two optional arguments:

ignore_errors
onerror

If ignore_errors were to be set to True, the addon removal would finish without the exception, which in my view would be preferable to the current implementation.
It won't completely remove the addon folder however, as the compiled module will remain in the addon's folder.

Using onerror a custom function can be passed in to deal with the problem. It turns out, that while python doesn't like the module to be removed on Windows, it doesn't mind it being moved elsewhere. So what could be done is this:

def move_file(func, path, err):
     os.rename(path, os.path.join(bpy.utils.resources('SCRIPTS'), 'trash', os.path.basename(path)))
...
shutil.rmtree(path, onerror=move_file)

In this example. the file is moved to a trash folder in the scripts users resources, so parallel to the addons folder.
Doing this Blender is able to completely remove the addon folder.
Perhaps, Blender could then also remove that trash folder when closing, or the next time it starts and before any addons are registered?

IMO, at the very least, shutil.rmtree() should use the ignore_errors argument.
The best solution would be to have python release the module somehow, but based on my research, that's not possible? Any input is appreciated.

References (aka other people having this issue)

**System Information** Operating system: Linux-4.15.0-101-generic-x86_64-with-debian-buster-sid 64 Bits Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 435.21 **Blender Version** Broken: version: 2.83.0, branch: master, commit date: 2020-06-03 14:38, hash: `211b6c29f7` **Short description of error** Any addon with complied modules can't be properly removed on Windows using `bpy.ops.preferences.addon_remove()` This error will pop up: ``` Error: Traceback (most recent call last): File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\scripts\startup\bl_operators\userpref.py", line 775, in execute shutil.rmtree(path) File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 516, in rmtree return _rmtree_unsafe(path, onerror) File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 395, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 400, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Program Files\Blender Foundation\Blender 2.83\2.83\python\lib\shutil.py", line 398, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 5] Access is denied: 'C:\\Users\\x\\AppData\\Roaming\\Blender Foundation\\Blender\\2.83\\scripts\\addons\\RemoveAddonException\ ``` **Exact steps for others to reproduce the error** * install the RemoveAddonException addon from the zip (tiny addon to test that has a single operator called "MACHIN3: Printer", which prints using a custom print function in `utils/print.py[d]`) * optionally confirm its working by running `MACHIN3: Printer` from the operator search menu, then take look at the terminal * remove the addon from the addon prefs [RemoveAddonException.zip](https://archive.blender.org/developer/F8615784/RemoveAddonException.zip) ---- **Thoughts** Addon removal is done in `/opt/blender 2.83/2.83/scripts/startup/bl_operators/userpref.py`, specifically in `class PREFERENCES_OT_addon_remove()`. in line 776, it's doing this: `shutil.rmtree(path)` shutil.rmtree() takes two optional arguments: 1. `ignore_errors` 2. `onerror` If `ignore_errors` were to be set to `True`, the addon removal would finish without the exception, which in my view would be preferable to the current implementation. It won't completely remove the addon folder however, as the compiled module will remain in the addon's folder. Using `onerror` a custom function can be passed in to deal with the problem. It turns out, that while python doesn't like the module to be removed on Windows, it doesn't mind it being moved elsewhere. So what could be done is this: ``` def move_file(func, path, err): os.rename(path, os.path.join(bpy.utils.resources('SCRIPTS'), 'trash', os.path.basename(path))) ... shutil.rmtree(path, onerror=move_file) ``` In this example. the file is moved to a trash folder in the scripts users resources, so parallel to the addons folder. Doing this Blender is able to completely remove the addon folder. Perhaps, Blender could then also remove that trash folder when closing, or the next time it starts and before any addons are registered? IMO, at the very least, `shutil.rmtree()` should use the `ignore_errors` argument. The best solution would be to have python release the module somehow, but based on my research, that's not possible? Any input is appreciated. **References (aka other people having this issue)** * https://discuss.python.org/t/how-to-unload-native-extensions-pyd-on-windows/3749 * https://python-forum.io/Thread-How-To-Unload-Windows-Native-Extension-pyd * https://github.com/rlguy/Blender-FLIP-Fluids/wiki/Addon-Installation-and-Uninstallation

MACHIN3 commented

2020-06-13 17:36:16 +02:00

Added subscriber: @MACHIN3

MACHIN3 changed title from ~~Removing Addon with compiled modules(ex. cython) on WIndows will cause PermissionError: [WinError 5] Access is denied:~~ to Removing Addon with compiled modules(e.g. cython) on WIndows will cause PermissionError: [WinError 5] Access is denied:

2020-06-13 17:36:52 +02:00

MACHIN3 changed title from ~~Removing Addon with compiled modules(e.g. cython) on WIndows will cause PermissionError: [WinError 5] Access is denied:~~ to Removing Addon with compiled modules(e.g. cython) on Windows will cause PermissionError: [WinError 5] Access is denied

2020-06-13 17:48:30 +02:00

Ryan Guy commented

2020-06-13 19:47:19 +02:00

Added subscriber: @rlguy

Ryan Guy commented

2020-06-13 19:47:19 +02:00

This issue also affects the FLIP Fluids addon and is a common error when our users try to uninstall our addon. If there is a solution or workaround to to this issue, it would be greatly appreciated!

Ray molenkamp commented

2020-06-13 20:09:09 +02:00

Added subscriber: @LazyDodo

Ray molenkamp commented

2020-06-13 20:09:09 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Ray molenkamp closed this issue

2020-06-13 20:09:09 +02:00

Ray molenkamp self-assigned this 2020-06-13 20:09:09 +02:00

Ray molenkamp commented

2020-06-13 20:09:09 +02:00

There's not a whole lot we can do about this from the blender side, i took a quick peek at the python source and they just seem to lack any kind of management that would allow you to do this.

you may be able to forcefully unload the library with ctypes and forcefully calling FreeLibrary a few times in a row, but that'll be pretty far off the supported path for both blender and python and could blow up in your face given python may not enjoy the dll being gone at shutdown time, you may have more control over this in a pure ctypes based solution (rather than a python module solution)

either way, not a bug we can fix in blender, so by the tracker rules i'll have to close this ticket.

There's not a whole lot we can do about this from the blender side, i took a quick peek at the python source and they just seem to lack any kind of management that would allow you to do this. you may be able to forcefully unload the library with ctypes and forcefully calling FreeLibrary a few times in a row, but that'll be pretty far off the supported path for both blender and python and could blow up in your face given python may not enjoy the dll being gone at shutdown time, you may have more control over this in a pure ctypes based solution (rather than a python module solution) either way, not a bug we can fix in blender, so by the tracker rules i'll have to close this ticket.

MACHIN3 commented

2020-06-13 20:15:29 +02:00

Why not at least prevent the error using the ignore_errors arg?

Why not at least prevent the error using the `ignore_errors` arg?

Ray molenkamp commented

2020-06-13 20:23:35 +02:00

Changed status from 'Archived' to: 'Needs Triage'

Ray molenkamp reopened this issue

2020-06-13 20:23:35 +02:00

Ray molenkamp commented

2020-06-13 20:23:35 +02:00

Sorry, overlooked that bit, i'll leave that for the python guys to decide, for unloading the library there is really nothing we can do.

Bent Hillerkus commented

2020-06-13 21:42:41 +02:00

Added subscriber: @bent

Evan Wilson commented

2020-06-14 22:08:10 +02:00

Added subscriber: @EAW

Evan Wilson commented

2020-06-14 22:08:10 +02:00

In #77837#953299, @rlguy wrote:
This issue also affects the FLIP Fluids addon and is a common error when our users try to uninstall our addon. If there is a solution or workaround to to this issue, it would be greatly appreciated!

I notice that your uninstall instructions don't tell users to first disable the addon, restart, and then click remove. That is how I workaround this issue myself.

> In #77837#953299, @rlguy wrote: > This issue also affects the FLIP Fluids addon and is a common error when our users try to uninstall our addon. If there is a solution or workaround to to this issue, it would be greatly appreciated! I notice that your [uninstall instructions](https://github.com/rlguy/Blender-FLIP-Fluids/wiki/Addon-Installation-and-Uninstallation#uninstalling-the-addon) don't tell users to first disable the addon, restart, and then click remove. That is how I workaround this issue myself.

Bent Hillerkus commented

2020-06-15 02:15:31 +02:00

In #77837#953659, @EAW wrote:

In #77837#953299, @rlguy wrote:
This issue also affects the FLIP Fluids addon and is a common error when our users try to uninstall our addon. If there is a solution or workaround to to this issue, it would be greatly appreciated!

I notice that your uninstall instructions don't tell users to first disable the addon, restart, and then click remove. That is how I workaround this issue myself.

This is what Animation Nodes tells the user aswell. Not really fond of it though, I wish Blender had a more user friendly and / or unified way of doing it.

> In #77837#953659, @EAW wrote: >> In #77837#953299, @rlguy wrote: >> This issue also affects the FLIP Fluids addon and is a common error when our users try to uninstall our addon. If there is a solution or workaround to to this issue, it would be greatly appreciated! > > I notice that your [uninstall instructions](https://github.com/rlguy/Blender-FLIP-Fluids/wiki/Addon-Installation-and-Uninstallation#uninstalling-the-addon) don't tell users to first disable the addon, restart, and then click remove. That is how I workaround this issue myself. This is what Animation Nodes tells the user aswell. Not really fond of it though, I wish Blender had a more user friendly and / or unified way of doing it.

Germano Cavalcante commented

2020-06-17 00:07:01 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Germano Cavalcante commented

2020-06-17 00:07:40 +02:00

Added subscriber: @mano-wii

Ray molenkamp removed their assignment 2020-07-03 22:23:14 +02:00

Arnaud Couturier commented

2020-07-17 13:37:39 +02:00

Added subscriber: @piiichan

Brian Savery (AMD) commented

2020-09-09 00:52:06 +02:00

Added subscriber: @BrianSavery

Brian Savery (AMD) commented

2020-09-09 00:52:06 +02:00

Also adding this being an issue with Radeon ProRender addon (only windows). I've tried forcefully unloading DLL but so far unsuccessfully.

Moritz Brückner commented

2021-05-30 15:44:15 +02:00

Added subscriber: @timodriaan

Christian Stolze commented

2021-12-20 09:08:51 +01:00

Added subscriber: @regcs

Christian Stolze commented

2021-12-21 17:32:34 +01:00

This comment was removed by @regcs

*This comment was removed by @regcs*

Christian Stolze commented

2021-12-29 02:21:46 +01:00

I submitted a patch (D13685), which follows the suggestion by @MACHIN3:

move the files that can't be deleted to a new folder /scripts/addons_trash/
every time Blender is initialized it deletes the trash folder (if it exists)

NOTE:
Some specific python modules still may need to be unloaded manually in the add-on's unregister() call. I noticed that, e.g., the logging module demands a logging.shutdown().

I submitted a patch ([D13685](https://archive.blender.org/developer/D13685)), which follows the suggestion by @MACHIN3: - move the files that can't be deleted to a new folder `/scripts/addons_trash/` - every time Blender is initialized it deletes the trash folder (if it exists) **NOTE:** Some specific python modules still may need to be unloaded manually in the add-on's `unregister()` call. I noticed that, e.g., the `logging` module demands a `logging.shutdown()`.

Zach Eastin commented

2022-10-19 21:39:33 +02:00

Added subscriber: @Zeastin

Juan Hernandez commented

2022-12-05 01:52:26 +01:00

Added subscriber: @ArmoredWolf

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Removing Addon with compiled modules(e.g. cython) on Windows will cause PermissionError: [WinError 5] Access is denied #77837