Blender is losing reference to collection property item when used in a recursive function? #61297

New Issue

Piotr Kowalczyk · 2019-02-07T23:02:10+01:00

Piotr Kowalczyk commented

2019-02-07 23:02:10 +01:00

System Information
Operating system: Ubuntu 18.04.1 LTS
Graphics card: GeForce GT 610. But I am not doing any graphics, so it's unrelated

Blender Version
Broken:
2.79b release
and
blender-2.80-9c68ac0448b6-linux-glibc224-x86_64 from 2019-02-07

Worked: none

Short description of error
I am creating a new item in collection property, and I am adding data to this item. After a recursive call of the same function, this item is not accessible. Details of errors are in the python code comments.

Exact steps for others to reproduce the error
Base on the default startup blend file, load my python script to a text editor and run a script. No errors yet. Find the button "Load tree from json" (properties in 3d viewport) and press it.
An error does not appear when pressed again. You will have to restart blender to repeat the error.

Script
error2.8.py

Crash report from blender 2.8
blender.crash.txt

**System Information** Operating system: Ubuntu 18.04.1 LTS Graphics card: GeForce GT 610. But I am not doing any graphics, so it's unrelated **Blender Version** Broken: 2.79b release and blender-2.80-9c68ac0448b6-linux-glibc224-x86_64 from 2019-02-07 Worked: none **Short description of error** I am creating a new item in collection property, and I am adding data to this item. After a recursive call of the same function, this item is not accessible. Details of errors are in the python code comments. **Exact steps for others to reproduce the error** Base on the default startup blend file, load my python script to a text editor and run a script. No errors yet. Find the button "Load tree from json" (properties in 3d viewport) and press it. An error does not appear when pressed again. You will have to restart blender to repeat the error. **Script** [error2.8.py](https://archive.blender.org/developer/F6547660/error2.8.py) **Crash report from blender 2.8** [blender.crash.txt](https://archive.blender.org/developer/F6547692/blender.crash.txt)

Piotr Kowalczyk commented

2019-02-07 23:02:10 +01:00

Added subscriber: @dopiotrko

Philipp Oeser commented

2019-02-08 11:34:58 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2019-02-08 11:34:58 +01:00

Changed status from 'Open' to: 'Resolved'

Philipp Oeser closed this issue

2019-02-08 11:34:58 +01:00

Philipp Oeser self-assigned this 2019-02-08 11:34:58 +01:00

Philipp Oeser commented

2019-02-08 11:34:58 +01:00

RNA_def_property: runtime property identifier "TreeItems.items" - this keyword is reserved by python

Since this is a reserved keyword, try renaming your property to something else:

class TreeItems(bpy.types.PropertyGroup):
    if bpy.app.version >= (2,80,0):
        name: bpy.props.StringProperty()
        id: bpy.props.IntProperty()
        str_id: bpy.props.StringProperty()
        _items: bpy.props.StringProperty(default='')

and change that elsewhere in the code as well, should be good to go then, see
T61297_phi.blend

Will close this as resolved, but feel free to comment again if issues persist...

`RNA_def_property: runtime property identifier "TreeItems.items" - this keyword is reserved by python` Since this is a reserved keyword, try renaming your property to something else: ``` class TreeItems(bpy.types.PropertyGroup): if bpy.app.version >= (2,80,0): name: bpy.props.StringProperty() id: bpy.props.IntProperty() str_id: bpy.props.StringProperty() _items: bpy.props.StringProperty(default='') ``` and change that elsewhere in the code as well, should be good to go then, see [T61297_phi.blend](https://archive.blender.org/developer/F6547885/T61297_phi.blend) Will close this as resolved, but feel free to comment again if issues persist...

Piotr Kowalczyk commented

2019-02-08 19:26:55 +01:00

I regret to write: it is not it :(.
I changed init to sub_init and I steel get TypeError: unsubscriptable object.
error2.8_sub_item.py
Regards

I regret to write: it is not it :(. I changed init to sub_init and I steel get TypeError: unsubscriptable object. [error2.8_sub_item.py](https://archive.blender.org/developer/F6551166/error2.8_sub_item.py) Regards

Philipp Oeser commented

2019-02-08 21:01:23 +01:00

Changed status from 'Resolved' to: 'Open'

Philipp Oeser reopened this issue

2019-02-08 21:01:23 +01:00

Philipp Oeser commented

2019-02-08 21:01:23 +01:00

That was probably a bit too quick on my side, need to look at that again (on Monday)...

Piotr Kowalczyk commented

2019-02-12 12:14:27 +01:00

?

Philipp Oeser commented

2019-02-12 13:11:37 +01:00

Added subscribers: @mont29, @ideasman42

Philipp Oeser commented

2019-02-12 13:11:37 +01:00

Can confirm the crash (when accessing/printing the current_item after the recursive call)

1   BPy_IDGroup_WrapData             idprop_py_api.c      143  0x23aac17      
2   BPy_Wrap_GetValues               idprop_py_api.c      963  0x23acb99      
3   pyrna_struct_values              bpy_rna.c            3394 0x2378203      
4   _PyMethodDef_RawFastCallKeywords                           0x7ffff79c4561 
5   _PyCFunction_FastCallKeywords                              0x7ffff79c4700 
6   _PyEval_EvalFrameDefault                                   0x7ffff7a33bd3 
7   _PyEval_EvalCodeWithName                                   0x7ffff797df28 
8   _PyFunction_FastCallKeywords                               0x7ffff79c3e81 
9   _PyEval_EvalFrameDefault                                   0x7ffff7a2ea9c 
10  _PyEval_EvalCodeWithName                                   0x7ffff797df28 
11  _PyFunction_FastCallKeywords                               0x7ffff79c3e81 
12  _PyEval_EvalFrameDefault                                   0x7ffff7a2ea9c 
13  _PyFunction_FastCallDict                                   0x7ffff797ef2a 
14  bpy_class_call                   bpy_rna.c            7919 0x237fda0      
15  rna_operator_execute_cb          rna_wm.c             1228 0x2ea0ad9      
16  wm_operator_invoke               wm_event_system.c    1347 0x1ca859a      
17  wm_operator_call_internal        wm_event_system.c    1544 0x1ca8c20      
18  WM_operator_name_call_ptr        wm_event_system.c    1592 0x1ca8e43      
19  ui_apply_but_funcs_after         interface_handlers.c 769  0x1fc0cf7      
20  ui_handler_region_menu           interface_handlers.c 9855 0x1fd8ca2      
... <More>

However, this might be expected [to get into trouble after a recursive call], since the local variable is just a reference to the same object?
https://stackoverflow.com/questions/24572089/trouble-with-variable-scope-in-recursive-function-python

You are also using enumerate here [which is different from just looping over a list, but I am uncertain about the python internals...]
I made a 2nd try here:

replace the enumerate with just plain loop over list
made a deepcopy of the sub_branch prior to passing it to the recursive call
not sure if everything is behaving as expected, but could you check on this?
#61297.blend

Since I am really on shaky ground here, I would like to ask:
@mont29, @ideasman42 : is this expected? [to get into trouble when using enumerate + recursive function calls?] Is dropping enumerate (in favor of in list) plus making a deepcopy the desired/correct solution here?

Will mark this as Normal priority for now...

Can confirm the crash (when accessing/printing the `current_item` after the recursive call) ``` 1 BPy_IDGroup_WrapData idprop_py_api.c 143 0x23aac17 2 BPy_Wrap_GetValues idprop_py_api.c 963 0x23acb99 3 pyrna_struct_values bpy_rna.c 3394 0x2378203 4 _PyMethodDef_RawFastCallKeywords 0x7ffff79c4561 5 _PyCFunction_FastCallKeywords 0x7ffff79c4700 6 _PyEval_EvalFrameDefault 0x7ffff7a33bd3 7 _PyEval_EvalCodeWithName 0x7ffff797df28 8 _PyFunction_FastCallKeywords 0x7ffff79c3e81 9 _PyEval_EvalFrameDefault 0x7ffff7a2ea9c 10 _PyEval_EvalCodeWithName 0x7ffff797df28 11 _PyFunction_FastCallKeywords 0x7ffff79c3e81 12 _PyEval_EvalFrameDefault 0x7ffff7a2ea9c 13 _PyFunction_FastCallDict 0x7ffff797ef2a 14 bpy_class_call bpy_rna.c 7919 0x237fda0 15 rna_operator_execute_cb rna_wm.c 1228 0x2ea0ad9 16 wm_operator_invoke wm_event_system.c 1347 0x1ca859a 17 wm_operator_call_internal wm_event_system.c 1544 0x1ca8c20 18 WM_operator_name_call_ptr wm_event_system.c 1592 0x1ca8e43 19 ui_apply_but_funcs_after interface_handlers.c 769 0x1fc0cf7 20 ui_handler_region_menu interface_handlers.c 9855 0x1fd8ca2 ... <More> ``` However, this might be expected [to get into trouble after a recursive call], since the local variable is just a reference to the same object? https://stackoverflow.com/questions/24572089/trouble-with-variable-scope-in-recursive-function-python You are also using `enumerate` here [which is different from just looping over a list, but I am uncertain about the python internals...] I made a 2nd try here: - replace the enumerate with just plain loop over list - made a deepcopy of the `sub_branch` prior to passing it to the recursive call - not sure if everything is behaving as expected, but could you check on this? [#61297.blend](https://archive.blender.org/developer/F6589319/T61297.blend) Since I am really on shaky ground here, I would like to ask: @mont29, @ideasman42 : is this expected? [to get into trouble when using enumerate + recursive function calls?] Is dropping `enumerate` (in favor of `in list`) plus making a deepcopy the desired/correct solution here? Will mark this as `Normal` priority for now...

Philipp Oeser removed their assignment 2019-02-12 13:11:56 +01:00

Piotr Kowalczyk commented

2019-02-12 14:08:23 +01:00

This .blend file You attached is (as far as I know) default init blend file, so I can't check what You have changed. Please attach the changed .py file.
I followed Your directions:

I removed enumerate (It was there for testing purposes anyway), but this did not solve the problem.
I assume that Your step 2 was passing 'deepcopy(branch['sub_items'])' instead of ''branch['sub_items'] into the recursive function. It did not solve the problem for me (did it for You?). Please remember that (like I wrote in the comments of my py file) calling operator again does not replicate the error: "I have to restart blender to repeat the error". And after that changes error is still there. Also, the problem from Your link is little different from mine, because my problem is with 'current_item' variable (not with ''branch['sub_items']"). I am not passing 'current_item' to the recursive call, and as far as I know my current_tems variables from every scope point to a different object, coz I create it in the scope, not passing it as a parameter.
Regards.

This .blend file You attached is (as far as I know) default init blend file, so I can't check what You have changed. Please attach the changed .py file. I followed Your directions: - I removed enumerate (It was there for testing purposes anyway), but this did not solve the problem. - I assume that Your step 2 was passing 'deepcopy(branch['sub_items'])' instead of ''branch['sub_items'] into the recursive function. It did not solve the problem for me (did it for You?). Please remember that (like I wrote in the comments of my py file) calling operator again does not replicate the error: "I have to restart blender to repeat the error". And after that changes error is still there. Also, the problem from Your link is little different from mine, because my problem is with 'current_item' variable (not with ''branch['sub_items']"). I am not passing 'current_item' to the recursive call, and as far as I know my current_tems variables from every scope point to a different object, coz I create it in the scope, not passing it as a parameter. Regards.

Piotr Kowalczyk commented

2019-02-12 14:34:57 +01:00

Sorry, Your blend file was not empty, my mistake. But the error still occurs.

Philipp Oeser commented

2019-02-12 14:42:24 +01:00

T61297_____.blend
@dopiotrko : changed py is in that file (just didnt rename it -- e.g. look for _sub_branch = deepcopy(sub_branch))
and yes, that solved the error on my side...

I can open my file, run the script, run the operator without errors
I can restart blender, do the above, without errors

I can only repeat myself: I am on shaky ground here, if the above file doesnt work for you, I am probably out of ideas...

[T61297_____.blend](https://archive.blender.org/developer/F6590033/T61297_____.blend) @dopiotrko : changed py is in that file (just didnt rename it -- e.g. look for `_sub_branch = deepcopy(sub_branch)`) and yes, that solved the error on my side... - I can open my file, run the script, run the operator without errors - I can restart blender, do the above, without errors I can only repeat myself: I am on shaky ground here, if the above file doesnt work for you, I am probably out of ideas...

Bastien Montagne commented

2019-02-12 14:43:56 +01:00

Changed status from 'Open' to: 'Archived'

Bastien Montagne closed this issue

2019-02-12 14:43:56 +01:00

Bastien Montagne self-assigned this 2019-02-12 14:43:56 +01:00

Bastien Montagne commented

2019-02-12 14:43:57 +01:00

First of all, this is not a bug report at all, and should not be handled here. We have a site for that kind of question: https://devtalk.blender.org/

Would be a bug report if you had done investigation, and generated from it a very simple demo script of the issue, instead of giving us your full project one:

import bpy


class TreeItems(bpy.types.PropertyGroup):

name: bpy.props.StringProperty()

def recursive_idprop_user(context, counter=10):

counter -= 1
if counter == 0:
return
curr_item = context.scene.tree_items.add()
recursive_idprop_user(context, counter)
curr_item.name = str(counter)
print(curr_item.values())



class FooBar_OT_test(bpy.types.Operator):

bl_idname = "foobar.test"
bl_label = "Test recursive IDProp handling"

def execute(self, context):
recursive_idprop_user(context, 10)
return {"FINISHED"}



def register():

bpy.utils.register_class(TreeItems)
bpy.utils.register_class(FooBar_OT_test)

bpy.types.Scene.tree_items = bpy.props.CollectionProperty(type=TreeItems)



if __name__ == "__main__":

register()

That should also have made it obvious that there is no bug at all here, you are just keeping reference of some Blender data while modifying container of that data, which is always dangerous. That simple modification is enough to make example work:

import bpy


class TreeItems(bpy.types.PropertyGroup):

name: bpy.props.StringProperty()

def recursive_idprop_user(context, counter=10):

counter -= 1
if counter == 0:
return
curr_item = context.scene.tree_items.add()
curr_item_idx = len(context.scene.tree_items) - 1
recursive_idprop_user(context, counter)
curr_item = context.scene.tree_items[curr_item_idx]
curr_item.name = str(counter)
print(curr_item.values())



class FooBar_OT_test(bpy.types.Operator):

bl_idname = "foobar.test"
bl_label = "Test recursive IDProp handling"

def execute(self, context):
recursive_idprop_user(context, 10)
return {"FINISHED"}



def register():

bpy.utils.register_class(TreeItems)
bpy.utils.register_class(FooBar_OT_test)

bpy.types.Scene.tree_items = bpy.props.CollectionProperty(type=TreeItems)



if __name__ == "__main__":

register()

Did not check your actual code, but pretty sure this will fix it. Also, will update the doc the make this point more obvious, we describe several common case of that issue but it's not enough standing out that this can happen with pretty much any data owned by Blender…

First of all, this is not a bug report at all, and should not be handled here. We have a site for that kind of question: https://devtalk.blender.org/ Would be a bug report if you had done investigation, and generated from it a very simple demo script of the issue, instead of giving us your full project one: ```lines=20, lang=python import bpy class TreeItems(bpy.types.PropertyGroup): ``` name: bpy.props.StringProperty() ``` def recursive_idprop_user(context, counter=10): ``` counter -= 1 if counter == 0: return curr_item = context.scene.tree_items.add() recursive_idprop_user(context, counter) curr_item.name = str(counter) print(curr_item.values()) ``` class FooBar_OT_test(bpy.types.Operator): ``` bl_idname = "foobar.test" bl_label = "Test recursive IDProp handling" ``` ``` def execute(self, context): recursive_idprop_user(context, 10) return {"FINISHED"} ``` def register(): ``` bpy.utils.register_class(TreeItems) bpy.utils.register_class(FooBar_OT_test) ``` ``` bpy.types.Scene.tree_items = bpy.props.CollectionProperty(type=TreeItems) ``` if __name__ == "__main__": ``` register() ``` ``` That should also have made it obvious that there is no bug at all here, you are just keeping reference of some Blender data while modifying container of that data, which is always dangerous. That simple modification is enough to make example work: ```lines=20, lang=python import bpy class TreeItems(bpy.types.PropertyGroup): ``` name: bpy.props.StringProperty() ``` def recursive_idprop_user(context, counter=10): ``` counter -= 1 if counter == 0: return curr_item = context.scene.tree_items.add() curr_item_idx = len(context.scene.tree_items) - 1 recursive_idprop_user(context, counter) curr_item = context.scene.tree_items[curr_item_idx] curr_item.name = str(counter) print(curr_item.values()) ``` class FooBar_OT_test(bpy.types.Operator): ``` bl_idname = "foobar.test" bl_label = "Test recursive IDProp handling" ``` ``` def execute(self, context): recursive_idprop_user(context, 10) return {"FINISHED"} ``` def register(): ``` bpy.utils.register_class(TreeItems) bpy.utils.register_class(FooBar_OT_test) ``` ``` bpy.types.Scene.tree_items = bpy.props.CollectionProperty(type=TreeItems) ``` if __name__ == "__main__": ``` register() ``` ``` Did not check your actual code, but pretty sure this will fix it. Also, will update [the doc](https://docs.blender.org/api/master/info_gotcha.html#help-my-script-crashes-blender) the make this point more obvious, we describe several common case of that issue but it's not enough standing out that this can happen with pretty much *any* data owned by Blender…

Piotr Kowalczyk commented

2019-02-12 14:57:43 +01:00

This comment was removed by @dopiotrko

*This comment was removed by @dopiotrko*

Philipp Oeser commented

2019-02-12 15:05:22 +01:00

thanx @mont29 for clearing that up! (sorry for my uneducated fiddeling here...)