ID user decrement error when removing object which is referred to with a PointerProperty #81345

New Issue

Pauan · 2020-10-01T09:02:51+02:00

Pauan commented

2020-10-01 09:02:51 +02:00

System Information
Operating system: Windows 10 Home 1909 (64-bit)
Graphics card: NVidia GeForce GTX 1050 Ti

Blender Version
Broken: 2.90.0
Worked: Unknown

Short description of error

I have added a PointerProperty to the EditBone type. This PointerProperty refers to an object (which is separate from the armature).

When removing the separate object, it causes the following error in the console:

ERROR (bke.lib_id): C:\b\buildbot-worker-windows\windows_290\blender.git\source\blender\blenkernel\intern\lib_id.c:288 id_us_min: ID user decrement error:  (from '[Main]'): 0 <= 0

Eventually this causes Blender to crash with a Windows fatal exception: access violation error.

Exact steps for others to reproduce the error

Starting with a new General Blend file:

Run the following code as a Text (it will automatically create an armature and panel):

import bpy

def make_all(context):
    bpy.ops.object.mode_set(mode='EDIT')

    armature = context.active_object

    for bone in armature.data.edit_bones:
        mesh = bpy.data.meshes.new(name="Test")
        object = bpy.data.objects.new("Test", mesh)
        context.collection.objects.link(object)
        bone.test = object


def remove_all(context):
    bpy.ops.object.mode_set(mode='EDIT')

    armature = context.active_object

    for bone in armature.data.edit_bones:
        if bone.test:
            bpy.data.objects.remove(bone.test)
            bone.test = None


def update(self, context):
    if self.enabled:
        print("Enabling")
        make_all(context)

    else:
        print("Disabling")
        remove_all(context)


class Panel(bpy.types.Panel):
    bl_idname = "DATA_PT_test"
    bl_label = "Test"
    bl_space_type = 'VIEW_3D'
    bl_region_type = 'UI'
    bl_category = "Item"
    bl_options = set()

    @classmethod
    def poll(cls, context):
        return (context.active_object.type == 'ARMATURE')

    def draw(self, context):
        self.layout.prop(context.active_object.data, "enabled")


bpy.types.Armature.enabled = bpy.props.BoolProperty(name="Test enabled", default=False, update=update)
bpy.types.EditBone.test = bpy.props.PointerProperty(type=bpy.types.Object)

bpy.utils.register_class(Panel)

# Create new armature
bpy.ops.object.armature_add()

Press the N key to open the sidebar.
In the Item category, find the Test panel.
Click the "Test enabled" checkbox 4 or more times. This triggers the ID user decrement error.

I have also attached a simple .blend file. You can simply open it and then click the "Test enabled" checkbox 4 or more times.

Test.blend

**System Information** Operating system: Windows 10 Home 1909 (64-bit) Graphics card: NVidia GeForce GTX 1050 Ti **Blender Version** Broken: 2.90.0 Worked: Unknown **Short description of error** I have added a PointerProperty to the EditBone type. This PointerProperty refers to an object (which is separate from the armature). When removing the separate object, it causes the following error in the console: ``` ERROR (bke.lib_id): C:\b\buildbot-worker-windows\windows_290\blender.git\source\blender\blenkernel\intern\lib_id.c:288 id_us_min: ID user decrement error: (from '[Main]'): 0 <= 0 ``` Eventually this causes Blender to crash with a `Windows fatal exception: access violation` error. **Exact steps for others to reproduce the error** Starting with a new General Blend file: 1. Run the following code as a Text (it will automatically create an armature and panel): ``` import bpy def make_all(context): bpy.ops.object.mode_set(mode='EDIT') armature = context.active_object for bone in armature.data.edit_bones: mesh = bpy.data.meshes.new(name="Test") object = bpy.data.objects.new("Test", mesh) context.collection.objects.link(object) bone.test = object def remove_all(context): bpy.ops.object.mode_set(mode='EDIT') armature = context.active_object for bone in armature.data.edit_bones: if bone.test: bpy.data.objects.remove(bone.test) bone.test = None def update(self, context): if self.enabled: print("Enabling") make_all(context) else: print("Disabling") remove_all(context) class Panel(bpy.types.Panel): bl_idname = "DATA_PT_test" bl_label = "Test" bl_space_type = 'VIEW_3D' bl_region_type = 'UI' bl_category = "Item" bl_options = set() @classmethod def poll(cls, context): return (context.active_object.type == 'ARMATURE') def draw(self, context): self.layout.prop(context.active_object.data, "enabled") bpy.types.Armature.enabled = bpy.props.BoolProperty(name="Test enabled", default=False, update=update) bpy.types.EditBone.test = bpy.props.PointerProperty(type=bpy.types.Object) bpy.utils.register_class(Panel) # Create new armature bpy.ops.object.armature_add() ``` 2. Press the N key to open the sidebar. 3. In the Item category, find the Test panel. 4. Click the "Test enabled" checkbox 4 or more times. This triggers the ID user decrement error. I have also attached a simple .blend file. You can simply open it and then click the "Test enabled" checkbox 4 or more times. [Test.blend](https://archive.blender.org/developer/F8943052/Test.blend)

Pauan commented

2020-10-01 09:02:51 +02:00

Added subscriber: @pauanyu_blender

Philipp Oeser commented

2020-10-01 09:47:30 +02:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2020-10-01 09:47:30 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2020-10-01 09:47:30 +02:00

Can confirm a crash, will check on this.

Philipp Oeser commented

2020-10-01 19:10:03 +02:00

There is one more user here than expected.

Not sure if #77557 (Python Operators that add/remove ID data without an undo step crash Blender) is related? Needs more investigation.

There is one more user here than expected. Not sure if #77557 (Python Operators that add/remove ID data without an undo step crash Blender) is related? Needs more investigation.

Bastien Montagne commented

2020-10-02 13:54:01 +02:00

Added subscribers: @Sergey, @dr.sybren, @brecht, @mont29

Bastien Montagne self-assigned this 2020-10-02 13:54:01 +02:00

Bastien Montagne commented

2020-10-02 13:54:01 +02:00

This is actually a fairly serious issue. Root of it is that bone IDProperties are duplicated over two structures, Bone and EditBone.

@brecht, @Sergey, @dr.sybren, EditBone is defined in ED_armature.h, so we cannot access it from BKE code that deals with ID refcounting (especially the foreach looper over ID usages from lib_query.c and the matching callback for Armature IDs). Do you see any reason not to have that one defined in BKE_armature.h instead?

This is actually a fairly serious issue. Root of it is that bone IDProperties are duplicated over two structures, `Bone` and `EditBone`. @brecht, @Sergey, @dr.sybren, `EditBone` is defined in `ED_armature.h`, so we cannot access it from BKE code that deals with ID refcounting (especially the foreach looper over ID usages from `lib_query.c` and the matching callback for Armature IDs). Do you see any reason not to have that one defined in `BKE_armature.h` instead?

Brecht Van Lommel commented

2020-10-02 14:05:26 +02:00

Having it in BKE_armature.h seems fine to me.

Having it in `BKE_armature.h` seems fine to me.

Sybren A. Stüvel commented

2020-10-02 15:10:30 +02:00

BKE_armature.h already has a lot of pose-related functions. Maybe the EditBone definition could go into a new file BKE_armature_edit.h? That may keep things a bit better separated.

`BKE_armature.h` already has a lot of pose-related functions. Maybe the `EditBone` definition could go into a new file `BKE_armature_edit.h`? That may keep things a bit better separated.

Bastien Montagne commented

2020-10-02 15:20:12 +02:00

@dr.sybren It could at some point, but am not convinced it would help at all currently, that would be a new header file with that single struct into it, and then having to link it in many extra places… I would rather do that later, when/if we actually add or move some armature bone editing code into BKE area?

blender-admin commented

2020-10-02 15:38:02 +02:00

This issue was referenced by 619e52eb82

This issue was referenced by 619e52eb82744c9dc2a403a0aa12fa9e9141fe3b

blender-admin commented

2020-10-02 16:12:27 +02:00

This issue was referenced by f393452394

This issue was referenced by f3934523946962b807b0dd7e0863a437cfc56e27

Sybren A. Stüvel commented

2020-10-02 17:30:02 +02:00

Ok, in that case BKE_armature.h is fine too.

Ok, in that case `BKE_armature.h` is fine too.

Bastien Montagne commented

2020-10-02 17:34:41 +02:00

This actually revealed a nest of bugs regarding ID management and IDProperties of bones… Fixes are now incoming.

@pauanyu_blender Please note that manipulating data-blocks from update callback as you do in your script is extremely discouraged, it will basically break undo/redo process, among other potential severe issues, see the API documentation .

This actually revealed a nest of bugs regarding ID management and IDProperties of bones… Fixes are now incoming. @pauanyu_blender Please note that manipulating data-blocks from `update` callback as you do in your script is extremely discouraged, it will basically break undo/redo process, among other potential severe issues, see [the API documentation](https://docs.blender.org/api/master/info_gotcha.html#abusing-rna-property-callbacks) .

blender-admin commented

2020-10-02 17:36:13 +02:00

This issue was referenced by 4b36552967

This issue was referenced by 4b36552967bf55b1bf707acd3917280b42b12c52

blender-admin commented

2020-10-02 17:38:28 +02:00

This issue was referenced by 0db98b214d

This issue was referenced by 0db98b214d468864630fc9bba72be4ff9b2548e5

Bastien Montagne commented

2020-10-02 17:46:47 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Bastien Montagne closed this issue

2020-10-02 17:46:47 +02:00

Pauan commented

2020-10-02 18:24:54 +02:00

@mont29 Thanks!

Then what is the idiomatic way to create/remove objects in response to a property changing?
Are the problems only for creating new data blocks, or also manipulating properties on existing data blocks (e.g. changing Object.location)?

If update callbacks are so harmful, I think the docs should clearly show what the alternative is, rather than essentially telling the developer "you can't do that", which is not helpful.

(As a meta note, I've spent many hours reading through the addons documentation, StackOverflow, and the devtalk forums, but I still find it very hard to figure out how to do even very common things with the addon API. Some more usage examples would be great, though of course I know other things have higher priority.)

@mont29 Thanks! 1. Then what is the idiomatic way to create/remove objects in response to a property changing? 2. Are the problems only for creating new data blocks, or also manipulating properties on existing data blocks (e.g. changing `Object.location`)? If update callbacks are so harmful, I think the docs should clearly show what the alternative is, rather than essentially telling the developer "you can't do that", which is not helpful. (As a meta note, I've spent many hours reading through the addons documentation, StackOverflow, and the devtalk forums, but I still find it very hard to figure out how to do even very common things with the addon API. Some more usage examples would be great, though of course I know other things have higher priority.)

Bastien Montagne commented

2020-10-06 13:30:56 +02:00

In #81345#1027359, @pauanyu_blender wrote:
@mont29 Thanks!

Then what is the idiomatic way to create/remove objects in response to a property changing?

Idiomatic way to handle this case is simple: you don't.
In Blender, significant changes to data should happen through operators. Any other way to do it is at best a hack that might work within certain boundaries.

If you really, really want to react to a property value change, then either you run a modal operator and check said value on each modal run (which will happen within the main even loop). Or you use an application handler, but this might not work always very well either.

Are the problems only for creating new data blocks, or also manipulating properties on existing data blocks (e.g. changing Object.location)?

Creating or deleting data in general is a critical 'don't' in properties callbacks, be it an ID or some sub-data structure. Changing other parameters is generally fine, though it should preferably be inside the same data-block, otherwise you may break undo/redo stack.

If update callbacks are so harmful, I think the docs should clearly show what the alternative is, rather than essentially telling the developer "you can't do that", which is not helpful.

(As a meta note, I've spent many hours reading through the addons documentation, StackOverflow, and the devtalk forums, but I still find it very hard to figure out how to do even very common things with the addon API. Some more usage examples would be great, though of course I know other things have higher priority.)

Best usages examples are official add-ons usually… Agree this is not as friendly as a complete comprehensive set of advanced examples (you do know about our 'templates' available in the Text editor of Blender, right? ) would be nice, but this is indeed quiet a lot of work to create and maintain, and we simply do not have the man power for such thing currently.

> In #81345#1027359, @pauanyu_blender wrote: > @mont29 Thanks! > > 1. Then what is the idiomatic way to create/remove objects in response to a property changing? Idiomatic way to handle this case is simple: you don't. In Blender, significant changes to data should happen through operators. Any other way to do it is at best a hack that might work within certain boundaries. If you really, really want to react to a property value change, then either you run a modal operator and check said value on each `modal` run (which will happen within the main even loop). Or you use an [application handler](https://docs.blender.org/api/master/bpy.app.handlers.html), but this might not work always very well either. > 2. Are the problems only for creating new data blocks, or also manipulating properties on existing data blocks (e.g. changing `Object.location`)? Creating or deleting data in general is a critical 'don't' in properties callbacks, be it an ID or some sub-data structure. Changing other parameters is generally fine, though it should preferably be inside the same data-block, otherwise you may break undo/redo stack. > If update callbacks are so harmful, I think the docs should clearly show what the alternative is, rather than essentially telling the developer "you can't do that", which is not helpful. > > (As a meta note, I've spent many hours reading through the addons documentation, StackOverflow, and the devtalk forums, but I still find it very hard to figure out how to do even very common things with the addon API. Some more usage examples would be great, though of course I know other things have higher priority.) Best usages examples are official add-ons usually… Agree this is not as friendly as a complete comprehensive set of advanced examples (you do know about our 'templates' available in the Text editor of Blender, right? ) would be nice, but this is indeed quiet a lot of work to create and maintain, and we simply do not have the man power for such thing currently.

Pauan commented

2020-10-07 10:01:16 +02:00

@mont29 My add-on allows the user to create rigid body physics for bones:

https://github.com/Pauan/blender-rigid-body-bones

The user selects a bone, clicks on a checkbox to enable rigid body physics for that bone, and then changes various rigid body settings:

https://raw.githubusercontent.com/Pauan/blender-rigid-body-bones/master/Usage%2002.PNG

So when the checkbox is enabled, it must create a rigid body object, and when the checkbox is disabled, it must remove that object.

Having a checkbox to enable/disable things does not seem like an extreme case to me, it seems like a common thing to do. So it's very bizarre that you are saying that Blender does not support it.

And using a modal operator or app handler sounds terribly wasteful and bad performance, since it will run all the time, but I only need to run the code when the checkbox is changed.

It also sounds like an over-complication, since my use case is not complicated, and modal operators are clearly not intended for my use case.

If the problem is a lack of Operator, could I solve that by creating an Operator and then calling that Operator inside the update callback? That would be very easy for me to do.

If that doesn't work, then that is a failing in the Python API, and the Python API should be changed (or new APIs created). Perhaps an API that says, "run this callback on the next update tick, during a time where it is safe to create objects".

If you do not provide a correct API, people will instead abuse the current APIs. You cannot simply tell people "don't do that", because their use cases are legitimate and their goals are correct (even if their methods aren't), so they will do it no matter how much you say no. So if you don't want them using a bad API, you must instead provide a good API as an alternative.

you do know about our 'templates' available in the Text editor of Blender, right?

Yes, I did see them, but none of them match my use case (which is probably intentional, since it sounds like my use case is simply not supported, which is quite odd).

Would you accept pull requests improving the Python API documentation? I am new to the Python API, but I am a quite experienced programmer in general.

@mont29 My add-on allows the user to create rigid body physics for bones: https://github.com/Pauan/blender-rigid-body-bones The user selects a bone, clicks on a checkbox to enable rigid body physics for that bone, and then changes various rigid body settings: https://raw.githubusercontent.com/Pauan/blender-rigid-body-bones/master/Usage%2002.PNG So when the checkbox is enabled, it *must* create a rigid body object, and when the checkbox is disabled, it *must* remove that object. Having a checkbox to enable/disable things does not seem like an extreme case to me, it seems like a common thing to do. So it's very bizarre that you are saying that Blender does not support it. And using a modal operator or app handler sounds terribly wasteful and bad performance, since it will run all the time, but I only need to run the code when the checkbox is changed. It also sounds like an over-complication, since my use case is not complicated, and modal operators are clearly not intended for my use case. ---- If the problem is a lack of Operator, could I solve that by creating an Operator and then calling that Operator inside the update callback? That would be very easy for me to do. If that doesn't work, then that is a failing in the Python API, and the Python API should be changed (or new APIs created). Perhaps an API that says, "run this callback on the next update tick, during a time where it is safe to create objects". If you do not provide a correct API, people will instead abuse the current APIs. You cannot simply tell people "don't do that", because their use cases are legitimate and their goals are correct (even if their methods aren't), so they *will* do it no matter how much you say no. So if you don't want them using a bad API, you must instead provide a good API as an alternative. > you do know about our 'templates' available in the Text editor of Blender, right? Yes, I did see them, but none of them match my use case (which is probably intentional, since it sounds like my use case is simply not supported, which is quite odd). Would you accept pull requests improving the Python API documentation? I am new to the Python API, but I am a quite experienced programmer in general.

Bastien Montagne commented

2020-10-07 10:13:12 +02:00

You should have an operator to enable/disable this, not use a property, if you need to create objects etc.

Patches are always welcome, whether it is for documentation or code. :)

Also please do not use this tracker for that kind of discussion, we have other channels for that (blender.chat, our devltalk).

You should have an operator to enable/disable this, not use a property, if you need to create objects etc. Patches are always welcome, whether it is for documentation or code. :) Also please do not use this tracker for that kind of discussion, we have other channels for that ([blender.chat](https:*blender.chat/channel/python), [our devltalk](https:*devtalk.blender.org/c/other-topics/python/6)).

Pauan commented

2020-10-07 14:28:50 +02:00

@mont29 Sorry, I will continue the discussion here:

https://devtalk.blender.org/t/proper-way-to-create-remove-objects-in-response-to-property-changes/15692

@mont29 Sorry, I will continue the discussion here: https://devtalk.blender.org/t/proper-way-to-create-remove-objects-in-response-to-property-changes/15692

Thomas Dinges added this to the 2.92 milestone 2023-02-08 16:16:03 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

ID user decrement error when removing object which is referred to with a PointerProperty #81345