UI widget for pointer property with custom type causes segmentation fault
Open, NormalPublic

Description

System Information
Ubuntu 17.04 64 bit + Win 10 64 bit

Blender Version
Broken: 2.79

Short description of error
Creating a UI widget to edit a pointer property that was created with type specified as a subclass of bpy.types.ID declared in Python (rather than one of the pre-existing subclasses) causes a segmentation fault when trying to edit.

Exact steps for others to reproduce the error
Run the following script. Editing the first property in the popup (points to bpy.types.NodeTree) acts as expected. Editing the second property in the popup (points to bpy.types.TestTreeType) causes a segmentation fault.

import bpy

class CustomNodeTree(bpy.types.NodeTree):
    bl_idname = "TestTreeType"
    bl_label = "Test tree"
    bl_icon = "PLUGIN"

def draw(self, context):
    self.layout.prop(context.scene, "testNodeTreeProp")
    self.layout.prop(context.scene, "testCustomProp")

bpy.utils.register_class(CustomNodeTree)
bpy.types.Scene.testNodeTreeProp = bpy.props.PointerProperty(type=bpy.types.NodeTree)
bpy.types.Scene.testCustomProp = bpy.props.PointerProperty(type=bpy.types.TestTreeType)
bpy.context.window_manager.popup_menu(draw)

Details

Type
Bug
Sergey Sharybin (sergey) triaged this task as Normal priority.Sep 14 2017, 9:42 AM

@Bastien Montagne (mont29), is it something related on recent ID custom properties implemented for 2.79?

@Sergey Sharybin (sergey) yes, but… wait… since when is it valid to subclass an ID type? Subclassing ID types in Python… who ever allowed such an evil scheme? :((((((

This is for sure not going to work with ID custom props, not with current code anyway… type here shall always be real ID one, not some fancy py subclassing. Will check on code, this should be possible to validate at prop definition time.

@Bastien Montagne (mont29), isn't it how custom node trees are supposed to work? You kind of subclass there.
Would be nice to at least prevent crashes, perhaps.

Ok… still, don’t think that was a good idea, like, at all… ID types are not virtual stuff like operators or render engines, they are data, stored in Blender file, they are kind of root of everything in Blender, so faking their subclassing in py sounds like a nice can of worms to me… Would have been so much cleaner to have registrable NodeTreeType classes instead! Anyway…

Crash happens because automated search button generation code (ui_but_add_search()) cannot find a proper ID collection for the type of subclassed nodetree, and hence does not call UI_but_func_search_set() for created button, which later when entering 'edit mode' of that button tries to call but->search_create_func(), which is NULL.

So we actually have two issues here, one fail in UI code to safely handle case where searchprop of a searchmenu is not valid (will fix, it's pretty trivial).

And the subclass-of-real-ID case, here I would just forbid any non-real-ID class… and curse to death that very idea of ID subclassing! :( @Campbell Barton (campbellbarton) may have better ideas here though?

Maybe I've got the wrong idea but the impression I got was that this change to PointerProperty was useful because it allowed search boxes to be more specific than just materials, objects, node trees etc... I can't think of any reason the user would ever need a search box that allows selecting of all node tree types but it would be useful to only allow a specific type of node group.

Other than the UI bit it acts as expected when using the PointerProperty from other Python code. It allows data blocks of the correct type to be assigned and in this case, for example, it doesn't allow a node group that isn't of type TestTreeType.