Page MenuHome

Crash on Rendering with Mesh Creation/Removal in frame_change_post handler.
Closed, InvalidPublicKNOWN ISSUE

Description

Linux, Debian
Broken: dc3f073d1c52 current master branch

I get non-deterministic/random segfaults on rendering (cycles). I would guess approximately on every 5th frame. Also sometimes. on the first rendering after launching blender. I am not able to fully render a single shot in my current project, so it's an absolute show-stopper for myself at least.

I can reproduce this in various form in a rather complex setup involving a 5k locs inhouse addon. Depending on the the scene update, I get rather different stacktraces, but I guess it's one and the same problem (related to depsgraph).

To reproduce:

  • Run the included python addon
  • With the default scene in 2.81a
  • Render an animation. The segfault may only happen after quite some frames.
bl_info = {
    "name": "Crash Min Ex",
    "author": "Aurel Wildfellner",
    "blender": (2, 81, 0),
    "location": "3D View > Toolbox",
    "description": "to reproduce a render crash",
    "warning": "",
    "wiki_url": "",
    "support": 'TESTING',
    "category": "Mesh"}

if "bpy" in locals():
    import importlib
else:{F8323293}
    import bpy
from bpy.app.handlers import persistent


def generateCubeMeshData():
    tri_vertices = []
    tri_indices = []

    tri_vertices.append([1,1,1])
    tri_vertices.append([-1,1,1])
    tri_vertices.append([1,-1,1])
    tri_vertices.append([-1,-1,1])

    tri_vertices.append([1,1,-1])
    tri_vertices.append([-1,1,-1])
    tri_vertices.append([1,-1,-1])
    tri_vertices.append([-1,-1,-1])

    tri_indices.append([0, 1, 2])
    tri_indices.append([1, 3, 2])
    tri_indices.append([4, 5, 7])
    tri_indices.append([5, 8, 6])

    tri_indices.append([1, 0, 5])
    tri_indices.append([5, 4, 0])

    return (tri_vertices, tri_indices)

def removeSupportMeshes(context): 
    #FIXME this seems to be the correct way, but we still get
    #      constant growing of memory footprint.
    scn = context.scene
    objects_to_be_removed = []
    for obj in scn.objects:
        if "support_mesh_foo" in obj.name:
            objects_to_be_removed.append(obj.name)
    for objname in objects_to_be_removed:
            bpy.data.meshes.remove(scn.objects[objname].data,
                                   do_unlink=True)
            #bpy.data.objects.remove(scn.objects[objname], 
            #                        do_unlink=True)
    context.view_layer.update()


def buildGeometry(context):
    print("Build Geometry")
    scn = context.scene

    # store active object's name, so we can reselect
    old_active_obj_name = ""
    if hasattr(bpy.context, "active_object"):
        if bpy.context.active_object:
            old_active_obj_name = bpy.context.active_object.name

    removeSupportMeshes(context)

    support_mesh_name = "support_mesh_foo"
    smesh = bpy.data.meshes.new(support_mesh_name)
    sobj = bpy.data.objects.new(support_mesh_name, smesh)
    # add mesh data
    tri_vertices, tri_indices = generateCubeMeshData()
    smesh.from_pydata(tri_vertices, [], tri_indices)
    ## update scene
    scn.collection.objects.link(sobj)
    context.view_layer.update()
            
    if old_active_obj_name in scn.objects:
        print("setting active object....")
        context.view_layer.objects.active = \
                scn.objects[old_active_obj_name]
        scn.objects[old_active_obj_name].select_set(True)
        context.view_layer.update()


@persistent
def postFrameChangeHandler(scn):
    print("FRAME CHANGE")
    buildGeometry(bpy.context)


def register():
    ## register handler
    bpy.app.handlers.frame_change_post.append(postFrameChangeHandler)


def unregister():
    for op in operator_cls:
        bpy.utils.unregister_class(op)

I am rather confused about when I should and should not call view_layer.update() after adding/removing mesh objects in the scene. However I get crashes also without updating the view_layer.

I get the following stacktrace for the segfault:

# Blender 2.81 (sub 16), Commit date: 2019-12-04 11:32, Hash f1aa4d18d49d

# backtrace
./blender(BLI_system_backtrace+0x1d) [0x14ee84d]
./blender() [0x12cbb79]
/lib/x86_64-linux-gnu/libc.so.6(+0x37840) [0x7f244c79f840]
./blender(_ZN3DEG29deg_copy_on_write_is_expandedEPK2ID+0) [0x150dc10]
./blender(_ZN3DEG20DepsgraphNodeBuilder11begin_buildEv+0xd7) [0x14ffd37]
./blender(DEG_graph_build_from_view_layer+0x6d) [0x14f45dd]
./blender() [0x13f9043]
./blender(wm_event_do_depsgraph+0xc3) [0x16a1d63]
./blender(wm_event_do_refresh_wm_and_depsgraph+0x87) [0x16a1ec7]
./blender(wm_event_do_notifiers+0x472) [0x16a79f2]
./blender(WM_main+0x28) [0x169dcd8]
./blender(main+0x2fe) [0x1237f7e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f244c78c09b]
./blender() [0x12c849c]

Event Timeline

Brecht Van Lommel (brecht) changed the task status from Needs Triage to Needs Information from User.Feb 3 2020, 12:05 PM

This needs at least the relevant sections of code from the add-on.

It matters which exact handler is being used, how dependency graph and evaluated data, if it's modifying original or evalauted data, etc.

@Brecht Van Lommel (brecht) I will put some more time into synthesizing a minimal working example. I was just not able reproduce the segfaults with my first attempt.

I expect that at least one of the crashes related to the deps_graph is cause by just deleting and creating a mesh object in a frame_change_post handler.

The other issues already pointed out seem related, also the fixes/workarounds done in the animation nodes addon (which i don't fully understand yet).

I was able to come up with a minimum example. My more complex code, including modifiers etc. resulted in segfaults with different stacktraces, but I guess it's the same issue triggering it.

Steps to reproduce:

  • Enable the provided Addon
  • Open blenders default scene (2.81a).
  • Render animation, a crash may only happen after quite a lot of frames
python
# Author: Aurel Wildfellner

bl_info = {
    "name": "Crash Min Ex",
    "author": "Aurel Wildfellner",
    "blender": (2, 81, 0),
    "location": "3D View > Toolbox",
    "description": "to reproduce a render crash",
    "warning": "",
    "wiki_url": "",
    "support": 'TESTING',
    "category": "Mesh"}

import math

if "bpy" in locals():
    import importlib
else:
    import bpy
from bpy.app.handlers import persistent
from bpy.types import Operator


def generateCubeMeshData():
    tri_vertices = []
    tri_indices = []

    tri_vertices.append([1,1,1])
    tri_vertices.append([-1,1,1])
    tri_vertices.append([1,-1,1])
    tri_vertices.append([-1,-1,1])

    tri_vertices.append([1,1,-1])
    tri_vertices.append([-1,1,-1])
    tri_vertices.append([1,-1,-1])
    tri_vertices.append([-1,-1,-1])

    tri_indices.append([0, 1, 2])
    tri_indices.append([1, 3, 2])
    tri_indices.append([4, 5, 7])
    tri_indices.append([5, 8, 6])

    tri_indices.append([1, 0, 5])
    tri_indices.append([5, 4, 0])

    return (tri_vertices, tri_indices)

def removeSupportMeshes(context): 
    #FIXME this seems to be the correct way, but we still get
    #      constant growing of memory footprint.
    scn = context.scene
    objects_to_be_removed = []
    for obj in scn.objects:
        if "support_mesh_foo" in obj.name:
            objects_to_be_removed.append(obj.name)
    for objname in objects_to_be_removed:
            bpy.data.meshes.remove(scn.objects[objname].data,
                                   do_unlink=True)
            #bpy.data.objects.remove(scn.objects[objname], 
            #                        do_unlink=True)
    context.view_layer.update()


def buildGeometry(context):
    print("Build Geometry")
    scn = context.scene
    if (hasattr(bpy.context, "active_object")):
        obj = bpy.context.active_object
    else:
        obj = None

    # store active object's name, so we can reselect
    old_active_obj_name = ""
    if hasattr(bpy.context, "active_object"):
        if bpy.context.active_object:
            old_active_obj_name = bpy.context.active_object.name

    removeSupportMeshes(context)

    support_mesh_name = "support_mesh_foo"
    smesh = bpy.data.meshes.new(support_mesh_name)
    sobj = bpy.data.objects.new(support_mesh_name, smesh)
    # add mesh data
    tri_vertices, tri_indices = generateCubeMeshData()
    smesh.from_pydata(tri_vertices, [], tri_indices)
    ## update scene
    scn.collection.objects.link(sobj)
    context.view_layer.update()
            
    if old_active_obj_name in scn.objects:
        print("setting active object....")
        context.view_layer.objects.active = \
                scn.objects[old_active_obj_name]
        scn.objects[old_active_obj_name].select_set(True)
        bpy.ops.object.mode_set(mode = 'OBJECT')
        context.view_layer.update()




class MyOperator(Operator):
    """Operator triggering crash"""
    bl_idname = "mesh.add_cube_crash"
    bl_label = "adding cube object"

    def execute(self, context):
        buildGeometry(context)

        return {'FINISHED'}

@persistent
def preFrameChangeHandler(scn):
    pass

@persistent
def postFrameChangeHandler(scn):
    print("FRAME CHANGE")
    bpy.ops.mesh.add_cube_crash()


operator_cls = (
    MyOperator,
)


def register():
    for op in operator_cls:
        print("registering operator: ", op.__name__)
        bpy.utils.register_class(op)

    ## register handlers
    bpy.app.handlers.frame_change_post.append(postFrameChangeHandler)
    bpy.app.handlers.frame_change_pre.append(preFrameChangeHandler)


def unregister():
    for op in operator_cls:
        bpy.utils.unregister_class(op)

Does it happen also without calling operators from the handler? It wouldn't surprise me if that causes problems, frame change and depsgraph handlers normally should only use direct API functions. Calling operators with associated undo pushes and other scene updates may well break something.

@Brecht Van Lommel (brecht) the operators from handlers was something I expected myself, but I get also segfaults without any operator involved.

However, I also get the following Python RuntimeError now as well:

buildGeometry(bpy.context)
  File "/home/aurel/packages/blender-2.81a-linux-glibc217-x86_64/2.81/scripts/addons/crash_min_ex/__init__.py", line 73, in buildGeometry
    removeSupportMeshes(context)
  File "/home/aurel/packages/blender-2.81a-linux-glibc217-x86_64/2.81/scripts/addons/crash_min_ex/__init__.py", line 60, in removeSupportMeshes
    context.view_layer.update()
RuntimeError: Error: Dependency graph update requested during evaluation

The stripped down addon:

# Author: Aurel Wildfellner

bl_info = {
    "name": "Crash Min Ex",
    "author": "Aurel Wildfellner",
    "blender": (2, 81, 0),
    "location": "3D View > Toolbox",
    "description": "to reproduce a render crash",
    "warning": "",
    "wiki_url": "",
    "support": 'TESTING',
    "category": "Mesh"}

import math

if "bpy" in locals():
    import importlib
else:
    import bpy
from bpy.app.handlers import persistent


def generateCubeMeshData():
    tri_vertices = []
    tri_indices = []

    tri_vertices.append([1,1,1])
    tri_vertices.append([-1,1,1])
    tri_vertices.append([1,-1,1])
    tri_vertices.append([-1,-1,1])

    tri_vertices.append([1,1,-1])
    tri_vertices.append([-1,1,-1])
    tri_vertices.append([1,-1,-1])
    tri_vertices.append([-1,-1,-1])

    tri_indices.append([0, 1, 2])
    tri_indices.append([1, 3, 2])
    tri_indices.append([4, 5, 7])
    tri_indices.append([5, 8, 6])

    tri_indices.append([1, 0, 5])
    tri_indices.append([5, 4, 0])

    return (tri_vertices, tri_indices)

def removeSupportMeshes(context): 
    #FIXME this seems to be the correct way, but we still get
    #      constant growing of memory footprint.
    scn = context.scene
    objects_to_be_removed = []
    for obj in scn.objects:
        if "support_mesh_foo" in obj.name:
            objects_to_be_removed.append(obj.name)
    for objname in objects_to_be_removed:
            bpy.data.meshes.remove(scn.objects[objname].data,
                                   do_unlink=True)
            #bpy.data.objects.remove(scn.objects[objname], 
            #                        do_unlink=True)
    context.view_layer.update()


def buildGeometry(context):
    print("Build Geometry")
    scn = context.scene

    # store active object's name, so we can reselect
    old_active_obj_name = ""
    if hasattr(bpy.context, "active_object"):
        if bpy.context.active_object:
            old_active_obj_name = bpy.context.active_object.name

    removeSupportMeshes(context)

    support_mesh_name = "support_mesh_foo"
    smesh = bpy.data.meshes.new(support_mesh_name)
    sobj = bpy.data.objects.new(support_mesh_name, smesh)
    # add mesh data
    tri_vertices, tri_indices = generateCubeMeshData()
    smesh.from_pydata(tri_vertices, [], tri_indices)
    ## update scene
    scn.collection.objects.link(sobj)
    context.view_layer.update()
            
    if old_active_obj_name in scn.objects:
        print("setting active object....")
        context.view_layer.objects.active = \
                scn.objects[old_active_obj_name]
        scn.objects[old_active_obj_name].select_set(True)
        context.view_layer.update()


@persistent
def postFrameChangeHandler(scn):
    print("FRAME CHANGE")
    buildGeometry(bpy.context)


def register():
    ## register handler
    bpy.app.handlers.frame_change_post.append(postFrameChangeHandler)


def unregister():
    for op in operator_cls:
        bpy.utils.unregister_class(op)
Aurel W (aurel) renamed this task from Crash on Rendering with Linked Scenes to Crash on Rendering with Mesh Creation/Removal in frame_change_post handler..Feb 5 2020, 11:42 AM

So after playing around with this a little bit more, I get all sorts of strange behavior.

First of all, that python RuntimeError also is triggered non-determinstically. I am actually not even able to reproduce it right now. Am I not required to update the view layer when adding/removing objects so the view layer is up to date for rendering?

I still get segfaults with this minimal setup, no matter if I view_layer.update() or not. I struggle to find clear, good Information on how to do this correctly, but even with wrong API usage, I guess non-determinstiv error reporting and random crashes are not desired.

I update the issue description accordingly.

Try locking the interface (Render → Lock Interface). If this prevents the crash, it's a known threading issue.

Ok, @Sybren A. Stüvel (sybren) I wasn't aware of that. That fixes the issue for me, both in the minimal provided example and in my rendering setup (at least after testing some example shots).

Should this still be considered as a bug then? From my perspective the next person running into such an issue is just a matter of time. Documenting this somehow along the Python API and handlers seems also a good idea.

Sybren A. Stüvel (sybren) claimed this task.
Sybren A. Stüvel (sybren) changed the subtype of this task from "Report" to "Known Issue".

I have added a note about this in the bpy.app.handlers API documentation.

@Sybren A. Stüvel (sybren) I tried your workaround (Render -> Lock Interface) and it prevented the crash, but the materials failed to load before the render started, so my mesh rendered as bright pink. Any ideas for this? Update: Sorry, turns out it was a poorly-formed .mtl file. This workaround seems to work fine for me.

(@Aurel W (aurel) Did you think your prophecy would come true so quickly? I filed a bug for this just yesterday.)