Blender crashes in texture paint window when trying to paint after going back from edit mode #56850

New Issue

Mateo · 2018-09-20T00:19:27+02:00

Mateo commented

2018-09-20 00:19:27 +02:00

System Information
Windows 10 64 bit
Intel Core i5-7600
Nvidia GeForce GTX 1060 6GB

Blender Version
Broken: 2.8 69aac13524
Worked: N/A

Short description of error
In texture paint window when going into edit mode and back to paint mode it crashes when trying to paint onto the object

Exact steps for others to reproduce the error

Delete everything from start scene
Navigate to texture paint window
Add a default cube
Change mode to texture paint
Add paint slot(base color) in missing data dropdown
Paint few strokes on a cube
Change to edit mode
Change to texture paint mode again
Try to paint on cube again(sometimes it only does not register the strokes, but mainly it is crashing)

**System Information** Windows 10 64 bit Intel Core i5-7600 Nvidia GeForce GTX 1060 6GB **Blender Version** Broken: 2.8 69aac135243 Worked: N/A **Short description of error** In texture paint window when going into edit mode and back to paint mode it crashes when trying to paint onto the object **Exact steps for others to reproduce the error** 1. Delete everything from start scene 2. Navigate to texture paint window 3. Add a default cube 4. Change mode to texture paint 5. Add paint slot(base color) in missing data dropdown 6. Paint few strokes on a cube 7. Change to edit mode 8. Change to texture paint mode again 9. Try to paint on cube again(sometimes it only does not register the strokes, but mainly it is crashing)

Mateo commented

2018-09-20 00:19:27 +02:00

Added subscriber: @shadercode

Mateo changed title from ~~Blender crashes in texture paint window when trying to paint after zooming in and out in edit mode~~ to Blender crashes in texture paint window when trying to paint after going back from edit mode

2018-09-20 00:19:59 +02:00

Bastien Montagne commented

2018-09-20 08:59:57 +02:00

Added subscriber: @mont29

Bastien Montagne commented

2018-09-20 08:59:57 +02:00

Please follow our submission template and guidelines, also read these tips about bug reports, and make a complete, valid bug report, with required info, precise description of the issue (only ONE issue per report!), precise steps to reproduce it, small and simple .blend and/or other files to do so if needed, etc.

Please follow our [submission template and guidelines](https:*developer.blender.org/maniphest/task/edit/form/1/), also read [these tips about bug reports](https:*wiki.blender.org/wiki/Process/Bug_Reports), and make a complete, valid bug report, with required info, precise description of the issue (only ONE issue per report!), precise steps to reproduce it, **small and simple** .blend and/or other files to do so if needed, etc.

Chris Holland commented

2018-09-27 15:00:29 +02:00

Added subscriber: @CodeSlinger19

Chris Holland commented

2018-09-27 15:00:29 +02:00

I was about to log this crash myself. How is it incomplete? It has everything the guidelines ask for especially very straight forward repro steps. If you're wanting a .blend file, there's no point, as you can reproduce simply from the startup scene with only the few steps described, you don't event need to remove the objects from the starting scene, and you can just use the default cube.

Here's the blender.crash.txt:

Blender 2.80 (sub 24), Commit date: 2018-09-27 11:19, Hash `250a377592`

Tool 'Cursor' not found. # Warning
Tool 'Cursor' not found. # Warning
bpy.ops.paint.texture_paint_toggle() # Operator
bpy.ops.paint.texture_paint_toggle() # Operator
bpy.ops.object.editmode_toggle() # Operator
bpy.ops.object.editmode_toggle() # Operator
bpy.ops.paint.texture_paint_toggle() # Operator
bpy.ops.paint.texture_paint_toggle() # Operator
bpy.ops.object.editmode_toggle() # Operator
bpy.ops.object.editmode_toggle() # Operator
bpy.ops.paint.texture_paint_toggle() # Operator

backtrace

/isodevice/software/blender-git/build_linux/bin/blender(BLI_system_backtrace+0x3e) [0x557eec5e3efe]
/isodevice/software/blender-git/build_linux/bin/blender(+0xac4889) [0x557eebb39889]
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f6de4fa9f20]
/isodevice/software/blender-git/build_linux/bin/blender(+0xe21877) [0x557eebe96877]
/isodevice/software/blender-git/build_linux/bin/blender(paint_proj_new_stroke+0x283) [0x557eebe9fdf3]
/isodevice/software/blender-git/build_linux/bin/blender(+0xe16b1c) [0x557eebe8bb1c]
/isodevice/software/blender-git/build_linux/bin/blender(paint_stroke_modal+0x1de) [0x557eebea935e]
/isodevice/software/blender-git/build_linux/bin/blender(+0xe16c89) [0x557eebe8bc89]
/isodevice/software/blender-git/build_linux/bin/blender(+0xac9e96) [0x557eebb3ee96]
/isodevice/software/blender-git/build_linux/bin/blender(+0xaccc15) [0x557eebb41c15]
/isodevice/software/blender-git/build_linux/bin/blender(+0xacd1f5) [0x557eebb421f5]
/isodevice/software/blender-git/build_linux/bin/blender(+0xacdb45) [0x557eebb42b45]
/isodevice/software/blender-git/build_linux/bin/blender(wm_event_do_handlers+0x50c) [0x557eebb433ec]
/isodevice/software/blender-git/build_linux/bin/blender(WM_main+0x30) [0x557eebb3a3d0]
/isodevice/software/blender-git/build_linux/bin/blender(main+0x345) [0x557eebb1f4f5]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f6de4f8cb97]
/isodevice/software/blender-git/build_linux/bin/blender(_start+0x2a) [0x557eebb35dfa]

I was about to log this crash myself. How is it incomplete? It has everything the guidelines ask for especially very straight forward repro steps. If you're wanting a .blend file, there's no point, as you can reproduce simply from the startup scene with only the few steps described, you don't event need to remove the objects from the starting scene, and you can just use the default cube. Here's the blender.crash.txt: # Blender 2.80 (sub 24), Commit date: 2018-09-27 11:19, Hash 250a377592c Tool 'Cursor' not found. # Warning Tool 'Cursor' not found. # Warning bpy.ops.paint.texture_paint_toggle() # Operator bpy.ops.paint.texture_paint_toggle() # Operator bpy.ops.object.editmode_toggle() # Operator bpy.ops.object.editmode_toggle() # Operator bpy.ops.paint.texture_paint_toggle() # Operator bpy.ops.paint.texture_paint_toggle() # Operator bpy.ops.object.editmode_toggle() # Operator bpy.ops.object.editmode_toggle() # Operator bpy.ops.paint.texture_paint_toggle() # Operator # backtrace /isodevice/software/blender-git/build_linux/bin/blender(BLI_system_backtrace+0x3e) [0x557eec5e3efe] /isodevice/software/blender-git/build_linux/bin/blender(+0xac4889) [0x557eebb39889] /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f6de4fa9f20] /isodevice/software/blender-git/build_linux/bin/blender(+0xe21877) [0x557eebe96877] /isodevice/software/blender-git/build_linux/bin/blender(paint_proj_new_stroke+0x283) [0x557eebe9fdf3] /isodevice/software/blender-git/build_linux/bin/blender(+0xe16b1c) [0x557eebe8bb1c] /isodevice/software/blender-git/build_linux/bin/blender(paint_stroke_modal+0x1de) [0x557eebea935e] /isodevice/software/blender-git/build_linux/bin/blender(+0xe16c89) [0x557eebe8bc89] /isodevice/software/blender-git/build_linux/bin/blender(+0xac9e96) [0x557eebb3ee96] /isodevice/software/blender-git/build_linux/bin/blender(+0xaccc15) [0x557eebb41c15] /isodevice/software/blender-git/build_linux/bin/blender(+0xacd1f5) [0x557eebb421f5] /isodevice/software/blender-git/build_linux/bin/blender(+0xacdb45) [0x557eebb42b45] /isodevice/software/blender-git/build_linux/bin/blender(wm_event_do_handlers+0x50c) [0x557eebb433ec] /isodevice/software/blender-git/build_linux/bin/blender(WM_main+0x30) [0x557eebb3a3d0] /isodevice/software/blender-git/build_linux/bin/blender(main+0x345) [0x557eebb1f4f5] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f6de4f8cb97] /isodevice/software/blender-git/build_linux/bin/blender(_start+0x2a) [0x557eebb35dfa]

Chris Holland commented

2018-09-27 17:38:34 +02:00

I tried to do some debugging and this is as far as I've managed to get tonight:

file: paint_image_proj.c 
function: proj_paint_state_mesh_eval_init
line 3419: ps->me_eval = mesh_get_eval_final

After flipping to edit mode and back to paint mode and then trying to paint a new stroke, it runs the line above and the result is that ps->me_eval->mpoly->mat_nr seems to incorrectly contains a negative number (-2080), and possibly the other attributes under mpoly are wrong too. This seems to lead to an index out of bounds problem which causes the crash later on in project_paint_face_paint_slot

I tried to do some debugging and this is as far as I've managed to get tonight: ``` file: paint_image_proj.c function: proj_paint_state_mesh_eval_init line 3419: ps->me_eval = mesh_get_eval_final ``` After flipping to edit mode and back to paint mode and then trying to paint a new stroke, it runs the line above and the result is that `ps->me_eval->mpoly->mat_nr` seems to incorrectly contains a negative number (-2080), and possibly the other attributes under mpoly are wrong too. This seems to lead to an index out of bounds problem which causes the crash later on in `project_paint_face_paint_slot`

Chris Holland commented

2018-10-02 05:13:21 +02:00

I've been trying to debug this and have a workaround patch to prove I'm on the right track.
I'm hoping with this information one of the developers who understands these structures and the associated memory management will be able to track down how to fix the disjoint between the runtime->mesh_eval and preservation of the memory use by it's various properties.

This is what happens in DerivedMesh.c: mesh_get_eval_final:

For the first paint stroke, ob->runtime.mesh_eval is 0x0, so the condition in mesh_get_eval_final causes it to run mesh_build_data which populates ob->runtime.mesh_eval and all is good in the world.
But.... mesh_eval itself contains pointers to other objects/structures, one of which is mpoly and after switching to edit mode and back to paint mode, it seems that the memory pointed to by mpoly has been freed and used for something else. It looks like it's freed by EDBM_mesh_load which already has a comment stating that it can invalidate the cache of other objects.
However ob->runtime.mesh_eval is still a valid pointer, so when you paint the next stroke after this, the condition in mesh_get_eval_final sees that mesh_eval is not NULL, so it does not run mesh_build_data, which means the mpoly attribute doesn't get repopulated with a valid pointer, and it's still pointing to invalid memory. It's probably the same for other attributes under mesh_eval, but mpoly was the one I saw causing the crash.
The patch below comments out the condition and just runs mesh_build_data every time, which is obviously inefficient and a workaround rather than a solution, but it prevents the crash.
The issue is that other functions assume that if ob->runtime.mesh_eval is not 0, then it contains valid data, but it currently contains properties that are pointers which can be freed by other parts of code rendering them invalid.

diff --git a/source/blender/blenkernel/intern/DerivedMesh.c b/source/blender/blenkernel/intern/DerivedMesh.c
index 4c55591db31..634c99fd310 100644
--- a/source/blender/blenkernel/intern/DerivedMesh.c
+++ b/source/blender/blenkernel/intern/DerivedMesh.c
@@ -2464,12 +2464,15 @@ Mesh *mesh_get_eval_final(
        bool need_mapping;
        dataMask |= object_get_datamask(depsgraph, ob, &need_mapping);
 
+/*
        if (!ob->runtime.mesh_eval ||
            ((dataMask & ob->lastDataMask) != dataMask) ||
            (need_mapping != ob->lastNeedMapping))
        {
                mesh_build_data(depsgraph, scene, ob, dataMask, false, need_mapping);
        }
+*/
+       mesh_build_data(depsgraph, scene, ob, dataMask, false, need_mapping);
 
        if (ob->runtime.mesh_eval) { BLI_assert(!(ob->runtime.mesh_eval->runtime.cd_dirty_vert & CD_MASK_NORMAL)); }
        return ob->runtime.mesh_eval;

I've been trying to debug this and have a workaround patch to prove I'm on the right track. I'm hoping with this information one of the developers who understands these structures and the associated memory management will be able to track down how to fix the disjoint between the `runtime->mesh_eval` and preservation of the memory use by it's various properties. This is what happens in `DerivedMesh.c: mesh_get_eval_final`: - For the first paint stroke, `ob->runtime.mesh_eval` is `0x0`, so the condition in `mesh_get_eval_final` causes it to run `mesh_build_data` which populates `ob->runtime.mesh_eval` and all is good in the world. - But.... `mesh_eval` itself contains pointers to other objects/structures, one of which is `mpoly` and after switching to edit mode and back to paint mode, it seems that the memory pointed to by `mpoly` has been freed and used for something else. It looks like it's freed by `EDBM_mesh_load` which already has a comment stating that it can invalidate the cache of other objects. - However `ob->runtime.mesh_eval` is still a valid pointer, so when you paint the next stroke after this, the condition in `mesh_get_eval_final` sees that `mesh_eval` is not `NULL`, so it does not run `mesh_build_data`, which means the `mpoly` attribute doesn't get repopulated with a valid pointer, and it's still pointing to invalid memory. It's probably the same for other attributes under `mesh_eval`, but `mpoly` was the one I saw causing the crash. The patch below comments out the condition and just runs `mesh_build_data` every time, which is obviously inefficient and a workaround rather than a solution, but it prevents the crash. The issue is that other functions assume that if `ob->runtime.mesh_eval` is not 0, then it contains valid data, but it currently contains properties that are pointers which can be freed by other parts of code rendering them invalid. ``` diff --git a/source/blender/blenkernel/intern/DerivedMesh.c b/source/blender/blenkernel/intern/DerivedMesh.c index 4c55591db31..634c99fd310 100644 --- a/source/blender/blenkernel/intern/DerivedMesh.c +++ b/source/blender/blenkernel/intern/DerivedMesh.c @@ -2464,12 +2464,15 @@ Mesh *mesh_get_eval_final( bool need_mapping; dataMask |= object_get_datamask(depsgraph, ob, &need_mapping); +/* if (!ob->runtime.mesh_eval || ((dataMask & ob->lastDataMask) != dataMask) || (need_mapping != ob->lastNeedMapping)) { mesh_build_data(depsgraph, scene, ob, dataMask, false, need_mapping); } +*/ + mesh_build_data(depsgraph, scene, ob, dataMask, false, need_mapping); if (ob->runtime.mesh_eval) { BLI_assert(!(ob->runtime.mesh_eval->runtime.cd_dirty_vert & CD_MASK_NORMAL)); } return ob->runtime.mesh_eval; ```

Chris Holland commented

2018-10-14 14:26:38 +02:00

Hi Bastien, can you please let us know what we need to do to get this bug out of "Incomplete" status? I tried posting to bf-committers, but was denied by the moderator because I'm not a member. They said to follow up here.
I'd be happy to do more debugging if someone could give me some suggestions on where to go from here as I've reached as far as I can go with my lack of understanding of the fundamental design of the eval structures and deps graph.
Thanks.

Hi Bastien, can you please let us know what we need to do to get this bug out of "Incomplete" status? I tried posting to bf-committers, but was denied by the moderator because I'm not a member. They said to follow up here. I'd be happy to do more debugging if someone could give me some suggestions on where to go from here as I've reached as far as I can go with my lack of understanding of the fundamental design of the eval structures and deps graph. Thanks.

Brecht Van Lommel commented

2018-10-28 16:24:44 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2018-10-28 16:24:44 +01:00

I think this patch indeed is not the right solution. The proper solution may be to tag the dependency graph to re-evaluate the mesh, but @mont29 or someone else familiar with this code needs to look at it in more detail.

Chris Holland commented

2018-12-10 15:05:24 +01:00

I just tested this again and it seems to be fixed. I checked probably a week or so ago (after the beta) and it was still crashing, so something done recently must have fixed it. Perhaps this commit?: Depsgraph: Make data allocation dependency more robust

Brecht Van Lommel commented

2018-12-10 15:32:48 +01:00

Changed status from 'Open' to: 'Resolved'

Brecht Van Lommel closed this issue

2018-12-10 15:32:48 +01:00

Brecht Van Lommel self-assigned this 2018-12-10 15:32:48 +01:00

Brecht Van Lommel commented

2018-12-10 15:32:49 +01:00

Thanks for testing, closing then.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender crashes in texture paint window when trying to paint after going back from edit mode #56850

Blender 2.80 (sub 24), Commit date: 2018-09-27 11:19, Hash 250a377592

backtrace

Blender 2.80 (sub 24), Commit date: 2018-09-27 11:19, Hash `250a377592`