Regression: Blender crash when moving camera immediately after opening the project (OPTIX) #103408

Closed
opened 2022-12-22 16:47:56 +01:00 by Michele · 25 comments

System Information
Operating system: Windows-10-10.0.22621-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 527.56

Blender Version
Broken: version: 3.4.1, branch: blender-v3.4-release, commit date: 2022-12-19 17:00, hash: 55485cb379
Worked: 3.3, 3.3.1 (not tested with 3.3.2)

Caused by 0d73d5c1a2

Short description of error
Blender crash when moving camera immediately after opening the project (Cycles - OPTIX)

Exact steps for others to reproduce the error

  1. Open the .blend file provided.
    file base da canc per prova blocco vuoto PROVA 2_test.blend
  2. In the bottom right viewport select the "Viewport shading" mode.
  3. Hold down the middle mouse button and move the mouse (above the viewport) to rotate the camera.
  4. Blender stop rensponding.
  • For the problem to arise the renderer need to be Cycles with OPTIX. No problem with CUDA.
  • The problem occurs if you perform camera movement before performing other operations. For example if you first move the Suzanne object and then the Blender camera doesn't freeze.
  • If the problem does not appear, reload the project and immediately move the camera as indicated.
**System Information** Operating system: Windows-10-10.0.22621-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 527.56 **Blender Version** Broken: version: 3.4.1, branch: blender-v3.4-release, commit date: 2022-12-19 17:00, hash: `55485cb379` Worked: 3.3, 3.3.1 (not tested with 3.3.2) Caused by 0d73d5c1a2 **Short description of error** Blender crash when moving camera immediately after opening the project (Cycles - OPTIX) **Exact steps for others to reproduce the error** 1. Open the .blend file provided. [file base da canc per prova blocco vuoto PROVA 2_test.blend](https://archive.blender.org/developer/F14083685/file_base_da_canc_per_prova_blocco_vuoto_PROVA_2_test.blend) 2. In the bottom right viewport select the "Viewport shading" mode. 3. Hold down the middle mouse button and move the mouse (above the viewport) to rotate the camera. 4. Blender stop rensponding. - For the problem to arise the renderer need to be Cycles with OPTIX. No problem with CUDA. - The problem occurs if you perform camera movement before performing other operations. For example if you first move the Suzanne object and then the Blender camera doesn't freeze. - If the problem does not appear, reload the project and immediately move the camera as indicated.
Author

Added subscriber: @Michele-Moscon

Added subscriber: @Michele-Moscon

Added subscriber: @iss

Added subscriber: @iss

Changed status from 'Needs Triage' to: 'Needs User Info'

Changed status from 'Needs Triage' to: 'Needs User Info'

I believe in step 2 you meant Rendered mode?

I can't reproduce this issue. Loading kernels step took about minute or 2, but eventually it started working well. Can you check if letting it work for few minutes would resolve this?

I believe in step 2 you meant Rendered mode? I can't reproduce this issue. Loading kernels step took about minute or 2, but eventually it started working well. Can you check if letting it work for few minutes would resolve this?
Author

Hi Richard.

In step 2 I meant to click here:
image.png

The problem occurs if the camera movement is performed immediately after opening the file. If the kernel starts loading please after the operation is finished try closing Blender and repeating the operations from step 1. This time at step 2 the render should start immediately.

Letting it work for a few minutes, like you suggested, doesn't fix the problem. And in any case, the rendering of this scene with the PCs I use takes a few moments (after the kernel has loaded the first time). Furthermore, the problem does not occur when opening the file in previous versions of Blender.

When the problem occurs Blender freezes (no longer responds) but does not close.

I tested on two different computers and both have the problem. One computer is the one described in the bug report, the other is a powerful workstation with Treadripper PRO, 64GB RAM, 2 x RTX 3090, 2 x RTX 3080 Ti.

Thank you.

Hi Richard. In step 2 I meant to click here: ![image.png](https://archive.blender.org/developer/F14085753/image.png) The problem occurs if the camera movement is performed immediately after opening the file. If the kernel starts loading please after the operation is finished try closing Blender and repeating the operations from step 1. This time at step 2 the render should start immediately. Letting it work for a few minutes, like you suggested, doesn't fix the problem. And in any case, the rendering of this scene with the PCs I use takes a few moments (after the kernel has loaded the first time). Furthermore, the problem does not occur when opening the file in previous versions of Blender. When the problem occurs Blender freezes (no longer responds) but does not close. I tested on two different computers and both have the problem. One computer is the one described in the bug report, the other is a powerful workstation with Treadripper PRO, 64GB RAM, 2 x RTX 3090, 2 x RTX 3080 Ti. Thank you.
Author

I noticed that in the first post the editor had added the endnotes to the numbered list. Now I've re-edited the task correcting it: there are only 4 steps and the subsequent texts are notes.

I noticed that in the first post the editor had added the endnotes to the numbered list. Now I've re-edited the task correcting it: there are only 4 steps and the subsequent texts are notes.

Changed status from 'Needs User Info' to: 'Needs Triage'

Changed status from 'Needs User Info' to: 'Needs Triage'

Unfortunately I can't reproduce the issue still. Do both PCs have dual GPUs? I would check if rendering on single GPU would cause freeze in such case, but that's just wild guess.

Unfortunately I can't reproduce the issue still. Do both PCs have dual GPUs? I would check if rendering on single GPU would cause freeze in such case, but that's just wild guess.
Member

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk
Member

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Member

I can confirm, cannot bisect this atm. (building without Optix -- but will try setting this up).

I can confirm, cannot bisect this atm. (building without Optix -- but will try setting this up).
Philipp Oeser changed title from Blender crash when moving camera immediately after opening the project (OPTIX) to Regression: Blender crash when moving camera immediately after opening the project (OPTIX) 2023-01-03 12:58:28 +01:00
Member

Added subscriber: @pmoursnv

Added subscriber: @pmoursnv
Member

@pmoursnv : this broke between good 05bf5c4e0e and bad 50c7eb14f4, I suspect c9eb583460 as the culprit commit (since disabling denoising seems to get rid of the freeze -- even though OSL is not used at all in this file...)

Mind checking?

@pmoursnv : this broke between good 05bf5c4e0e and bad 50c7eb14f4, I suspect c9eb5834607 as the culprit commit (since disabling denoising seems to get rid of the freeze -- even though OSL is not used at all in this file...) Mind checking?
Author

In #103408#1468341, @iss wrote:
Unfortunately I can't reproduce the issue still. Do both PCs have dual GPUs? I would check if rendering on single GPU would cause freeze in such case, but that's just wild guess.

One PC has one GPU, the other has 4. Both have the problem.

> In #103408#1468341, @iss wrote: > Unfortunately I can't reproduce the issue still. Do both PCs have dual GPUs? I would check if rendering on single GPU would cause freeze in such case, but that's just wild guess. One PC has one GPU, the other has 4. Both have the problem.
Member

Can't reproduce a freeze or crash with the provided steps, neither with ToT, nor 3.4.1 branch.
The aforementioned culprit commit is unlikely to be the problem, it only takes effect when OSL is enabled, otherwise behavior is unchanged (it's also not really related to denoising, only that changing the denoising setting after rendering was already started happened to force a kernel reload). There have been a lot of Cycles changes in the suspected range of commits though.

Can't reproduce a freeze or crash with the provided steps, neither with ToT, nor 3.4.1 branch. The aforementioned culprit commit is unlikely to be the problem, it only takes effect when OSL is enabled, otherwise behavior is unchanged (it's also not really related to denoising, only that changing the denoising setting after rendering was already started happened to force a kernel reload). There have been a lot of Cycles changes in the suspected range of commits though.
Member

Added subscriber: @LukasStockner

Added subscriber: @LukasStockner
Member

I can reproduce this as well, but the required timing is very specific.
Also, I can reproduce it in 05bf5c4e0e as well, so maybe something went wrong doring bisecting?

I can reproduce this as well, but the required timing is very specific. Also, I can reproduce it in 05bf5c4e0e as well, so maybe something went wrong doring bisecting?
Member

Appears to be caused by 0d73d5c1a2, makes a lot more sense since that change involves locks.

Appears to be caused by 0d73d5c1a2, makes a lot more sense since that change involves locks.
Member

Added subscriber: @Sergey

Added subscriber: @Sergey
Member

Thread A:

- 1  0x00007f0c2eca2c12 in pthread_mutex_lock () from /usr/lib/libc.so.6
- 2  0x0000000002aca291 in ccl::Session::run_update_for_next_iteration() ()
- 3  0x0000000002acb73b in ccl::Session::run_main_render_loop() ()
- 4  0x0000000002acc1ec in ccl::Session::thread_render() ()
- 5  0x0000000002acc323 in ccl::Session::thread_run() ()
- 6  0x0000000006ded88e in ccl::thread::run(void*) ()

Thread B:

- 1  0x00007f0c2eca2c12 in pthread_mutex_lock () from /usr/lib/libc.so.6
- 2  0x0000000002ac9518 in ccl::Session::reset(ccl::SessionParams const&, ccl::BufferParams const&) ()
- 3  0x00000000026cbbe5 in ccl::BlenderSession::synchronize(BL::Depsgraph&) ()
- 4  0x00000000026c6702 in ccl::sync_func(_object*, _object*) ()

The problem appears to be that:

  • Thread A in run_update_for_next_iteration releases the scene->mutex but continues to hold the delayed_reset_.mutex
  • Thread B in BlenderSession::synchronize grabs scene->mutex before calling Session::reset
  • Thread B ends up deadlocking on delayed_reset_.mutex which never gets released because A waits to get scene->mutex back before releasing it, which B still holds

The problem goes away if I just unlock reset_lock in run_update_for_next_iteration after the if (delayed_reset_.do_reset) block, but I'm not sure if that's safe to do. I don't see further accesses to delayed reset stuff below that, but I'm not really sure what the sequence here is supposed to be. @Sergey, does that sound reasonable?

Thread A: ``` - 1 0x00007f0c2eca2c12 in pthread_mutex_lock () from /usr/lib/libc.so.6 - 2 0x0000000002aca291 in ccl::Session::run_update_for_next_iteration() () - 3 0x0000000002acb73b in ccl::Session::run_main_render_loop() () - 4 0x0000000002acc1ec in ccl::Session::thread_render() () - 5 0x0000000002acc323 in ccl::Session::thread_run() () - 6 0x0000000006ded88e in ccl::thread::run(void*) () ``` Thread B: ``` - 1 0x00007f0c2eca2c12 in pthread_mutex_lock () from /usr/lib/libc.so.6 - 2 0x0000000002ac9518 in ccl::Session::reset(ccl::SessionParams const&, ccl::BufferParams const&) () - 3 0x00000000026cbbe5 in ccl::BlenderSession::synchronize(BL::Depsgraph&) () - 4 0x00000000026c6702 in ccl::sync_func(_object*, _object*) () ``` The problem appears to be that: - Thread A in `run_update_for_next_iteration` releases the `scene->mutex` but continues to hold the `delayed_reset_.mutex` - Thread B in `BlenderSession::synchronize` grabs `scene->mutex` before calling `Session::reset` - Thread B ends up deadlocking on `delayed_reset_.mutex` which never gets released because A waits to get `scene->mutex` back before releasing it, which B still holds The problem goes away if I just unlock `reset_lock` in `run_update_for_next_iteration` after the `if (delayed_reset_.do_reset)` block, but I'm not sure if that's safe to do. I don't see further accesses to delayed reset stuff below that, but I'm not really sure what the sequence here is supposed to be. @Sergey, does that sound reasonable?
Member

Thx checking Lukas!

Thx checking Lukas!
Member

Also Patrick!

Also Patrick!

@LukasStockner, I think it is safe to release the reset_lock early on. I don't really see reason why we really need to hold it longer.

@LukasStockner, I think it is safe to release the `reset_lock` early on. I don't really see reason why we really need to hold it longer.

This issue was referenced by 2895c67086

This issue was referenced by 2895c6708656b749b76331a907b539654ca125e1
Member

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Lukas Stockner self-assigned this 2023-01-07 20:41:43 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
7 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#103408
No description provided.