Cycles Metal crash with simultaneous viewport and final render #94142

Closed
opened 2021-12-16 09:42:47 +01:00 by Frederik De Bleser · 32 comments

System Information
Operating system: macOS-12.0.1-arm64-arm-64bit 64 Bits
Graphics card: Apple M1 Apple 4.1 Metal - 76.1

Blender Version
Broken: version: 3.1.0 Alpha, branch: master, commit date: 2021-12-15 00:57, hash: 67b657f07c
Worked: none (testing the new Metal renderer)

Short description of error
Blender crashes when using Cycles with Metal GPU in both viewport and renderer

Exact steps for others to reproduce the error

  • Open the attached file on a Mac with M1 processor
  • In Preferences > System, set Cycles Render Devices to Apple M1 (GPU).
  • Choose Rendered viewport shading
  • Press F12 to start the render. This will crash Blender.

human001-no-texture.blend

**System Information** Operating system: macOS-12.0.1-arm64-arm-64bit 64 Bits Graphics card: Apple M1 Apple 4.1 Metal - 76.1 **Blender Version** Broken: version: 3.1.0 Alpha, branch: master, commit date: 2021-12-15 00:57, hash: `67b657f07c` Worked: none (testing the new Metal renderer) **Short description of error** Blender crashes when using Cycles with Metal GPU in both viewport and renderer **Exact steps for others to reproduce the error** - Open the attached file on a Mac with M1 processor - In Preferences > System, set Cycles Render Devices to Apple M1 (GPU). - Choose Rendered viewport shading - Press F12 to start the render. This will crash Blender. [human001-no-texture.blend](https://archive.blender.org/developer/F12752831/human001-no-texture.blend)

Added subscriber: @FrederikDeBleser

Added subscriber: @FrederikDeBleser

I've attached the crash report:
2021-12-15-blender-crash-report.txt

I've attached the crash report: [2021-12-15-blender-crash-report.txt](https://archive.blender.org/developer/F12752847/2021-12-15-blender-crash-report.txt)

Added subscriber: @JurgenS-4

Added subscriber: @JurgenS-4

I have tested with this file and on my Mac mini M1 16 GB with the same build of blender, sometimes it works fine and other times it indeed chrashed.

I have tested with this file and on my Mac mini M1 16 GB with the same build of blender, sometimes it works fine and other times it indeed chrashed.

Added subscriber: @Staars

Added subscriber: @Staars

More or less the same here.

When I wait for the viewport to finish the render completely (checking with start option --verbose 2 in the console), I can finish the first render and even finish a second render with that method. But when I want to close the render window the second time, I get a crash too.

More or less the same here. When I wait for the viewport to finish the render completely (checking with start option `--verbose 2` in the console), I can finish the first render and even finish a second render with that method. But when I want to close the render window the second time, I get a crash too.

Added subscriber: @hatchli

Added subscriber: @hatchli

Added subscribers: @Michael-Jones, @brecht

Added subscribers: @Michael-Jones, @brecht

CC @michael_jones.

CC @michael_jones.

Added subscriber: @Caden-Mitchell

Added subscriber: @Caden-Mitchell

I also get a crash. With GPU only set in preferences and with CPU+GPU set. My scene also consumes ~30 GB of memory when rendering so maybe it is too advanced for the current Metal GPU render anyways? M1 16 GB RAM, macOS Monterey. It crashes on default cube, too.

I also get a crash. With GPU only set in preferences and with CPU+GPU set. My scene also consumes ~30 GB of memory when rendering so maybe it is too advanced for the current Metal GPU render anyways? M1 16 GB RAM, macOS Monterey. It crashes on default cube, too.

Added subscriber: @michael64

Added subscriber: @michael64

The render on M1 with cycles crash with GPU compute came through 35b1e9fc.
Running Blender in Xcode revealed a few error lines about missing explicit address space qualifiers.

Failed to compile library:
program_source:79651:96: error: pointer type must have explicit address space qualifier
    KernelGlobals kg, ccl_private const ShaderData *sd, const AttributeDescriptor desc, float2 *dx, float2 *dy)

Adding ccl_private in front of every type that was incomplete as in the other shaders was enough to fix the crash.
Please note that I have not tested if the point cloud actually works on M1 (no test scene at hand).

The patch is D13612.
Also #94182 can probably merged into this report.

The render on M1 with cycles crash with GPU compute came through 35b1e9fc. Running Blender in Xcode revealed a few error lines about missing explicit address space qualifiers. ``` Failed to compile library: program_source:79651:96: error: pointer type must have explicit address space qualifier KernelGlobals kg, ccl_private const ShaderData *sd, const AttributeDescriptor desc, float2 *dx, float2 *dy) ``` Adding ccl_private in front of every type that was incomplete as in the other shaders was enough to fix the crash. Please note that I have not tested if the point cloud actually works on M1 (no test scene at hand). The patch is [D13612](https://archive.blender.org/developer/D13612). Also #94182 can probably merged into this report.

This issue was referenced by blender/cycles@f60ff20029

This issue was referenced by blender/cycles@f60ff200293478360216692e85019a861559a77f

This issue was referenced by 67734d1853

This issue was referenced by 67734d18539743494152428e1c1c105f2ee2fd29

Changed status from 'Needs Triage' to: 'Resolved'

Changed status from 'Needs Triage' to: 'Resolved'
Brecht Van Lommel self-assigned this 2021-12-17 14:48:08 +01:00

Changed status from 'Resolved' to: 'Needs Triage'

Changed status from 'Resolved' to: 'Needs Triage'

This report predates point cloud rendering so must have a different cause.

This report predates point cloud rendering so must have a different cause.
Brecht Van Lommel changed title from Blender crashes when using Cycles with Metal GPU in both viewport and renderer to Cycles crash with simultaneous viewport and final render 2021-12-17 14:54:48 +01:00
Brecht Van Lommel changed title from Cycles crash with simultaneous viewport and final render to Cycles Metal crash with simultaneous viewport and final render 2021-12-17 14:55:39 +01:00
Brecht Van Lommel removed their assignment 2021-12-17 14:55:39 +01:00

I forgot to mention the root cause of the crash which I have also seen before the combined M1 + M1 GPU worked.
These crashes happen in intern/cycles/device/metal/kernel.mm:379 'string errors = [err UTF8String];' at the moment:

      for (int i = 0; i < METALRT_FUNC_NUM; i++) {
        const char *function_name = function_names[i];
        desc.name = [@(function_name) copy];

        NSError *error = NULL;
        rt_intersection_funcs[kernel_type][i] = [device->mtlLibrary[kernel_type]
            newFunctionWithDescriptor:desc
                                error:&error];

        if (rt_intersection_funcs[kernel_type][i] == nil) {
          NSString *err = [error localizedDescription];
          string errors = [err UTF8String];

The Metal compiler fails for example when the shader source is incorrent there may be other reasons as well.
Both kernels (device->mtlLibrary[kernel_type]) are therefore nil.
This method call is made on a nil object which is not a bad Objective-C style btw. but a common idiom.

 [device->mtlLibrary[kernel_type] newFunctionWithDescriptor:desc error:&error];

But in this case -[newFunctionWithDescriptor:desc:error] returns nil but *error is also nil.
'[error localizedDescription];' is still ok and returns nil but then [err UTF8String] crashes when err is nil.

That was what I meant, there are different ways to fail a Metal library compilation but so far they all crashed at this line:

    string errors = [err UTF8String];

That could be made more robust.
Also if the environment variable CYCLES_METAL_DUMP_SOURCE is set the source is dumped.
Maybe it is a good idea for the moment to save the metal source string and if this error condition is reached
the metal source is dumped even if CYCLES_METAL_DUMP_SOURCE is not set.

I forgot to mention the root cause of the crash which I have also seen before the combined M1 + M1 GPU worked. These crashes happen in intern/cycles/device/metal/kernel.mm:379 'string errors = [err UTF8String];' at the moment: ``` for (int i = 0; i < METALRT_FUNC_NUM; i++) { const char *function_name = function_names[i]; desc.name = [@(function_name) copy]; NSError *error = NULL; rt_intersection_funcs[kernel_type][i] = [device->mtlLibrary[kernel_type] newFunctionWithDescriptor:desc error:&error]; if (rt_intersection_funcs[kernel_type][i] == nil) { NSString *err = [error localizedDescription]; string errors = [err UTF8String]; ``` The Metal compiler fails for example when the shader source is incorrent there may be other reasons as well. Both kernels (device->mtlLibrary[kernel_type]) are therefore nil. This method call is made on a nil object which is not a bad Objective-C style btw. but a common idiom. ``` [device->mtlLibrary[kernel_type] newFunctionWithDescriptor:desc error:&error]; ``` But in this case -[newFunctionWithDescriptor:desc:error] returns nil but *error is also nil. '[error localizedDescription];' is still ok and returns nil but then [err UTF8String] crashes when err is nil. That was what I meant, there are different ways to fail a Metal library compilation but so far they all crashed at this line: ``` string errors = [err UTF8String]; ``` That could be made more robust. Also if the environment variable CYCLES_METAL_DUMP_SOURCE is set the source is dumped. Maybe it is a good idea for the moment to save the metal source string and if this error condition is reached the metal source is dumped even if CYCLES_METAL_DUMP_SOURCE is not set.

As a workaround, this crash can be fixed by setting the environment variable: CYCLES_METAL_DISABLE_BINARY_ARCHIVES="1"

It occurs when additional instances of the same underlying Pipeline State Objects are created in order to do multi-instanced rendering. Pending further investigations as to the root cause...

As a workaround, this crash can be fixed by setting the environment variable: `CYCLES_METAL_DISABLE_BINARY_ARCHIVES="1"` It occurs when additional instances of the same underlying Pipeline State Objects are created in order to do multi-instanced rendering. Pending further investigations as to the root cause...

Added subscriber: @deadpin

Added subscriber: @deadpin

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'

Will at least confirm this report based on the previous comment.

Will at least confirm this report based on the previous comment.

Will mark as high priority, since there is a workaround we can do for the 3.1 release.

Will mark as high priority, since there is a workaround we can do for the 3.1 release.

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Brecht Van Lommel self-assigned this 2022-01-27 23:38:06 +01:00

Added subscriber: @Carter-LaSalle

Added subscriber: @Carter-LaSalle

I am also having this issue am I am using:
Blender Version: 3.2.0 Alpha, master, 2022-02-20 02:31, 1f79132287
OS: macOS-12.3-arm64-arm-64bit
GPU: Apple M1 Pro Apple 4.1 Metal - 76.3

I am also having this issue am I am using: Blender Version: 3.2.0 Alpha, master, 2022-02-20 02:31, 1f7913228779 OS: macOS-12.3-arm64-arm-64bit GPU: Apple M1 Pro Apple 4.1 Metal - 76.3

@Carter-LaSalle this bug is supposed to be fixed, if you still encounter it please open a new bug report with example .blend file.

@Carter-LaSalle this bug is supposed to be fixed, if you still encounter it please open a new bug report with example .blend file.

Removed subscriber: @Caden-Mitchell

Removed subscriber: @Caden-Mitchell

Added subscriber: @jackattack

Added subscriber: @jackattack

Im still having the exact same issue. Not only is blender crashing for me- my whole computer crashes. Has this maybe resolved? or is there a different thread i can read about the topic?
When rendering in cycles with metal GPU and having viewport also in cycles everything freezes and crashes...

Blender Version: 3.3.1

I mac 2020, Retina, 27"
Processor: 3.6 GHz 10-Core Intel Core i9
Graphic card: AMD Radeon Pro 5700 8 GB
Ram: 16 GB 2667 MHz DDR4

OS ventura 13.0.1

Workaround at the moment is to not have the viewport in cycles when rendering in cycles

Thank you allready :)

Im still having the exact same issue. Not only is blender crashing for me- my whole computer crashes. Has this maybe resolved? or is there a different thread i can read about the topic? When rendering in cycles with metal GPU and having viewport also in cycles everything freezes and crashes... Blender Version: 3.3.1 I mac 2020, Retina, 27" Processor: 3.6 GHz 10-Core Intel Core i9 Graphic card: AMD Radeon Pro 5700 8 GB Ram: 16 GB 2667 MHz DDR4 OS ventura 13.0.1 Workaround at the moment is to not have the viewport in cycles when rendering in cycles Thank you allready :)

@jackattack , thanks for the report. I was unable to reproduce the issue on an M1 Max laptop with 64GB shared memory so I think the root cause may be different. One thing that springs to mind: the Metal backend doesn't handle out-of-memory cases very gracefully yet, so it's possible that doing two renders overflows your GPU's 8GB of VRAM, but one render is okay. Do you know roughly how much memory your scene uses?

@jackattack , thanks for the report. I was unable to reproduce the issue on an M1 Max laptop with 64GB shared memory so I think the root cause may be different. One thing that springs to mind: the Metal backend doesn't handle out-of-memory cases very gracefully yet, so it's possible that doing two renders overflows your GPU's 8GB of VRAM, but one render is okay. Do you know roughly how much memory your scene uses?
Sign in to join this conversation.
12 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: blender/blender#94142
No description provided.