Cycles HIP crash with smoke volume #92984

New Issue

Thomas Dinges · 2021-11-10T10:14:33+01:00

Thomas Dinges commented

2021-11-10 10:14:33 +01:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: Radeon RX 5500 XT ATI Technologies Inc. 4.5.14802 Core Profile Context 21.Q4 30.0.14002.136

Blender Version
Broken: version: 3.0.0 Beta, branch: cycles-hip-binaries, commit date: 2021-11-09 21:58, hash: ab57a3cfac

Short description of error
Cycles HIP rendering crash with Quick Smoke

Exact steps for others to reproduce the error
Run Quick Smoke operator (F3, type quick smoke), set Cycles to GPU rendering with HIP and press F12.

Crashlog: blender.crash.txt

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: Radeon RX 5500 XT ATI Technologies Inc. 4.5.14802 Core Profile Context 21.Q4 30.0.14002.136 **Blender Version** Broken: version: 3.0.0 Beta, branch: cycles-hip-binaries, commit date: 2021-11-09 21:58, hash: `ab57a3cfac` **Short description of error** Cycles HIP rendering crash with Quick Smoke **Exact steps for others to reproduce the error** Run Quick Smoke operator (F3, type quick smoke), set Cycles to GPU rendering with HIP and press F12. Crashlog: [blender.crash.txt](https://archive.blender.org/developer/F11784695/blender.crash.txt)

Thomas Dinges commented

2021-11-10 10:14:33 +01:00

Added subscriber: @ThomasDinges

Jeroen Bakker commented

2021-11-10 10:35:37 +01:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2021-11-10 10:35:37 +01:00

I can confirm using a W6800.

Thomas Dinges commented

2021-11-10 10:45:40 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Brecht Van Lommel commented

2021-11-10 21:33:42 +01:00

Added subscribers: @Sayak-Biswas, @BrianSavery, @brecht

Brecht Van Lommel commented

2021-11-10 21:33:42 +01:00

CC @Sayak-Biswas @bsavery.

(Let me know if we should notify only one of you or someone else on future bug reports, not sure how you want to handle this.)

CC @Sayak-Biswas @bsavery. (Let me know if we should notify only one of you or someone else on future bug reports, not sure how you want to handle this.)

Brian Savery (AMD) self-assigned this 2021-11-10 21:55:54 +01:00

Brian Savery (AMD) commented

2021-11-10 21:55:54 +01:00

Sure, I can. Is there a scene file here?

Ray molenkamp commented

2021-11-10 22:00:06 +01:00

Added subscriber: @LazyDodo

Ray molenkamp commented

2021-11-10 22:00:06 +01:00

not needed, instructions are in the description, i'll clarify a bit just in case

Thomas Dinges commented

2021-11-10 22:47:50 +01:00

quick_smoke.blend
Just in case, here is a ready to crash blend file :)

[quick_smoke.blend](https://archive.blender.org/developer/F11790675/quick_smoke.blend) Just in case, here is a ready to crash blend file :)

周建峰 commented

2021-11-11 12:48:15 +01:00

Added subscriber: @2905710881

周建峰 commented

2021-11-11 12:48:15 +01:00

In #92984#1252037, @ThomasDinges wrote:
quick_smoke.blend
Just in case, here is a ready to crash blend file :)

Why amd gpu often have problems with smoke, 2.93 opencl can not render smoke bug has not been fixed yet, but he can render the volume (add volume in the environment)

> In #92984#1252037, @ThomasDinges wrote: > [quick_smoke.blend](https://archive.blender.org/developer/F11790675/quick_smoke.blend) > Just in case, here is a ready to crash blend file :) Why amd gpu often have problems with smoke, 2.93 opencl can not render smoke bug has not been fixed yet, but he can render the volume (add volume in the environment)

Thomas Dinges commented

2021-11-11 12:50:30 +01:00

@2905710881 Please, this is a site for developers who work on the code, not for discussions. The HIP backend is new and just needs some fixes, that's to be expected with new code.

Mike Pan commented

2021-11-11 19:10:22 +01:00

Added subscriber: @MikePan

周建峰 commented

2021-11-12 20:20:13 +01:00

In #92984#1252306, @ThomasDinges wrote:
@2905710881 Please, this is a site for developers who work on the code, not for discussions. The HIP backend is new and just needs some fixes, that's to be expected with new code.

I'm very sorry, I may seem a bit anxious.

> In #92984#1252306, @ThomasDinges wrote: > @2905710881 Please, this is a site for developers who work on the code, not for discussions. The HIP backend is new and just needs some fixes, that's to be expected with new code. I'm very sorry, I may seem a bit anxious.

Diogo Valadares Reis dos Santos commented

2021-11-15 19:41:02 +01:00

Added subscriber: @Diogo_Valadares

Diogo Valadares Reis dos Santos commented

2021-11-15 19:41:02 +01:00

it also crashes on 5700xt with AMD 21.11.2. just volume BSDF doesn't crash, but quick smoke crashes instantly.

Brecht Van Lommel commented

2021-11-18 14:08:17 +01:00

With OpenVDB/NanoVDB volume rendering working now, we could work around this by converting dense 3D volume to sparse NanoVDB for HIP. Though that's not ideal for e.g. viewport interactivity with smoke simulation playback, so it would still be good to fix the core of this issue.

In the backtrace we have:

Stack trace:
amdhip64.dll        :0x00007FFA01AA3590  hipTexRefGetMipmappedArray
amdhip64.dll        :0x00007FFA01AAEFA0  hipTexObjectCreate
blender.exe         :0x00007FF7A0BF44A0  ccl::HIPDevice::tex_alloc

We are passing the 3D texture as an array, not a mipmapped array, so potentially that hipTexRefGetMipmappedArray should not have been called.

I checked the open source Linux implementation to see if there is some condition where this could happen, but as far as I can tell it never internally calls hipTexRefGetMipmappedArray, it's only a public API function.
d2d2cacfe2/src/hip_texture.cpp

So I assume the Windows implementation is different here.

With OpenVDB/NanoVDB volume rendering working now, we could work around this by converting dense 3D volume to sparse NanoVDB for HIP. Though that's not ideal for e.g. viewport interactivity with smoke simulation playback, so it would still be good to fix the core of this issue. In the backtrace we have: ``` Stack trace: amdhip64.dll :0x00007FFA01AA3590 hipTexRefGetMipmappedArray amdhip64.dll :0x00007FFA01AAEFA0 hipTexObjectCreate blender.exe :0x00007FF7A0BF44A0 ccl::HIPDevice::tex_alloc ``` We are passing the 3D texture as an array, not a mipmapped array, so potentially that `hipTexRefGetMipmappedArray` should not have been called. I checked the open source Linux implementation to see if there is some condition where this could happen, but as far as I can tell it never internally calls `hipTexRefGetMipmappedArray`, it's only a public API function. https://github.com/ROCm-Developer-Tools/hipamd/blob/d2d2cacfe210307ec10c77400e1dafdeafefbc0f/src/hip_texture.cpp So I assume the Windows implementation is different here.

Brecht Van Lommel commented

2021-11-19 16:41:45 +01:00

@BrianSavery, @Sayak-Biswas, is there any update on this from your side? We really like to have all high priority bugs fixed at least a week before the release, which means by next Wednesday.

We could work around this by using NanoVDB if there is no solution to use 3D textures, but would like to know as soon as possible then.

@BrianSavery, @Sayak-Biswas, is there any update on this from your side? We really like to have all high priority bugs fixed at least a week before the release, which means by next Wednesday. We could work around this by using NanoVDB if there is no solution to use 3D textures, but would like to know as soon as possible then.

Sayak Biswas commented

2021-11-19 18:04:41 +01:00

I am working on this. Is the quick smoke feature working with NanoVDB enabled? I tried it with the latest from the release branch and it still seems to crash. Debugging the crash, it seems to me like a pointer for the hipArray isn't correctly setup in hipew. I'm looking into this more, will try and update more details by Monday.

Brecht Van Lommel commented

2021-11-19 18:13:46 +01:00

Thanks for the update.

(Quick) smoke in Blender does not use NanoVDB, so it is expected that this is still crashing. We could make it uses NanoVDB if we can't solve the 3D texture issue in time.

Thanks for the update. (Quick) smoke in Blender does not use NanoVDB, so it is expected that this is still crashing. We could make it uses NanoVDB if we can't solve the 3D texture issue in time.

Sayak Biswas commented

2021-11-19 20:55:02 +01:00

Okay, I think I have a fix for the issue with quick smoke{F11829719}. How can I test other volume rendering? Is there a sample scene?

Okay, I think I have a fix for the issue with quick smoke{[F11829719](https://archive.blender.org/developer/F11829719/image.png)}. How can I test other volume rendering? Is there a sample scene?

Ray molenkamp commented

2021-11-19 23:08:13 +01:00

There's various volume tests available in the unit tests, unsure if there's an easy way to make them automatically run on the GPU these days , but you should be able to manually open/render them.

There's various [volume tests available](https://svn.blender.org/svnroot/bf-blender/tags/blender-2.93-release/lib/tests/render/volume/) in the unit tests, unsure if there's an easy way to make them automatically run on the GPU these days , but you should be able to manually open/render them.

Thomas Dinges commented

2021-11-19 23:59:15 +01:00

Other volume rendering didn't show issues in my tests, the issue was only related to 3D textures, which is used for Smoke/Fire. So if the Quick Smoke works now, that should be fine.

Sayak Biswas commented

2021-11-20 00:01:14 +01:00

Okay, cool! Running through the Cycles demos once to make sure nothing else broke. Will submit that patch in a bit.

Brecht Van Lommel commented

2021-11-20 00:06:45 +01:00

That's great.

For reference, running the tests with HIP would be done by adding HIP to CYCLES_TEST_DEVICES. There's more detail here on how to set up automated tests here:
https://wiki.blender.org/wiki/Tools/Tests/Setup

That's great. For reference, running the tests with HIP would be done by adding `HIP` to `CYCLES_TEST_DEVICES`. There's more detail here on how to set up automated tests here: https://wiki.blender.org/wiki/Tools/Tests/Setup

blender-admin commented

2021-11-20 01:26:30 +01:00

This issue was referenced by blender/cycles@bec1ee3891

This issue was referenced by blender/cycles@bec1ee3891055257119c672f3e0e6a79579bc1ae

blender-admin commented

2021-11-20 01:26:30 +01:00

This issue was referenced by f2bb42a095

This issue was referenced by f2bb42a095acde57c540ff619c422132f491ef1c

Brecht Van Lommel commented

2021-11-20 14:04:46 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Brecht Van Lommel closed this issue