LLVM ERROR: Error while trying to spill SGPR4_SGPR5 from class SReg_64: Cannot scavenge register without an emergency spill slot! #75895

New Issue

Rey Leonard M. Amorato · 2020-04-19T14:26:09+02:00

2020-04-19 14:26:09 +02:00

System Information
Operating system: Linux-5.3.0-46-generic-x86_64-with-Ubuntu-19.10-eoan 64 Bits
Graphics card: AMD Radeon RX 5700 XT (NAVI10, DRM 3.33.0, 5.3.0-46-generic, LLVM 9.0.0) X.Org 4.5 (Core Profile) Mesa 19.2.8
Graphics card driver: amdgpu-pro-20.10-1048554-ubuntu-18.04

Blender Version
Broken: version: 2.90 (sub 0), branch: master, commit date: 2020-04-18 16:48, hash: 1f1520a045
Worked: -

Short description of error
Blender crashes during opencl kernel compilation

Exact steps for others to reproduce the error
Using the supplied blend file, render using GPU (OpenCL)

**System Information** Operating system: Linux-5.3.0-46-generic-x86_64-with-Ubuntu-19.10-eoan 64 Bits Graphics card: AMD Radeon RX 5700 XT (NAVI10, DRM 3.33.0, 5.3.0-46-generic, LLVM 9.0.0) X.Org 4.5 (Core Profile) Mesa 19.2.8 Graphics card driver: amdgpu-pro-20.10-1048554-ubuntu-18.04 **Blender Version** Broken: version: 2.90 (sub 0), branch: master, commit date: 2020-04-18 16:48, hash: `1f1520a045` Worked: - **Short description of error** Blender crashes during opencl kernel compilation **Exact steps for others to reproduce the error** Using the supplied blend file, render using GPU (OpenCL)

Rey Leonard M. Amorato commented

2020-04-19 14:26:09 +02:00

Added subscriber: @RL

Richard Antalik commented

2020-04-20 17:40:31 +02:00

Added subscriber: @iss

Richard Antalik commented

2020-04-20 17:40:31 +02:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Richard Antalik commented

2020-04-20 17:40:31 +02:00

Can you try to strip .blend file down as much as possible? It is possible, that you will end up with very small file that wouldn't contain any data you can't share.

Also please click on File > Defaults > Load Fatctory Setings and let us know if this helps.

Can you try to strip .blend file down as much as possible? It is possible, that you will end up with very small file that wouldn't contain any data you can't share. Also please click on File > Defaults > Load Fatctory Setings and let us know if this helps.

Rey Leonard M. Amorato commented

2020-04-21 07:23:39 +02:00

I've managed to recreate a scene that produces the crash on my system. Note that it's not specific to this material; any material I append or link from my external material library will produce a crash on render.

This was ran with blender --factory-startup.

TEST-GPU-RENDER.blend

I've managed to recreate a scene that produces the crash on my system. Note that it's not specific to this material; any material I append or link from my external material library will produce a crash on render. This was ran with blender --factory-startup. [TEST-GPU-RENDER.blend](https://archive.blender.org/developer/F8486478/TEST-GPU-RENDER.blend)

Richard Antalik commented

2020-04-21 15:32:51 +02:00

This haven't crashed in my case, but render result was quite wild

Or should I make library and append it to example file and add material from lib to cube?

This haven't crashed in my case, but render result was quite wild ![untitled.png](https://archive.blender.org/developer/F8487407/untitled.png) Or should I make library and append it to example file and add material from lib to cube?

Rey Leonard M. Amorato commented

2020-04-21 17:03:05 +02:00

Are you using the same GPU?

Richard Antalik commented

2020-04-21 19:17:44 +02:00

In #75895#914882, @RL wrote:
Are you using the same GPU?

No
Operating system: Windows-10-10.0.18362-SP0 64 Bits
Graphics card: Radeon RX550/550 Series ATI Technologies Inc. 4.5.13587 Core Profile Context 20.4.1 26.20.15029.20013

> In #75895#914882, @RL wrote: > Are you using the same GPU? No Operating system: Windows-10-10.0.18362-SP0 64 Bits Graphics card: Radeon RX550/550 Series ATI Technologies Inc. 4.5.13587 Core Profile Context 20.4.1 26.20.15029.20013

secureworkstation commented

2020-04-21 22:46:41 +02:00

Added subscriber: @secureworkstation

secureworkstation commented

2020-04-21 22:46:41 +02:00

I can confirm the issue. It started to happen once I updated the AMDGPU-PRO OpenCL driver from 19.50 to 20.10.

My software is similar to @RL's. The same Ubuntu driver, Blender master. Hardware - 5700 XT too. As a temporary measure, I recommend downgrading to 19.50... even though it crashes a lot for me.

I run it under Fedora though. I repackage the Ubuntu driver: https://github.com/secureworkstation/rpm-amdgpu-pro-opencl

I can confirm the issue. It started to happen once I updated the AMDGPU-PRO OpenCL driver from 19.50 to 20.10. My software is similar to @RL's. The same Ubuntu driver, Blender master. Hardware - 5700 XT too. As a temporary measure, I recommend downgrading to 19.50... even though it crashes a lot for me. I run it under Fedora though. I repackage the Ubuntu driver: https://github.com/secureworkstation/rpm-amdgpu-pro-opencl

Rey Leonard M. Amorato commented

2020-04-21 23:41:52 +02:00

@iss I think the issue is exclusive to this particular setup, at least the GPU.

@secureworkstation I'm kinda glad I'm not the only one having problems with this GPU. I've had numerous problems with it since.
With the 19.50 driver, the render just hangs indefinitely, see #74262

@iss I think the issue is exclusive to this particular setup, at least the GPU. @secureworkstation I'm kinda glad I'm not the only one having problems with this GPU. I've had numerous problems with it since. With the 19.50 driver, the render just hangs indefinitely, see #74262

secureworkstation commented

2020-04-22 00:06:38 +02:00

Yes - it happens to hang quite randomly and then I can't reboot Blender (seems like some Linux kernel issue) and I must hard reboot the whole machine. I found some workaround - it's possible to render (ALMOST always hang-free) in text mode (as in, Ctrl+Alt+F2, not XTerm) using:

$ blender --background test.blend --render-output out --render-frame 1

Sometimes it kills my Wayland session, but I learned to live with it. I hoped that with the 20.10 driver it works better, but I guess we need some fix to the Blender kernels to acomodate the stack upgrades, but unfortunately that's outside of my expertise.

Yes - it happens to hang quite randomly and then I can't reboot Blender (seems like some Linux kernel issue) and I must hard reboot the whole machine. I found some workaround - it's possible to render (ALMOST always hang-free) in text mode (as in, Ctrl+Alt+F2, not XTerm) using: $ blender --background test.blend --render-output out --render-frame 1 Sometimes it kills my Wayland session, but I learned to live with it. I hoped that with the 20.10 driver it works better, but I guess we need some fix to the Blender kernels to acomodate the stack upgrades, but unfortunately that's outside of my expertise.

Brecht Van Lommel commented

2020-04-22 00:13:14 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2020-04-22 00:13:14 +02:00

This error message seems to indicate a bug in the AMD OpenCL driver, an internal compiler error. We may be able to work around it, or get AMD to fix it.

I will try to install this driver version and reproduce.

This error message seems to indicate a bug in the AMD OpenCL driver, an internal compiler error. We may be able to work around it, or get AMD to fix it. I will try to install this driver version and reproduce.

secureworkstation commented

2020-04-22 00:19:29 +02:00

I tested the 20.10 a bit further. For rendering it works well - as in, the kernel compilation works well, but under a Wayland session it hangs - the same as with 19.50, it only crashes when I change the shading to Viewport Shading. I assume the kernels are compiled a bit differently in those cases.

Brecht Van Lommel commented

2020-04-22 16:08:58 +02:00

For reference, 20.10-1048554 works fine for final and viewport render here with a WX 7100 on Ubuntu 18.04. Installed using:

amdgpu-pro-install --headless --opencl=legacy,pal

For reference, 20.10-1048554 works fine for final and viewport render here with a WX 7100 on Ubuntu 18.04. Installed using: ``` amdgpu-pro-install --headless --opencl=legacy,pal ```

Jeroen Bakker commented

2020-04-22 16:09:30 +02:00

Added subscriber: @Jeroen-Bakker

secureworkstation commented

2020-04-28 21:03:10 +02:00

@brecht Well, our installations do differ somehow. Let's find out what causes those issues.

@secureworkstation - upstream Linux AMDKFD driver, unclean installation, Ubuntu driver, RDNA card, PAL driver
@RL - AMD packaged AMDKFD driver (dkms-amdgpu), clean installation, Ubuntu driver, RDNA card, PAL driver
@brecht - AMD packaged AMDKFD driver (dkms-amdgpu), clean installation, Ubuntu driver, GCN 4g card, ORCA (legacy) driver

Please fix if I am wrong. But if I'm right, then it seems that the issue persists at least across users the of the PAL driver, at most across users of the RDNA generation cards.

@brecht Well, our installations do differ somehow. Let's find out what causes those issues. @secureworkstation - upstream Linux AMDKFD driver, unclean installation, Ubuntu driver, RDNA card, PAL driver @RL - AMD packaged AMDKFD driver (dkms-amdgpu), clean installation, Ubuntu driver, RDNA card, PAL driver @brecht - AMD packaged AMDKFD driver (dkms-amdgpu), clean installation, Ubuntu driver, GCN 4g card, ORCA (legacy) driver Please fix if I am wrong. But if I'm right, then it seems that the issue persists at least across users the of the PAL driver, at most across users of the RDNA generation cards.

secureworkstation commented

2020-04-28 21:10:31 +02:00

This is how we can ensure that we use the PAL driver (which is for Vega10+, and WX 7100 is Polaris10):

$ clinfo | grep 'Driver Version'
  Driver Version                                  3004.6 (PAL,LC)

This is how we can ensure that we use the PAL driver (which is for Vega10+, and WX 7100 is Polaris10): ``` $ clinfo | grep 'Driver Version' Driver Version 3004.6 (PAL,LC) ```

Brecht Van Lommel commented

2020-04-28 21:22:32 +02:00

What I have installed currently is:

$ clinfo | grep 'Driver Version
 Driver Version                                  3075.9 (PAL,HSAIL)

What I have installed currently is: ``` $ clinfo | grep 'Driver Version Driver Version 3075.9 (PAL,HSAIL) ```

Rey Leonard M. Amorato commented

2020-04-29 07:44:21 +02:00

$ /opt/amdgpu-pro/bin/clinfo | grep -i "driver version"
  Driver version:				 3075.10 (PAL,LC)

I don't think I'm using the dkms version. I installed my driver with amdgpu-pro-install --opencl=pal --headless --no-dkms

I've also upgraded my system to Ubuntu 20.04, though I don't think that's essential to the equation here. Attaching system info and clinfo output.

system-info.txt

clinfo.txt

``` $ /opt/amdgpu-pro/bin/clinfo | grep -i "driver version" Driver version: 3075.10 (PAL,LC) ``` I don't think I'm using the dkms version. I installed my driver with `amdgpu-pro-install --opencl=pal --headless --no-dkms` I've also upgraded my system to Ubuntu 20.04, though I don't think that's essential to the equation here. Attaching system info and clinfo output. [system-info.txt](https://archive.blender.org/developer/F8500541/system-info.txt) [clinfo.txt](https://archive.blender.org/developer/F8500542/clinfo.txt)

Brecht Van Lommel commented

2020-04-29 18:05:28 +02:00

I guess the difference is in the HSAIL and LC intermediate languages. Only Navi hardware uses LC.

I'm not sure it's safe to install using --no-dkms though, maybe that leads to a mismatch between the kernel module and the rest of the driver.

I guess the difference is in the HSAIL and LC intermediate languages. Only Navi hardware uses LC. I'm not sure it's safe to install using `--no-dkms` though, maybe that leads to a mismatch between the kernel module and the rest of the driver.

Rey Leonard M. Amorato commented

2020-04-30 05:55:19 +02:00

Installing with dkms breaks my whole display stack, (Unable to login to desktop; black screen with blinking white underscore) which is understandable because neither Ubuntu 20.04 nor 19.10 is supported by the current driver, hence the --no-dkms

What I find weird is if you look at my original report, I was able to use OpenCL (--no-dkms) without problems using Ubuntu 19.10 with the amdgpu-pro-19.30-838629-ubuntu-18.04 and amdgpu-pro-19.50-967956 drivers, until the USD changes. So it might not be entirely the driver's fault.

Installing with dkms breaks my whole display stack, (Unable to login to desktop; black screen with blinking white underscore) which is understandable because neither Ubuntu 20.04 nor 19.10 is supported by the current driver, hence the `--no-dkms` What I find weird is if you look at my original report, I was able to use OpenCL (`--no-dkms`) without problems using Ubuntu 19.10 with the `amdgpu-pro-19.30-838629-ubuntu-18.04` and `amdgpu-pro-19.50-967956` drivers, until the USD changes. So it might not be entirely the driver's fault.

Jeroen Bakker commented

2020-04-30 10:14:37 +02:00

Unable to reproduce using Ubuntu 18.04.4 HWE + amdgpu-pro-20.10

system-info.txt

When upgrading I made sure that I completely removed the previous amdgpu-pro driver. Did a clean boot. and installed the OpenCL drivers only of the amdgpu-pro-20.10.

amdgpu-pro-install --headless --opencl=PAL

I used the open source drivers for display.

Unable to reproduce using Ubuntu 18.04.4 HWE + amdgpu-pro-20.10 [system-info.txt](https://archive.blender.org/developer/F8502569/system-info.txt) When upgrading I made sure that I completely removed the previous amdgpu-pro driver. Did a clean boot. and installed the OpenCL drivers only of the amdgpu-pro-20.10. `amdgpu-pro-install --headless --opencl=PAL` I used the open source drivers for display.

Brecht Van Lommel commented

2020-04-30 10:18:02 +02:00

@Jeroen-Bakker, can you post clinfo output? Did you check both F12 render and viewport render?

@Jeroen-Bakker, can you post `clinfo` output? Did you check both F12 render and viewport render?

Jeroen Bakker commented

2020-04-30 10:22:44 +02:00

clinfo.txt I tested with f12 render, but yes it breaks during viewport rendering.

[clinfo.txt](https://archive.blender.org/developer/F8502581/clinfo.txt) I tested with f12 render, but yes it breaks during viewport rendering.

Rey Leonard M. Amorato commented

2020-04-30 12:25:53 +02:00

Can we also test on Ubuntu 20.04 LTS? I understand it's not currently supported by the driver, it's just so we have an apple-to-apple comparison.

The way I did it is to edit /etc/os-release and change VERSION_ID="20.04" to VERSION_ID="18.04" so the installer won't complain that the OS is not supported.

Can we also test on Ubuntu 20.04 LTS? I understand it's not currently supported by the driver, it's just so we have an apple-to-apple comparison. The way I did it is to edit `/etc/os-release` and change `VERSION_ID="20.04"` to `VERSION_ID="18.04"` so the installer won't complain that the OS is not supported.

Jeroen Bakker self-assigned this 2020-04-30 13:19:00 +02:00

Jeroen Bakker commented

2020-04-30 13:19:00 +02:00

I isolated the part that fails to compile. You could check if it works with P1357: (An Untitled Masterwork)

diff --git a/intern/cycles/kernel/svm/svm_voronoi.h b/intern/cycles/kernel/svm/svm_voronoi.h
index 2ad22592eef..e4fe91095ce 100644
--- a/intern/cycles/kernel/svm/svm_voronoi.h
+++ b/intern/cycles/kernel/svm/svm_voronoi.h
@@ -1089,6 +1089,7 @@ ccl_device void svm_node_tex_voronoi(KernelGlobals *kg,
                                &color_out,
                                &position_out_4d);
           break;
+#  if 0
         case NODE_VORONOI_F2:
           voronoi_f2_4d(coord_4d,
                         exponent,
@@ -1098,6 +1099,7 @@ ccl_device void svm_node_tex_voronoi(KernelGlobals *kg,
                         &color_out,
                         &position_out_4d);
           break;
+#  endif
         case NODE_VORONOI_DISTANCE_TO_EDGE:
           voronoi_distance_to_edge_4d(coord_4d, randomness, &distance_out);
           break;

applied.

I isolated the part that fails to compile. You could check if it works with [P1357: (An Untitled Masterwork)](https://archive.blender.org/developer/P1357.txt) ``` diff --git a/intern/cycles/kernel/svm/svm_voronoi.h b/intern/cycles/kernel/svm/svm_voronoi.h index 2ad22592eef..e4fe91095ce 100644 --- a/intern/cycles/kernel/svm/svm_voronoi.h +++ b/intern/cycles/kernel/svm/svm_voronoi.h @@ -1089,6 +1089,7 @@ ccl_device void svm_node_tex_voronoi(KernelGlobals *kg, &color_out, &position_out_4d); break; +# if 0 case NODE_VORONOI_F2: voronoi_f2_4d(coord_4d, exponent, @@ -1098,6 +1099,7 @@ ccl_device void svm_node_tex_voronoi(KernelGlobals *kg, &color_out, &position_out_4d); break; +# endif case NODE_VORONOI_DISTANCE_TO_EDGE: voronoi_distance_to_edge_4d(coord_4d, randomness, &distance_out); break; ``` applied.

Rey Leonard M. Amorato commented

2020-04-30 13:56:41 +02:00

Successful renders with both F12 and viewport! Simply genius! Thank you!

blender-admin commented

2020-04-30 14:15:10 +02:00

This issue was referenced by blender/cycles@c96dfd93f8

This issue was referenced by blender/cycles@c96dfd93f8bdeda625d79ac4ac2b7b661ca6a5c9

blender-admin commented

2020-04-30 14:15:10 +02:00

This issue was referenced by 6121c28501

This issue was referenced by 6121c28501eff722717fb8b777f6004fb6d4e152

Jeroen Bakker commented

2020-04-30 15:07:48 +02:00

Changed status from 'Needs User Info' to: 'Resolved'

Jeroen Bakker closed this issue

2020-04-30 15:07:48 +02:00

secureworkstation commented

2020-05-02 07:30:07 +02:00

Everything works with the latest master. Thank you!

Furthermore, it seems that since Linux kernel 5.6.7-300.fc32.x86_64 the freezes (unrelated to this bug) are gone.

Everything works with the latest master. Thank you! Furthermore, it seems that since Linux kernel 5.6.7-300.fc32.x86_64 the freezes (unrelated to this bug) are gone.

Alex Barcelo commented

2020-05-11 11:00:33 +02:00

Added subscriber: @alexbarcelo

Alex Barcelo commented

2020-05-11 11:00:33 +02:00

This comment was removed by @alexbarcelo

*This comment was removed by @alexbarcelo*

Arvind commented

2020-07-14 15:38:57 +02:00

Added subscriber: @Arvind_Ravichandran

Arvind commented

2020-07-14 15:38:57 +02:00

In #75895#920575, @Jeroen-Bakker wrote:
Unable to reproduce using Ubuntu 18.04.4 HWE + amdgpu-pro-20.10

system-info.txt

When upgrading I made sure that I completely removed the previous amdgpu-pro driver. Did a clean boot. and installed the OpenCL drivers only of the amdgpu-pro-20.10.

amdgpu-pro-install --headless --opencl=PAL

I used the open source drivers for display.

The issue is only with amdgpu-pro drivers <> NAVI architecture card (Radeon Rx 5000 series) combination. F12 render crashes during kernel compile only if the scene has volumetrics. Entering viewport rendering mode always crashes. Have tested my card with amdgpu 20.10 with the latest stable build (2.83.2) of blender and it still has the issue.

> In #75895#920575, @Jeroen-Bakker wrote: > Unable to reproduce using Ubuntu 18.04.4 HWE + amdgpu-pro-20.10 > > [system-info.txt](https://archive.blender.org/developer/F8502569/system-info.txt) > > When upgrading I made sure that I completely removed the previous amdgpu-pro driver. Did a clean boot. and installed the OpenCL drivers only of the amdgpu-pro-20.10. > > `amdgpu-pro-install --headless --opencl=PAL` > > I used the open source drivers for display. > The issue is only with amdgpu-pro drivers <> NAVI architecture card (Radeon Rx 5000 series) combination. F12 render crashes during kernel compile only if the scene has volumetrics. Entering viewport rendering mode always crashes. Have tested my card with amdgpu 20.10 with the latest stable build (2.83.2) of blender and it still has the issue.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

LLVM ERROR: Error while trying to spill SGPR4_SGPR5 from class SReg_64: Cannot scavenge register without an emergency spill slot! #75895