CPU render + OptiX denoiser leaves empty tiles unprocessed (Blender from Master) #84650

New Issue

YAFU · 2021-01-12T18:42:39+01:00

YAFU commented

2021-01-12 18:42:39 +01:00

System Information
Operating system: Linux-5.4.0-14-generic-x86_64-with-debian-bullseye-sid 64 Bits
Graphics card: GeForce GTX 960/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.32.03

CPU: i7 3770

Blender Version
Broken: version:

2.92.0 Beta, branch: master, commit date: 2021-01-17 10:36, hash: 2ee2f87f29
2.92.0 Alpha, branch: master, commit date: 2021-01-11 21:30, hash: acbda123ad
Worked:
2.92.0 Alpha, branch: master, commit date: 2021-01-07 22:28, hash: 61f1faac3f (for @rjg)
2.91.0

Short description of error
When I render in Cycles with Sampling> Denoising> Render> Optix, strange things happen. The resulting render is left with empty tiles / holes and if you cancel (Esc) the render job you get the following error message:

CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816)

CPU_OptiX denoiser problem.blend

Depending on the scene there may be more than one empty tile.

Probably related to this fix?:
https://developer.blender.org/T83925#1087610

**System Information** Operating system: Linux-5.4.0-14-generic-x86_64-with-debian-bullseye-sid 64 Bits Graphics card: GeForce GTX 960/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.32.03 CPU: i7 3770 **Blender Version** Broken: version: - 2.92.0 Beta, branch: master, commit date: 2021-01-17 10:36, hash: `2ee2f87f29` - 2.92.0 Alpha, branch: master, commit date: 2021-01-11 21:30, hash: `acbda123ad` Worked: - 2.92.0 Alpha, branch: master, commit date: 2021-01-07 22:28, hash: `61f1faac3f` (for @rjg) - 2.91.0 **Short description of error** When I render in Cycles with Sampling> Denoising> Render> Optix, strange things happen. The resulting render is left with empty tiles / holes and if you cancel (Esc) the render job you get the following error message: ``` CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816) ``` [CPU_OptiX denoiser problem.blend](https://archive.blender.org/developer/F9561290/CPU_OptiX_denoiser_problem.blend) ![cpu_optix_denoiser.jpg](https://archive.blender.org/developer/F9561291/cpu_optix_denoiser.jpg) Depending on the scene there may be more than one empty tile. Probably related to this fix?: https://developer.blender.org/T83925#1087610

YAFU commented

2021-01-12 18:42:39 +01:00

Added subscriber: @YAFU

blender-admin commented

2021-01-12 18:42:39 +01:00

#84793 was marked as duplicate of this issue

YAFU changed title from ~~CPU render + OptiX denoiser leaves empty tiles unprocessed~~ to CPU render + OptiX denoiser leaves empty tiles unprocessed (Blender from Master)

2021-01-12 18:43:13 +01:00

Robert Guetzkow commented

2021-01-13 12:06:05 +01:00

Added subscriber: @rjg

Robert Guetzkow commented

2021-01-13 12:06:05 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Robert Guetzkow commented

2021-01-13 12:06:05 +01:00

I couldn't reproduce the issue on Windows, it renders fine and I'm not seeing that error.

Have you checked if this is the first error message you're getting?
~~Have you checked that your TDR delay is high enough?~~
What is the most recent version that worked correctly on your system?

I couldn't reproduce the issue on Windows, it renders fine and I'm not seeing that error. - Have you checked if this is the first error message you're getting? - ~~Have you checked that your TDR delay is high enough?~~ - What is the most recent version that worked correctly on your system?

YAFU commented

2021-01-13 12:36:16 +01:00

The latest version I know of that works without problems is 2.91.0. From master in versions prior to January 5 the file Crash Blender, this before this Fix:
https://developer.blender.org/rB3373d14b1b05b2ee5dc88efff5dc8b1a5fe20f36

TDR delay is primarily a Windows problem I understand.
The aforementioned error message is the only one I get when finishing the render or canceling the render. After that, without closing blender the next time I try to render I get the message: Failed to create CUDA context (Illegal address)

The latest version I know of that works without problems is 2.91.0. From master in versions prior to January 5 the file Crash Blender, this before this Fix: https://developer.blender.org/rB3373d14b1b05b2ee5dc88efff5dc8b1a5fe20f36 TDR delay is primarily a Windows problem I understand. The aforementioned error message is the only one I get when finishing the render or canceling the render. After that, without closing blender the next time I try to render I get the message: Failed to create CUDA context (Illegal address)

Robert Guetzkow commented

2021-01-13 13:06:28 +01:00

Changed status from 'Needs User Info' to: 'Needs Triage'

Robert Guetzkow commented

2021-01-13 13:41:26 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Robert Guetzkow commented

2021-01-13 13:41:26 +01:00

Unfortunately, I don't have a Nvidia GPU under Linux, so I can't test this specific case. You're right about TDR that is Windows only.

Could you please run Blender with the following debug flags and upload the log file.

./blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1

Unfortunately, I don't have a Nvidia GPU under Linux, so I can't test this specific case. You're right about TDR that is Windows only. Could you please run Blender with the following debug flags and upload the log file. ```lines ./blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1 ```

YAFU commented

2021-01-13 15:00:36 +01:00

Ok I'll do it. But first I would like to know why when I open Blender in the terminal or in the log, confidential information such as my email address, user and username appears. I am opening blender without being logged into any blender site, and I have also downloaded blender without being logged into any blender site. How does Blender collect that information? The information shown seems to be from my old gmail account that I was registered with here, so I'm really confused.

I guess it could be some addon?

EDIT:
Ok, I have run with an empty config folder and the confidential information does not appear. I am going to investigate which addon it is. BlenderKit is the only one that I remember having to sign up for.
Here is the log file:
blender_debug_output.txt

Ok I'll do it. But first I would like to know why when I open Blender in the terminal or in the log, confidential information such as my email address, user and username appears. I am opening blender without being logged into any blender site, and I have also downloaded blender without being logged into any blender site. How does Blender collect that information? The information shown seems to be from my old gmail account that I was registered with here, so I'm really confused. I guess it could be some addon? EDIT: Ok, I have run with an empty config folder and the confidential information does not appear. I am going to investigate which addon it is. BlenderKit is the only one that I remember having to sign up for. Here is the log file: [blender_debug_output.txt](https://archive.blender.org/developer/F9577312/blender_debug_output.txt)

Robert Guetzkow commented

2021-01-13 15:09:44 +01:00

Blender doesn't collect, save or require an email address or username. Perhaps that is printed by some add-on the requires an online account?

The log file may contain paths and therefore the name of your current active user on your system. That isn't considered a secret though, but you can redact that information from the log file if you want.

Blender doesn't collect, save or require an email address or username. Perhaps that is printed by some add-on the requires an online account? The log file may contain paths and therefore the name of your current active user on your system. That isn't considered a secret though, but you can redact that information from the log file if you want.

YAFU commented

2021-01-13 15:18:11 +01:00

Thank you. I have edited the previous message.

YAFU commented

2021-01-13 17:58:16 +01:00

If from Debug menu I set "CUDA Streams = 10" the number of empty tiles changes some times:

If from Debug menu I set "CUDA Streams = 10" the number of empty tiles changes some times: ![CUDA streams 10.jpg](https://archive.blender.org/developer/F9577569/CUDA_streams_10.jpg) ![CUDA streams 10_2.jpg](https://archive.blender.org/developer/F9577578/CUDA_streams_10_2.jpg)

YAFU commented

2021-01-15 13:35:40 +01:00

Added subscriber: @pmoursnv

YAFU commented

2021-01-15 13:35:40 +01:00

Hi @pmoursnv , I dare to quote you since there are already several reports similar to this and because 2.92 is beta now:

https://developer.blender.org/T84711

https://developer.blender.org/T84746

Thanks.

Hi @pmoursnv , I dare to quote you since there are already several reports similar to this and because 2.92 is beta now: https://developer.blender.org/T84711 https://developer.blender.org/T84746 Thanks.

CRAZ commented

2021-01-15 19:58:38 +01:00

Added subscriber: @zrp

CRAZ commented

2021-01-15 20:15:06 +01:00

I'm getting the same error on the above file. and it does occur in large larger scale environments too.

Error: CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816)CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816)

System: Windows 10 Pro 64bit, Ryzen 5800x, Nvidia 3070 461.09

Blender Version 2.92 Beta

I'm getting the same error on the above file. and it does occur in large larger scale environments too. Error: CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816)CUDA_ERROR_ILLEGAL_ADDRESS in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:816) System: Windows 10 Pro 64bit, Ryzen 5800x, Nvidia 3070 461.09 Blender Version 2.92 Beta ![error_cuda.PNG](https://archive.blender.org/developer/F9580793/error_cuda.PNG)

Robert Guetzkow commented

2021-01-18 13:48:53 +01:00

I'm able to reproduce this in 2.92.0 Beta, branch: master, commit date: 2021-01-17 10:36, hash: 2ee2f87f29 with GeForce GTX 1080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.89. Perhaps the last time I tested it I got lucky and it didn't happen, will test older versions now.

This does seem to be a genuine issue, definitely not caused by TDR delay (rendering takes less time than the TDR) or the other typical reasons.

I'm able to reproduce this in 2.92.0 Beta, branch: master, commit date: 2021-01-17 10:36, hash: `2ee2f87f29` with GeForce GTX 1080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.89. Perhaps the last time I tested it I got lucky and it didn't happen, will test older versions now. This does seem to be a genuine issue, definitely not caused by TDR delay (rendering takes less time than the TDR) or the other typical reasons.

Robert Guetzkow commented

2021-01-18 14:11:31 +01:00

Changed status from 'Needs User Info' to: 'Confirmed'

Robert Guetzkow commented

2021-01-18 14:11:31 +01:00

I'm confirming this as it appears to be a bug in OptiX denoiser implementation. The issue is very reliably reproducible for me in the referenced build.

@pmoursnv @brecht This is likely something for you to look at.

I'm confirming this as it appears to be a bug in OptiX denoiser implementation. The issue is very reliably reproducible for me in the referenced build. @pmoursnv @brecht This is likely something for you to look at.

Robert Guetzkow commented

2021-01-18 14:13:14 +01:00

Added subscriber: @brecht

Patrick Mours commented

2021-01-18 14:19:07 +01:00

This is a bug introduced with the tile stealing implementation (517ff40b12). Since the OptiX denoiser runs on the GPU, the tile stealing code erroneously steals CPU tiles and moves them to the OptiX device. But in this configuration the OptiX device was only set up for denoising, not rendering, so it crashes. Will look for a fix.

This is a bug introduced with the tile stealing implementation (517ff40b12). Since the OptiX denoiser runs on the GPU, the tile stealing code erroneously steals CPU tiles and moves them to the OptiX device. But in this configuration the OptiX device was only set up for denoising, not rendering, so it crashes. Will look for a fix.

Robert Guetzkow commented

2021-01-18 14:38:00 +01:00

Added subscriber: @Aiduss

blender-admin commented

2021-01-18 15:30:25 +01:00

This issue was referenced by 4fde594fda

This issue was referenced by 4fde594fda13abf98437bc5d0012decc2bd3d5f6

Patrick Mours commented

2021-01-18 15:33:16 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Patrick Mours closed this issue

2021-01-18 15:33:16 +01:00

Patrick Mours self-assigned this 2021-01-18 15:33:16 +01:00

Thomas Dinges added this to the 2.92 milestone 2023-02-08 16:14:53 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

CPU render + OptiX denoiser leaves empty tiles unprocessed (Blender from Master) #84650