TDR Crash #83000

New Issue

Joachim Przibyllla · 2020-11-25T14:51:54+01:00

2020-11-25 14:51:54 +01:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce 940M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 376.54

Blender Version
Broken: version: 2.90.1, branch: master, commit date: 2020-09-23 06:43, hash: 3e85bb34d0
Worked: Unkown

Short description of error
I am repeatably getting a crash in Blender when going into Rendered Preview Mode. I searched for the Error and in the Event Viewer, I get a Log

A TDR has been detected.
The application must close.


Error code: 7
 (pid=44184 tid=42568 blender.exe 64bit)

It seems like Blender is not propertly putting load on the graphic card or something along those lines.

Exact steps for others to reproduce the error

Fire & Gas Simulation: Going into the Rendered View in the Viewport. It renders the pixelated Preview fine, but when it tries to render at a it starts to hang and crashes after a few seconds. The .blend file + the sim would be too big to upload here.

Also I dont know if this is a system problem or a problem with blender.
If you ask, I can upload them, but It doesnt seem like the scene is the problem

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: GeForce 940M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 376.54 **Blender Version** Broken: version: 2.90.1, branch: master, commit date: 2020-09-23 06:43, hash: `3e85bb34d0` Worked: Unkown **Short description of error** I am repeatably getting a crash in Blender when going into Rendered Preview Mode. I searched for the Error and in the Event Viewer, I get a Log ``` A TDR has been detected. The application must close. Error code: 7 (pid=44184 tid=42568 blender.exe 64bit) ``` It seems like Blender is not propertly putting load on the graphic card or something along those lines. **Exact steps for others to reproduce the error** Fire & Gas Simulation: Going into the Rendered View in the Viewport. It renders the pixelated Preview fine, but when it tries to render at a it starts to hang and crashes after a few seconds. The .blend file + the sim would be too big to upload here. Also I dont know if this is a system problem or a problem with blender. If you ask, I can upload them, but It doesnt seem like the scene is the problem

Joachim Przibyllla commented

2020-11-25 14:51:54 +01:00

Added subscriber: @Blender_Render

Joachim Przibyllla commented

2020-11-25 14:52:41 +01:00

I am also available for further questions.

Robert Guetzkow commented

2020-11-25 15:10:52 +01:00

Added subscriber: @rjg

Robert Guetzkow commented

2020-11-25 15:10:52 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Robert Guetzkow commented

2020-11-25 15:10:52 +01:00

What is your current TDR delay and have you previously changed this value? See instructions on [this website ]] on how to view and edit the configured value. Please also update your graphics driver by downloading the most recent version [ https:*www.nvidia.com/Download/driverResults.aspx/166629/en-us | Nvidia's website . The current version you're using is from before 2017.

What is your current TDR delay and have you previously changed this value? See instructions on [this website ]] on how to view and edit the configured value. Please also update your graphics driver by downloading the most recent version [[ https:*www.nvidia.com/Download/driverResults.aspx/166629/en-us | Nvidia's website ](https:*docs.substance3d.com/spdoc/gpu-drivers-crash-with-long-computations-128745489.html). The current version you're using is from before 2017.

Joachim Przibyllla commented

2020-11-25 15:27:56 +01:00

The TDR registry keys are not set, so the default 2s. I didn't change the values before. I'm gonna try just to update the drivers

Joachim Przibyllla commented

2020-11-25 16:27:48 +01:00

I updated to the newest driver now (457.30), but Blender still crashes... Maybe manually set the TDR Timeout higher?

Robert Guetzkow commented

2020-11-25 16:35:58 +01:00

Please open Blender's installation directory and double click on the blender_debug_gpu.cmd. This will start Blender in debug mode and create log files. Try to make Blender crash again. Once it crashes the Windows Explorer should open and show you up to two files, a debug log and the system information. Add them to your bug report by clicking on the upload button as shown in the screenshot below or via drag and drop. Please also upload the crash log located in C:\Users\[your username]\AppData\Local\Temp\[project name].crash.txt (or simply type %TEMP% into the path bar of the Windows Explorer).

Once you've got the logs files, try to increase the TDR.

Please open Blender's installation directory and double click on the `blender_debug_gpu.cmd`. This will start Blender in debug mode and create log files. Try to make Blender crash again. Once it crashes the Windows Explorer should open and show you up to two files, a debug log and the system information. Add them to your bug report by clicking on the upload button as shown in the screenshot below or via drag and drop. Please also upload the crash log located in `C:\Users\[your username]\AppData\Local\Temp\[project name].crash.txt` (or simply type `%TEMP%` into the path bar of the Windows Explorer). ![2019_12_04_upload_icon_developer_blender_org.png](https://archive.blender.org/developer/F8190038/2019_12_04_upload_icon_developer_blender_org.png) Once you've got the logs files, try to increase the TDR.

Joachim Przibyllla commented

2020-11-25 16:53:15 +01:00

blender_debug_output.txt

blender_system_info.txt

I don't see a Fire.crash.txt in %TEMP%

[blender_debug_output.txt](https://archive.blender.org/developer/F9373274/blender_debug_output.txt) [blender_system_info.txt](https://archive.blender.org/developer/F9373272/blender_system_info.txt) I don't see a Fire.crash.txt in %TEMP%

Joachim Przibyllla commented

2020-11-26 17:27:15 +01:00

Also I just noticed in my registry by default there is no TdrLevel Reg Key, is that normal?

It seems like changing the Timeout helped, It now is able to render in the viewport, it is a bit laggy but works without crashes. It's using the gpu only for a short time every sample,

When rendering not from the Viewport, so normal rendering, I get the Error

CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769)

after 1-2 Minutes, before there was GPU usage that seemed like single cycles sample spikes, after that if I try to render I instantly get

Failed to create CUDA context (Launch failed)

and I can't even render in the Viewport after
rendering normal, same Error.

From the Blender Console (first crash):

CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769)

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

Launch failed in cuMemcpyDtoH( (char *)mem.host_pointer + offset, (CUdeviceptr)mem.device_pointer + offset, size) (device_cuda_impl.cpp:1028)

Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964)
Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964)
Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964)
Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964)
... [ ca 30x]
Error: CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769)

After the first crash:

Failed to create CUDA context (Launch failed)

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

Invalid value in cuCtxPushCurrent(device->cuContext) (device_cuda_impl.cpp:2468)
Invalid context in cuArray3DCreate(&array_3d, &desc) (device_cuda_impl.cpp:1196)
Invalid context in cuCtxPopCurrent(NULL) (device_cuda_impl.cpp:2473)
Invalid value in cuCtxPushCurrent(device->cuContext) (device_cuda_impl.cpp:2468)
Invalid context in cuArray3DCreate(&array_3d, &desc) (device_cuda_impl.cpp:1196)
Invalid context in cuCtxPopCurrent(NULL) (device_cuda_impl.cpp:2473)
Invalid value in cuCtxDestroy(cuContext) (device_cuda_impl.cpp:252)
Error: Failed to create CUDA context (Launch failed)

Also I just noticed in my registry by default there is no TdrLevel Reg Key, is that normal? It seems like changing the Timeout helped, It now is able to render in the viewport, it is a bit laggy but works without crashes. It's using the gpu only for a short time every sample, When rendering not from the Viewport, so normal rendering, I get the Error ``` CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769) ``` after 1-2 Minutes, before there was GPU usage that seemed like single cycles sample spikes, after that if I try to render I instantly get ``` Failed to create CUDA context (Launch failed) ``` and I can't even render in the Viewport after rendering normal, same Error. From the Blender Console (first crash): ``` CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769) Refer to the Cycles GPU rendering documentation for possible solutions: https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html Launch failed in cuMemcpyDtoH( (char *)mem.host_pointer + offset, (CUdeviceptr)mem.device_pointer + offset, size) (device_cuda_impl.cpp:1028) Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964) Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964) Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964) Launch failed in cuMemFree(mem.device_pointer) (device_cuda_impl.cpp:964) ... [ ca 30x] Error: CUDA_ERROR_LAUNCH_FAILED in cuStreamSynchronize(cuda_stream[thread_index]) (device_optix.cpp:769) ``` After the first crash: ``` Failed to create CUDA context (Launch failed) Refer to the Cycles GPU rendering documentation for possible solutions: https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html Invalid value in cuCtxPushCurrent(device->cuContext) (device_cuda_impl.cpp:2468) Invalid context in cuArray3DCreate(&array_3d, &desc) (device_cuda_impl.cpp:1196) Invalid context in cuCtxPopCurrent(NULL) (device_cuda_impl.cpp:2473) Invalid value in cuCtxPushCurrent(device->cuContext) (device_cuda_impl.cpp:2468) Invalid context in cuArray3DCreate(&array_3d, &desc) (device_cuda_impl.cpp:1196) Invalid context in cuCtxPopCurrent(NULL) (device_cuda_impl.cpp:2473) Invalid value in cuCtxDestroy(cuContext) (device_cuda_impl.cpp:252) Error: Failed to create CUDA context (Launch failed) ```

Joachim Przibyllla commented

2020-11-26 17:39:31 +01:00

Event Viewer Log at the first crash:

Quelle:        Windows Error Reporting
Datum:         26.11.2020 17:12:10
Ereignis-ID:   1001
Aufgabenkategorie:Keine
Ebene:         Informationen
Schlüsselwörter:Klassisch
Benutzer:      Nicht zutreffend
Computer:      LAPTOP-U62S5VID
Beschreibung:
Fehlerbucket , Typ 0
Ereignisname: LiveKernelEvent
Antwort: Nicht verfügbar
CAB-Datei-ID: 0

Problemsignatur:
[P1](https://archive.blender.org/developer/P1.txt): 141
[P2](https://archive.blender.org/developer/P2.txt): ffffcf0926055010
[P3](https://archive.blender.org/developer/P3.txt): fffff807870d2298
[P4](https://archive.blender.org/developer/P4.txt): 0
[P5](https://archive.blender.org/developer/P5.txt): 40a4
[P6](https://archive.blender.org/developer/P6.txt): 10_0_19041
[P7](https://archive.blender.org/developer/P7.txt): 0_0
P8: 768_1
[P9](https://archive.blender.org/developer/P9.txt): 
[P10](https://archive.blender.org/developer/P10.txt): 

Angefügte Dateien:
\\?\C:\WINDOWS\LiveKernelReports\WATCHDOG\WATCHDOG-20201126-1711.dmp
\\?\C:\WINDOWS\TEMP\WER-2600453-0.sysdata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9A88.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9B73.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BB0.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BE0.tmp.txt

Diese Dateien befinden sich möglicherweise hier:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\Kernel_141_a615cfd2edc08fe74b7ed2e8e096312148813e9b_00000000_cab_94845f1a-1ed0-4ab1-909b-391fd8993be6

Analysesymbol: 
Es wird erneut nach einer Lösung gesucht: 0
Berichts-ID: 94845f1a-1ed0-4ab1-909b-391fd8993be6
Berichtstatus: 4
Bucket mit Hash: 
CAB-Datei-Guid: 0

After the second all following like this one:

Protokollname: Application
Quelle:        Windows Error Reporting
Datum:         26.11.2020 17:22:09
Ereignis-ID:   1001
Aufgabenkategorie:Keine
Ebene:         Informationen
Schlüsselwörter:Klassisch
Benutzer:      Nicht zutreffend
Computer:      LAPTOP-U62S5VID
Beschreibung:
Fehlerbucket LKD_0x141_Tdr:6_IMAGE_nvlddmkm.sys_Maxwell_3DLowLatency, Typ 0
Ereignisname: LiveKernelEvent
Antwort: Nicht verfügbar
CAB-Datei-ID: 02baea8b-0394-4e97-a659-17e1477ab75d

Problemsignatur:
[P1](https://archive.blender.org/developer/P1.txt): 141
[P2](https://archive.blender.org/developer/P2.txt): ffffcf0926055010
[P3](https://archive.blender.org/developer/P3.txt): fffff807870d2298
[P4](https://archive.blender.org/developer/P4.txt): 0
[P5](https://archive.blender.org/developer/P5.txt): 40a4
[P6](https://archive.blender.org/developer/P6.txt): 10_0_19041
[P7](https://archive.blender.org/developer/P7.txt): 0_0
P8: 768_1
[P9](https://archive.blender.org/developer/P9.txt): 
[P10](https://archive.blender.org/developer/P10.txt): 

Angefügte Dateien:
\\?\C:\WINDOWS\LiveKernelReports\WATCHDOG\WATCHDOG-20201126-1711.dmp
\\?\C:\WINDOWS\TEMP\WER-2600453-0.sysdata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9A88.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9B73.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BB0.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BE0.tmp.txt
\\?\C:\Windows\Temp\WER58A5.tmp.WERDataCollectionStatus.txt

Diese Dateien befinden sich möglicherweise hier:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\Kernel_141_a615cfd2edc08fe74b7ed2e8e096312148813e9b_00000000_cab_94845f1a-1ed0-4ab1-909b-391fd8993be6

Analysesymbol: 
Es wird erneut nach einer Lösung gesucht: 0
Berichts-ID: 94845f1a-1ed0-4ab1-909b-391fd8993be6
Berichtstatus: 268435456
Bucket mit Hash: 
CAB-Datei-Guid: 0

I know thats a lot of text, I just hope one of these could help so I didn't want to not send these, I dont know if this helps

Event Viewer Log at the first crash: ```Protokollname: Application Quelle: Windows Error Reporting Datum: 26.11.2020 17:12:10 Ereignis-ID: 1001 Aufgabenkategorie:Keine Ebene: Informationen Schlüsselwörter:Klassisch Benutzer: Nicht zutreffend Computer: LAPTOP-U62S5VID Beschreibung: Fehlerbucket , Typ 0 Ereignisname: LiveKernelEvent Antwort: Nicht verfügbar CAB-Datei-ID: 0 Problemsignatur: [P1](https://archive.blender.org/developer/P1.txt): 141 [P2](https://archive.blender.org/developer/P2.txt): ffffcf0926055010 [P3](https://archive.blender.org/developer/P3.txt): fffff807870d2298 [P4](https://archive.blender.org/developer/P4.txt): 0 [P5](https://archive.blender.org/developer/P5.txt): 40a4 [P6](https://archive.blender.org/developer/P6.txt): 10_0_19041 [P7](https://archive.blender.org/developer/P7.txt): 0_0 P8: 768_1 [P9](https://archive.blender.org/developer/P9.txt): [P10](https://archive.blender.org/developer/P10.txt): Angefügte Dateien: \\?\C:\WINDOWS\LiveKernelReports\WATCHDOG\WATCHDOG-20201126-1711.dmp \\?\C:\WINDOWS\TEMP\WER-2600453-0.sysdata.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9A88.tmp.WERInternalMetadata.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9B73.tmp.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BB0.tmp.csv \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BE0.tmp.txt Diese Dateien befinden sich möglicherweise hier: \\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\Kernel_141_a615cfd2edc08fe74b7ed2e8e096312148813e9b_00000000_cab_94845f1a-1ed0-4ab1-909b-391fd8993be6 Analysesymbol: Es wird erneut nach einer Lösung gesucht: 0 Berichts-ID: 94845f1a-1ed0-4ab1-909b-391fd8993be6 Berichtstatus: 4 Bucket mit Hash: CAB-Datei-Guid: 0 ``` After the second all following like this one: ``` Protokollname: Application Quelle: Windows Error Reporting Datum: 26.11.2020 17:22:09 Ereignis-ID: 1001 Aufgabenkategorie:Keine Ebene: Informationen Schlüsselwörter:Klassisch Benutzer: Nicht zutreffend Computer: LAPTOP-U62S5VID Beschreibung: Fehlerbucket LKD_0x141_Tdr:6_IMAGE_nvlddmkm.sys_Maxwell_3DLowLatency, Typ 0 Ereignisname: LiveKernelEvent Antwort: Nicht verfügbar CAB-Datei-ID: 02baea8b-0394-4e97-a659-17e1477ab75d Problemsignatur: [P1](https://archive.blender.org/developer/P1.txt): 141 [P2](https://archive.blender.org/developer/P2.txt): ffffcf0926055010 [P3](https://archive.blender.org/developer/P3.txt): fffff807870d2298 [P4](https://archive.blender.org/developer/P4.txt): 0 [P5](https://archive.blender.org/developer/P5.txt): 40a4 [P6](https://archive.blender.org/developer/P6.txt): 10_0_19041 [P7](https://archive.blender.org/developer/P7.txt): 0_0 P8: 768_1 [P9](https://archive.blender.org/developer/P9.txt): [P10](https://archive.blender.org/developer/P10.txt): Angefügte Dateien: \\?\C:\WINDOWS\LiveKernelReports\WATCHDOG\WATCHDOG-20201126-1711.dmp \\?\C:\WINDOWS\TEMP\WER-2600453-0.sysdata.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9A88.tmp.WERInternalMetadata.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9B73.tmp.xml \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BB0.tmp.csv \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER9BE0.tmp.txt \\?\C:\Windows\Temp\WER58A5.tmp.WERDataCollectionStatus.txt Diese Dateien befinden sich möglicherweise hier: \\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\Kernel_141_a615cfd2edc08fe74b7ed2e8e096312148813e9b_00000000_cab_94845f1a-1ed0-4ab1-909b-391fd8993be6 Analysesymbol: Es wird erneut nach einer Lösung gesucht: 0 Berichts-ID: 94845f1a-1ed0-4ab1-909b-391fd8993be6 Berichtstatus: 268435456 Bucket mit Hash: CAB-Datei-Guid: 0 ``` I know thats a lot of text, I just hope one of these could help so I didn't want to not send these, I dont know if this helps

Robert Guetzkow commented

2020-12-04 13:46:56 +01:00

Changed status from 'Needs User Info' to: 'Needs Triage'

Riccardo Giovanetti commented

2022-02-18 21:03:06 +01:00

Added subscriber: @Harvester

Riccardo Giovanetti commented

2022-02-18 21:03:06 +01:00

Sometimes, it happens that when new NVIDIA drivers are installed the registry key where the TDR variable is set (if set) it is being reset to default values, and that variable is deleted.
You have to follow the instructions given by Robert about to the TDR setup in the Windows registry, as described at the given link.
This doesn't seam to be a Blender software bug.

Sometimes, it happens that when new NVIDIA drivers are installed the registry key where the TDR variable is set (if set) it is being reset to default values, and that variable is deleted. You have to follow the instructions given by Robert about to the TDR setup in the Windows registry, as described at the given link. This doesn't seam to be a Blender software bug.

Omar Emara commented

2022-02-24 14:39:08 +01:00

Added subscriber: @OmarEmaraDev

Omar Emara commented

2022-02-24 14:39:08 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Omar Emara commented

2022-02-24 14:39:08 +01:00

Does reducing the rendering tile size make things work?

Riccardo Giovanetti commented

2022-02-24 15:14:16 +01:00

Reducing the tile size might have an impact on the memory used at render time, but I doubt it might "fix" the TDR warning/issue. Have you tried? Did it make any difference? Possibly, a developer might provide you with a more detailed answer, if needed. I still recommend to follow the given instructions and make some tests.

Omar Emara commented

2022-02-24 15:28:22 +01:00

@Harvester This is one of the recommendations available in the documentation to alleviate the issue, and it does indeed make sense as scheduling smaller tiles will reduce the GPU side computations per tile. We just need to confirm if this is indeed the cause of the CUDA synchronization errors.

Riccardo Giovanetti commented

2022-02-25 17:35:52 +01:00

I cannot tell you if this is the cause of the CUDA synchronization errors, because it's a quite dated solution to CUDA TDR error, and I usually set and forget it. This is where I found a solution the first time I got this problem: http://artificialflight.org/blog/2013/cycles-crash-cuda-tdr-error/

I wish to be more useful, sorry.

I cannot tell you if this is the cause of the CUDA synchronization errors, because it's a quite dated solution to CUDA TDR error, and I usually set and forget it. This is where I found a solution the first time I got this problem: http://artificialflight.org/blog/2013/cycles-crash-cuda-tdr-error/ I wish to be more useful, sorry.

Robert Guetzkow commented

2022-05-02 14:57:08 +02:00

Changed status from 'Needs User Info' to: 'Archived'

Robert Guetzkow closed this issue

2022-05-02 14:57:08 +02:00

Robert Guetzkow commented

2022-05-02 14:57:08 +02:00

Based on the provided information this does not appear to be a bug in Blender, but rather a system configuration issue related to the TDR value and the GPU state being reset by the OS.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

TDR Crash #83000