Page MenuHome

GPU render crash
Closed, ArchivedPublic

Description

System Information
Operating system: Windows 10 1803
Graphics card: GTX970

Blender Version
Broken:
blender-2.80-4c31bed6b46-win64

Worked (ish):
2.79.6 - at least it render just stops and I get CUDA error message.

Short description of error

Blender crashes when I render this scene when GPU Compute enabled with CPU and GPU enabled in CUDA Cycles Compute Devices :

. No crash with CPU Compute.
It maybe related to VRAM limitation, since shrinking texture size helps. Still, there's about 400 mb of free VRAM when Blender crashes without any error messages.

Exact steps for others to reproduce the error

Open attached scene and press F12 - on my config it crashes when about 20% rendered.

Details

Type
Bug

Event Timeline

Sebastian Parborg (zeddb) triaged this task as Needs Information from User priority.Jan 23 2019, 11:36 AM

Is this still an issue with the latest 2.8 beta build?

Yes. Another thing that I discovered is that it won't crash if Denoising is disabled.
Tested on dc3b5024be1a

Does 2.80 still actually crash, or only show the CUDA error message?

It's quite possible this is just running out of memory. 400MB being reported as free is not really reliable, there is temporary working memory needed that may not be counted.

Yes, it's still crashing: Blender window is just disappears with no messages, no nothing. Nvidia Driver 417.71 / Windows 1809 17763.253

UPD: looks like now it crashes only when Blender's window isn't in focus while rendering. If I won't switch to another window, it won't crash.

Does it crash if you try to render from command line without gui?

There are some example how to do that on this page (using the -b flag);
https://docs.blender.org/manual/en/latest/render/workflows/command_line.html

Tried on 8c8979067490 build config files were deleted beforehand, just to make sure its not an addon or something. 2 out of 5 times with window out focus it crashed. Same statistics for the render from cmd: 2 out of 5 crashes.
Logs of two crashes: https://pastebin.com/2YhiZ4vc After that I made a few more runs from command, just to be sure, and got another error, now with the direct CUDA mention https://pastebin.com/qbvsgYtY Another one after that, even without closing cmd window was successful, attached.


I hope it's somehow helpful.

Can you run this CUDA memtest program? https://simtk.org/projects/memtest (MemtestG80)

You can also try this and see if you manage to trigger any problems: http://geeks3d.com/furmark/kombustor/

Yup, here's the result: https://pastebin.com/qwiXnJ2a
Although your link requires registration with manual approval by a staff, but from a view of forum it doesn't look too much alive. So I downloaded MemtestG80 compiled by someone, so it maybe a bit out of date, mind you. If confirmation mail from SimTK eventually arrive, I'll redo the test. Also it wasn't able to run it on more that 3072 mb, it kept showing and error: Error: unable to allocate 4094 MiB of GPU memory to test, bailing!

About the Kombustor - it didn't showed any error messages or artifacts and GPU didn't explode.

The result does say that there are memory errors.
You can try http://www.programming4beginners.com/gpumemtest too and see if that reports any issue too.

It says All tests OK https://pastebin.com/dFyCZyEp

But still, it saying Allocated 3312 MiB total, in 4 chunks which is not 4096 MiB of total memory of that GPU.
Of course last chunk of 512 mb is weird with GTX 970, maybe that's the reason?

I've encountered something similar with Substance Painter recently, with Sparse Virtual Textures in particular. GTX 970 and Quado M2000 is the only GPUs that can work with it due to memory issues on Windows, as I can understand: https://support.allegorithmic.com/documentation/spdoc/sparse-virtual-textures-172823866.html

I tried to recreate this scene as close as possible from scratch (maybe issue was in the file itself since it's migrated trough a lot of version), just copy-pasting objects and values and there's actually a difference: Blender doesn't crash as a whole now, only render process stops.

CUDA error: Illegal address in cuCtxSynchronize(), line 1527{F6465676}

Because we can't reproduce this on our end, I'm guessing that it might be indeed the weird memory config of your card.

As you pointed out, the guys at substance painter also ran into problems with your specific card. So I guess that the cuda driver for it is buggy.

Bohdan Lvov (ostapblender) claimed this task.

Well then, let's close it if there's nothing we can do at the moment.