Page MenuHome

Cycles: Cuda 9.2 and 10 Issues.
Closed, ResolvedPublic

Description

Cycles: Cuda 9.2 and 10 Issues.

Cuda92 has some issues with cycles, that shown most when rendering bmw2.7 gpu

I am sure we all hoped, that cuda 10 would fix this, which it didn't.

I played around with the usual suspects, aka don't inline this and even tried
disabling all inlining all together. didn't help.

then I played with the optimizers (both nvcc and ptax have their own optimizers
which you can independently control) and the issue seems to occur whenever you
call ptxas with O1 or higher (O0 makes a working, kernel) however there's no
documentation what optimizations O1 enables compared to O0 or anyway to enable
specific optimizer stages. so that turned out a dead end as well.

Then I figured if we know were the problem is we might be able to find a work
around. (my idea was to disable the optimizer for that one function, which you can't
for some reason , still it be nice to know where the issue is)

New problem: bmw27 has a lot of parts to it so after taking out various bits I ended
up with these 2 cases.

error.blend: matches the noise pattern you see in bmw27 it's a simple scene but there's
still a couple of mix shaders to it, simpler would be nicer..


error2.blend: no noise, but it renders completely black on the gpu, no fancy shader graphs
it's a straight up glass shader..


So I'm pretty sure it's something in the glass bsdf, but i don't know the code well enough
to look further into this.


Event Timeline

I've been told the next minor CUDA 10 release will have a bugfix for this. It's a code generation bug and depends on the optimization level and function inlining. Hopefully it solves the problem on for all GPU architectures we support, we'll need to test that in detail when it comes out.

I have some test builds to verify if this bug is indeed solved with the upcoming CUDA bugfix.

It would help if users can test if these render the BMW scene correctly. If it's wrong there will be very clear artifacts. There are nvcc and nvrtc builds, both need to be tested.
https://download.blender.org/ftp/brecht/cuda_bug_T56858/
https://download.blender.org/demo/test/BMW27_2.blend.zip

Particularly GTX 6xx and 7xx would be useful, since I don't have a card from that generation.

is the bmw scene linked different from the one in our benchmark suite? or was it just linked for convenience?

Just for convenience, should make no difference which one is tested.

on win7 with sm_30 (gtx670) which was one of the problematic ones

nvrtc RenderTime:04:25.32

nvcc: RenderTime:04:31.64

Also the 2 sample files above render ok on both.

Soo yeah seems solved.

I confirmed it works well on an RTX 5000 and GTX 1080, so it indeed seems to be fully resolved. Performance also seems a few % better than 10.0 for me.

So now we wait until the next CUDA release is officially out, and then upgrade all platforms to it so we don't need to use both 9.1 and 10.0 toolkits anymore.