(Development) Make CUDA dependency optional when compiling Cycles
AbandonedPublic

Authored by Inês Almeida (brita_) on Feb 13 2018, 12:39 AM.

Details

Summary

Currently, when compiling WITH_CYCLES, the CUDA dependency ends up being required, regardless of the WITH_CYCLES_CUDA_BINARIES option.
This patch makes it so that if both WITH_CYCLES_CUDA_BINARIES and WITH_CUDA_DYNLOAD are unchecked, CUDA will not be required.

  • I am not entirely sure that I got the meaning of the options correct. If not, I can add an extra one that encompasses those two.

This patch saves some time for developers who don't have NVidia cards or don't want to install CUDA and all the proprietary drivers.

Diff Detail

Repository
rB Blender

Cycles should compile fine without CUDA installed already, but maybe something broke with the recent changes? Which platform and cmake options are giving a problem? At least on macOS I have no CUDA installed and Cycles builds ok for me.

By default, WITH_CUDA_DYNLOAD should be on, and WITH_CYCLES_CUDA_BINARIES should be off. With that configuration, no CUDA installation should be needed.

Note that setting WITH_CUDA_DYNLOAD=OFF is purely a debugging feature, unless you want to debug CUDA kernels there is no reason to change the default. The option could get a better name and description though.

I guess the confusing might have been, you want to turn all the CUDA stuff off so you set this to OFF, but then suddenly a CUDA install becomes required. We could rename it to e.g. WITH_CUDA_STATIC_LOAD to avoid that problem.

Sergey Sharybin (sergey) requested changes to this revision.Feb 13 2018, 10:33 AM

I do not find naming confusing, disabling DYNLOAD disables dynamic load, but does not affect on whether library itself is used or not. This name also matches other DYNLOAD flags we've got SDL and JACK. So either we rename all of them (which is a bad idea, since it is an intrusive change without really measurable benefit). Or we just clarify what option actually means (which we should do in any case). One shouldn't be changing options without reading their description anyway.

One more thing we can do is to make WITH_{OPENCL,CUDA,NETWORKING} a CMake options. So setting WITH_CUDA will disable everything related on CUDA.z

WITH_CUDA_STATIC_LOAD is easy to confuse with static linking. Static linking is what users are always asking for, and such name will surely confuse people.

The patch itself is wrong, if you don't want to link against libcuda at compile time, don't disable CUDA_DYNLOAD.

This revision now requires changes to proceed.Feb 13 2018, 10:33 AM

The naming of the options was indeed confusing to me and I wasn't sure if 'WITH_CYCLES_CUDA_BINARIES' meant something else than 'WITH_CUDA'.

Indeed I just don't want CUDA, and I had both options off.
If both are off, then on intern/cycles/cmake/external_libs.cmake the option WITH_CUDA_DYNLOAD will be set to ON, and from there fail to compile.
I didn't see the point of turning on DYNLOAD if I just don't want CUDA anyway.

The file intern/cycles/device/device_cuda.cpp still needs to be guarded? I had missing includes iirc. I am on Slackware Linux 64bit, no graphics card, only intel cpu and mesa drivers.

Here are the points:

  • CUDA is NOT required to compile Cycles with CUDA support.
  • CUDA can NOT be disabled in Cycles,
  • You should be able to compile Blender with whatever CUDA flags are set to, if something is not detected or missing, build system will fall back to a configuration which builds Blender witf dynamically loaded CUDA, which only requires extern/cuew (see code in intern/cycles/cmake/external_libs, especifically line message(STATUS "Additionally falling back to dynamic CUDA load")).

My GUESS is that your configuration is failing with undefined references to cuewErrorString, cuewInit, cuewCompilerPath, and cuewCompilerVersion. But this is mainly due to fiasco of order of Cycles disabling WITH_CUDA_DYNLOAD when toolkit is not found and external libraries checking whether or not to compile cuew.

So all we need to do is to move check for missing CUDA toolkit to a higher CMake level, so both extern/ and liblink macros are aware of missing toolkit:

1diff --git a/CMakeLists.txt b/CMakeLists.txt
2index c0a0b4bd27f..1a82c1dd48b 100644
3--- a/CMakeLists.txt
4+++ b/CMakeLists.txt
5@@ -744,6 +744,15 @@ if(WITH_AUDASPACE)
6​ endif()
7​ endif()
8
9+# Auto-enable CUDA dynload if toolkit is not found
10+if(NOT WITH_CUDA_DYNLOAD)
11+ find_package(CUDA)
12+ if (NOT CUDA_FOUND)
13+ message("CUDA toolkit not found, using dynamic runtime loading of libraries instead")
14+ set(WITH_CUDA_DYNLOAD ON)
15+ endif()
16+endif()
17+
18​ #-----------------------------------------------------------------------------
19​ # Check for valid directories
20​ # ... a partial checkout may cause this.

Think this is least intrusive change, which makes things to work as they are expected.

ADDITIONALLY, i wouldn't mind committing this change, it's handy to disable CUDA and OpenCL when you use valgrind (valgrind is confused by magic happening in drivers when initializing CUDA/OpenCL):

1diff --git a/CMakeLists.txt b/CMakeLists.txt
2index c0a0b4bd27f..c50517bfb93 100644
3--- a/CMakeLists.txt
4+++ b/CMakeLists.txt
5@@ -420,6 +420,15 @@ mark_as_advanced(WITH_CYCLES_LOGGING)
6​ mark_as_advanced(WITH_CYCLES_DEBUG)
7​ mark_as_advanced(WITH_CYCLES_NATIVE_ONLY)
8
9+option(WITH_CYCLES_DEVICE_CUDA "Enable Cycles CUDA compute support" ON)
10+option(WITH_CYCLES_DEVICE_OPENC "Enable Cycles OpenCL compute support" ON)
11+option(WITH_CYCLES_DEVICE_MULTI "Enable Cycles multi-device compute support" ON)
12+option(WITH_CYCLES_NETWORK "Enable Cycles compute over network support (EXPERIMENTAL and unfinished)" OFF)
13+mark_as_advanced(WITH_CYCLES_DEVICE_CUDA)
14+mark_as_advanced(WITH_CYCLES_DEVICE_OPENCL)
15+mark_as_advanced(WITH_CYCLES_DEVICE_MULTIDEVICE)
16+mark_as_advanced(WITH_CYCLES_DEVICE_NETWORK)
17+
18​ option(WITH_CUDA_DYNLOAD "Dynamically load CUDA libraries at runtime" ON)
19​ mark_as_advanced(WITH_CUDA_DYNLOAD)
20
21diff --git a/intern/cycles/CMakeLists.txt b/intern/cycles/CMakeLists.txt
22index e7b934ec74f..184a7c73559 100644
23--- a/intern/cycles/CMakeLists.txt
24+++ b/intern/cycles/CMakeLists.txt
25@@ -167,9 +167,11 @@ if(WITH_CYCLES_OPENSUBDIV)
26​ )
27​ endif()
28
29-set(WITH_CYCLES_DEVICE_OPENCL TRUE)
30-set(WITH_CYCLES_DEVICE_CUDA TRUE)
31-set(WITH_CYCLES_DEVICE_MULTI TRUE)
32+if(WITH_CYCLES_STANDALONE)
33+ set(WITH_CYCLES_DEVICE_OPENCL TRUE)
34+ set(WITH_CYCLES_DEVICE_CUDA TRUE)
35+ set(WITH_CYCLES_DEVICE_MULTI TRUE)
36+endif()
37
38​ if(CYCLES_STANDALONE_REPOSITORY)
39​ TEST_UNORDERED_MAP_SUPPORT()

That's all fine with me, except for WITH_CYCLES_DEVICE_MULTI. I would remove that option, it only makes things fail when you try to render with multiple devices, and I see no reason to add the code complexity for properly adjusting the UI to hide multiple devices.

Also a typo: WITH_CYCLES_DEVICE_OPENC should be WITH_CYCLES_DEVICE_OPENCL.

Fixed typo and removed device multi from options. Blender should be compilable now with any CUDA* options configuration.

Let's close this one then.