Page MenuHome

Blender complains that it cannot find "stdosl.h", despite it existing at "/usr/share/OSL/shaders/stdosl.h"
Closed, ResolvedPublic

Description

System Information
Operating system: Arch Linux
Graphics card: RX 580

Blender Version
Broken: df1e9b662bd6938f74579cea9d30341f3b6dd02b, "Cleanup: Fix build warnings from OSL shader compilation"
Worked: 6a28d14f7236b3e6fbe7409da9ee916878d65feb, "Fix T73740: Auto-key "Only Available" de-selects f-curves"

Short description of error
Blender cannot find "stdosl.h", even though it exists.

Exact steps for others to reproduce the error
On Arch Linux, install openshadinglanguage, and then attempt to compile Blender master.

Other

Related: https://developer.blender.org/D6812

Upon examining the borked commit, I noticed this comment:

FindOSL.cmake does not give us the location of the shader library but generally it can be figured out from the location of the oslc compiler. However if this fails you can set OSL_SHADER_DIR to bypass the auto discovery.

Unfortunately, there is no provided CMake option allowing a custom OSL_SHADER_DIR location to be set, as the current assumptions don't account for where packagers may place system files. In Arch Linux, the compiler is here, "/usr/bin/oslc", while the shader directory is here, "/usr/share/OSL/shaders".

Event Timeline

Kyle (Valmar33) updated the task description. (Show Details)
Kyle (Valmar33) updated the task description. (Show Details)

@Ray molenkamp (LazyDodo), probably we should improve FindOpenShadingLanguage.cmake to find the shaders directory.

I'm not on linux so i can't really test this but can anyone give this a whirl?

diff --git a/build_files/cmake/Modules/FindOpenShadingLanguage.cmake b/build_files/cmake/Modules/FindOpenShadingLanguage.cmake
index 01ed72051f7..72e40e74895 100644
--- a/build_files/cmake/Modules/FindOpenShadingLanguage.cmake
+++ b/build_files/cmake/Modules/FindOpenShadingLanguage.cmake
@@ -85,6 +85,17 @@ IF(OSL_FOUND)
          "\\1" OSL_LIBRARY_VERSION_MINOR ${OSL_LIBRARY_VERSION_MINOR})
 ENDIF(OSL_FOUND)

+FIND_PATH(OSL_SHADER_DIR
+  NAMES
+    stdosl.h
+  HINTS
+    ${OSL_ROOT_DIR}
+    ${ENV{OSLHOME}}
+    /usr/share/osl
+  PATH_SUFFIXES
+    shaders
+)
+
 MARK_AS_ADVANCED(
   OSL_INCLUDE_DIR
 )
`

CMake doesn't like it:

-- Found OSL: /usr/lib/liboslcomp.so;/usr/lib/liboslexec.so;/usr/lib/liboslquery.so  
CMake Error at build_files/cmake/Modules/FindOpenShadingLanguage.cmake:88 (FIND_PATH):
  Syntax error in cmake code at

    /tmp/makepkg/blender-git/src/blender/build_files/cmake/Modules/FindOpenShadingLanguage.cmake:93

  when parsing string

    ${ENV{OSLHOME}}

  Invalid character ('{') in a variable name: 'ENV'
Call Stack (most recent call first):
  build_files/cmake/platform/platform_unix.cmake:71 (find_package)
  build_files/cmake/platform/platform_unix.cmake:248 (find_package_wrapper)
  CMakeLists.txt:801 (include)


-- Configuring incomplete, errors occurred!

toss that one line for now then, just to see if it picks up your headers in /usr/share

This bit me. Gentoo Linux here, with media-libs/osl-1.10.5 providing shaders in system folder /usr/include/OSL/shaders
For what it is worth, my workaround:

  1. Define in build directory CMakeCache.txt the symbol OSL_SHADER_DIR:PATH=/usr/include/OSL/shaders
  2. Modified blender/intern/cycles/kernel/shaders/CMakeLists.txt:
diff --git a/intern/cycles/kernel/shaders/CMakeLists.txt b/intern/cycles/kernel/shaders/CMakeLists.txt
index 1c9445107ad..b1604978cce 100644
--- a/intern/cycles/kernel/shaders/CMakeLists.txt
+++ b/intern/cycles/kernel/shaders/CMakeLists.txt
@@ -131,7 +131,7 @@ foreach(_file ${SRC_OSL})
   string(REPLACE ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR} _OSO_FILE ${_OSO_FILE})
   add_custom_command(
     OUTPUT ${_OSO_FILE}
-    COMMAND ${OSL_COMPILER} -q -O2  -I"${CMAKE_CURRENT_SOURCE_DIR}" -o ${_OSO_FILE} ${_OSL_FILE}
+    COMMAND ${OSL_COMPILER} -q -O2  -I"${CMAKE_CURRENT_SOURCE_DIR}" -I"${OSL_SHADER_DIR}" -o ${_OSO_FILE} ${_OSL_FILE}
     DEPENDS ${_OSL_FILE} ${SRC_OSL_HEADERS} ${OSL_COMPILER})
   list(APPEND SRC_OSO
     ${_OSO_FILE}

TL;DR I added a second include directive so that the OSL compiler looks in the installed OSL shader directory as well as the specific blender build directory.

Haven't tested if this approach works beyond my peculiar little world. Hope it helps point the way to a more general solution.

[edit]
@Kyle (Valmar33)

Unfortunately, there is no provided CMake option...

Try: cmake ../blender -D OSL_SHADER_DIR:PATH=/usr/share/OSL/shaders
Maybe substitute ../blender with the path to your blender source tree. In my case, the build directory is a co-directory of the source directory.

Your patch fails, sadly.

@Ray molenkamp (LazyDodo) It would probably be a good idea to allow packagers to set the OSL_SHADER_DIR variable themselves, as the comment in the OP states.

Yet, I think you may have simply forgotten to offer that as a CMake option, as there's currently zero way to set it.

@Ray molenkamp (LazyDodo) Ah, I see... your patch had /usr/share/osl, when it should be /usr/share/OSL, as most Linux filesystems are distinguish between lower and uppercase.

Actually, no, nevermind, that fails, too...

Hmmmmm......

@Kyle (Valmar33)
Note that my workaround is peculiar to a Gentoo linux box; the steps need a certain amount of remapping to work on other distros. Fortunately, Arch is not that far removed from Gentoo.
In your case, introduce a CMake symbol via

[blender_build_dir]$ cmake -S <path to your blender source tree> -B <path to your blender_build_dir> -D OSL_SHADER_DIR:PATH=<path to your installed OSL shaders, i.e. /usr/share/OSL/shaders>

Patch /intern/cycles/kernel/shaders/CMakeLists.txt as indicated in my first comment
Probably be a good idea to create a working branch first to keep your git repository tidy

$ git branch --no-track osl_fix master
$ git checkout osl_fix

Then patch CMakeLists.txt

This is just a Linux-centric workaround. Unfortunately, various Linux distros put the shader directory in various places. Witness Gentoo: /usr/include/OSL/shaders vs. Arch: /usr/share/OSL/shaders. It's an all-around PITA to deal with this. Probably, to duck this issue, is why blender developers from ages ago made their private copies, the issue that the good, brave @Ray molenkamp (LazyDodo) is trying to address.

Huh ~ it works, if I symlink /usr/share/OSL/shaders to /usr/shaders. That's... something. Not what's needed, anyhow. :|

Yeah, your solution works. Cheers. :)

Great! Don't forget this is a hack - unofficial, unsupported, probably radioactive. If you've followed my suggestion of creating a local git branch osl_fix to contain the patch, and you sit on the osl_fix branch to do your builds, then you've got (just a little) extra work to keep osl_fix aligned with master whenever you perform make update. See Creating Patches from Local Git Branches. The gist of it is: from osl_fix: git checkout master, make update, git checkout osl_fix, git rebase master. When the fix for this lands in master, you can just checkout master and do the update, then abandon osl_fix: git branch -D osl_fix.

I manged to test some of this out on WSL , can you guys test the patch in D6865 ? i included both the arch and gentoo search paths so it should work out of the box now.

Yep - but not now. up to neck in alligators again. Never can tell what comes out of a Brooklyn sewer. Late this evening probably; post results in the wee hours Monday (-5 UTC, less wee +1 UTC).
Thanks thee, for the rapid turnaround!

Wait, what?! The 'again' is troublesome since it seems to indicate that this is a common occurrence....

Different, unrelated alligators. These are pink. Alligators related to build systems are putrid green, have warts, big teeth. Pink ones have been dispatched now. Starting cleanup...

@Ray molenkamp (LazyDodo)
D6865 plays fine here. Completely uneventful. Ran a couple of builds, one Release and one Debug from empty build directories. Deleted CMakeCache.txt to make sure no strange symbols were floating around. Running ctest now; nothing out of the ordinary there either, up to now.
Current to: version: 2.83 (sub 3), branch: osl_rayfix, commit date: 2020-02-16 22:53, hash: f1a13b1ab8d6, type: Release

Thanks! Will test. :)

This comment was removed by Kyle (Valmar33).
Kyle (Valmar33) closed this task as Resolved.Feb 17 2020, 4:34 AM
Kyle (Valmar33) claimed this task.

This completely resolves the issue! Thanks. :)