Cycles OSL - compiling shaders failing due "\" and "n" symbols combination inside include paths - Needs OSL update #83631

New Issue

Nikita Sirgienko · 2020-12-10T14:35:02+01:00

Nikita Sirgienko commented

2020-12-10 14:35:02 +01:00

System Information
Operating system: Windows 10 Enterprise 1909 18363.1198
Graphics card: Intel(R) UHD Graphics 620
OSL Compiler version: 1.10.10

Blender Version
Broken: 46e66d154c

Short description of error
During building OSL shaders a compiler reports errors like this:

C:\Users\nsirgien\Documents\>C:\Users\nsirgien\Documents\Intel\lib\win64_vc15\osl\bin\oslc.exe -q -O2 -I"C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders" -I"C:/Users/nsirgien/Documents/Intel/blender/../lib/win64_vc15/osl/shaders" -o C:/Users/nsirgien/Documents/Intel/winbuild/intern/cycles/kernel/shaders/node_rgb_to_bw.oso C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl
C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl:1: error: error: could not find include file: C:\Users
sirgien\Documents\Intel\lib\win64_vc15\osl/shaders/stdosl.h
FAILED C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl

The problem is, that "" and "n" symbol combinations, which naturally appears in file paths, handled as new line, which cause fail of the file including.

Exact steps for others to reproduce the error
Based on the default startup or an attached .blend file (as simple as possible).

Make a directory with "\n"
Download blender and blender dependencies in this directory
Try to build blender with WITH_CYCLES_OSL=ON

P.S. Is it upstream problem and should be reported to OSL compiler tracker?

**System Information** Operating system: Windows 10 Enterprise 1909 18363.1198 Graphics card: Intel(R) UHD Graphics 620 OSL Compiler version: 1.10.10 **Blender Version** Broken: 46e66d154c9a1d2b25128cbcd3445148df34a581 **Short description of error** During building OSL shaders a compiler reports errors like this: ``` C:\Users\nsirgien\Documents\>C:\Users\nsirgien\Documents\Intel\lib\win64_vc15\osl\bin\oslc.exe -q -O2 -I"C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders" -I"C:/Users/nsirgien/Documents/Intel/blender/../lib/win64_vc15/osl/shaders" -o C:/Users/nsirgien/Documents/Intel/winbuild/intern/cycles/kernel/shaders/node_rgb_to_bw.oso C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl:1: error: error: could not find include file: C:\Users sirgien\Documents\Intel\lib\win64_vc15\osl/shaders/stdosl.h FAILED C:/Users/nsirgien/Documents/Intel/blender/intern/cycles/kernel/shaders/node_rgb_to_bw.osl ``` The problem is, that "\" and "n" symbol combinations, which naturally appears in file paths, handled as new line, which cause fail of the file including. **Exact steps for others to reproduce the error** Based on the default startup or an attached .blend file (as simple as possible). 1) Make a directory with "\n" 2) Download blender and blender dependencies in this directory 3) Try to build blender with WITH_CYCLES_OSL=ON P.S. Is it upstream problem and should be reported to OSL compiler tracker?

Nikita Sirgienko commented

2020-12-10 14:35:02 +01:00

Added subscriber: @Sirgienko

Nikita Sirgienko commented

2020-12-10 16:42:32 +01:00

Added subscriber: @maxim_d33

Nikita Sirgienko commented

2020-12-10 16:42:32 +01:00

CC: @maxim_d33

Evan Wilson commented

2020-12-10 16:57:03 +01:00

Added subscribers: @LazyDodo, @EAW

Evan Wilson commented

2020-12-10 16:57:03 +01:00

Researching this, it looks like this was fixed upstream already, fixed in Blender, then broken again? I'll have to test.

@LazyDodo reported it here: https://github.com/imageworks/OpenShadingLanguage/issues/866 in Feb 2018.
Pull Request here: https://github.com/imageworks/OpenShadingLanguage/pull/1101. It was merged upstream in January.
Merge into Blender Feb 8 b9c9b00171
A related stdosl.h issue fixed in a commit Feb 11 D6812: Fix: OSL warnings during build df1e9b662b
8d60e6fa38 Fixed on Feb 17 see #73830 (Blender complains that it cannot find "stdosl.h", despite it existing at "/usr/share/OSL/shaders/stdosl.h") for further details.

@LazyDodo Is a pair of parenthesis missing in a CMake file? Did fixing Linux break Windows perhaps?

Researching this, it looks like this was fixed upstream already, fixed in Blender, then broken again? I'll have to test. 1. @LazyDodo reported it here: https://github.com/imageworks/OpenShadingLanguage/issues/866 in Feb 2018. 2. Pull Request here: https://github.com/imageworks/OpenShadingLanguage/pull/1101. It was merged upstream in January. 3. Merge into Blender Feb 8 b9c9b00171 4. A related `stdosl.h` issue fixed in a commit Feb 11 [D6812: Fix: OSL warnings during build](https://archive.blender.org/developer/D6812) df1e9b662b 5. 8d60e6fa38 Fixed on Feb 17 see #73830 (Blender complains that it cannot find "stdosl.h", despite it existing at "/usr/share/OSL/shaders/stdosl.h") for further details. @LazyDodo Is a pair of parenthesis missing in a CMake file? Did fixing Linux break Windows perhaps?

Ray molenkamp commented

2020-12-10 17:11:39 +01:00

No this is an issue in oslc nothing we do on the blender side can fix/unfix this behavior.

i'd have to test with the latest osl to see if the issue has been fixed already and if not report it upstream.

No this is an issue in oslc nothing we do on the blender side can fix/unfix this behavior. i'd have to test with the latest osl to see if the issue has been fixed already and if not report it upstream.

Ray molenkamp self-assigned this 2020-12-10 17:13:02 +01:00

Ray molenkamp commented

2020-12-10 18:10:04 +01:00

Added subscriber: @brecht

Ray molenkamp commented

2020-12-10 18:10:04 +01:00

The bug was fixed in the clang code path in #1101, but the boost_wave codepath above it still has the issue which sadly is the current path our libraries take.

couple of options here

rebuild the libs and force it to take the fixed codepath by disabling whatever enabled the WITH_BOOST_WAVE define
patch OSL and fix the bug (I have no CLA on record so I cannot upstream, also see point 3)
the broken codepath seems to be gone completely in v1.11.10.0 so we could probably just bump the osl version, but I'm unsure if there's any re-integration work associated with that and if @brecht has time for that

@brecht thoughts?

The bug was fixed in the clang code path in [#1101](https:*github.com/imageworks/OpenShadingLanguage/pull/1101), but the boost_wave codepath above it [still has the issue](https:*github.com/imageworks/OpenShadingLanguage/blob/RB-1.10/src/liboslcomp/oslcomp.cpp#L195) which sadly is the current path our libraries take. couple of options here 1) rebuild the libs and force it to take the fixed codepath by disabling whatever enabled the WITH_BOOST_WAVE define 2) patch OSL and fix the bug (I have no CLA on record so I cannot upstream, also see point 3) 3) the broken codepath seems to be gone completely in v1.11.10.0 so we could probably just bump the osl version, but I'm unsure if there's any re-integration work associated with that and if @brecht has time for that @brecht thoughts?

Brecht Van Lommel commented

2020-12-10 21:20:27 +01:00

I wouldn't mind upgrading to 1.11. I already did adjustments for API changes a bit before the release, hopefully it will just build and work as is. But if note I can update the code.

Brecht Van Lommel commented

2020-12-10 21:20:44 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Ray molenkamp commented

2020-12-11 16:25:51 +01:00

I managed to build 1.11, passes all tests without any modifications in blender (and fixes this issue). The deps builder for osl needed some more substantial changes and cleanup, i'll try to get that polished up in the next couple of days.

Brecht Van Lommel commented

2020-12-11 16:28:55 +01:00

Thanks!

Bastien Montagne changed title from ~~Cycles OSL - compiling shaders failing due "\" and "n" symbols combination inside include paths~~ to Cycles OSL - compiling shaders failing due "\" and "n" symbols combination inside include paths - Needs OSL update

2021-02-03 16:25:36 +01:00

blender-admin commented

2021-02-24 15:13:37 +01:00

This issue was referenced by blender/cycles@5fe5537a34

This issue was referenced by blender/cycles@5fe5537a344e5e20e9a370e5a3b51ae3a4232aab

blender-admin commented

2021-02-24 15:13:37 +01:00

This issue was referenced by 4fb0c83c1c

This issue was referenced by 4fb0c83c1c8529b61d38d8a409caf58d05a53bd9

Phil Hargett commented

2021-02-25 16:06:10 +01:00

Added subscriber: @hargettp

Phil Hargett commented

2021-02-25 16:06:10 +01:00

FYI, the commit in D10212 broke builds on Mac OS X Big Sur (Apple M1 Mac / darwin arm64).

The error after running make is:

-- Could NOT find Clang (missing: _CLANG_LIBRARIES) 
CMake Error at build_files/cmake/platform/platform_apple.cmake:339 (message):
  Clang not found.
Call Stack (most recent call first):
  CMakeLists.txt:916 (include)

The contents of CMakeError.log are:

Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lSystem
clang: error: linker command failed with exit code 1 (use -v to see invocation)


Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)

The commit just before this one still builds just fine.

FYI, the commit in [D10212](https://archive.blender.org/developer/D10212) broke builds on Mac OS X Big Sur (Apple M1 Mac / darwin arm64). The error after running `make` is: ``` -- Could NOT find Clang (missing: _CLANG_LIBRARIES) CMake Error at build_files/cmake/platform/platform_apple.cmake:339 (message): Clang not found. Call Stack (most recent call first): CMakeLists.txt:916 (include) ``` The contents of CMakeError.log are: ``` Compiling the C compiler identification source file "CMakeCCompilerId.c" failed. Compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc Build flags: Id flags: The output was: 1 ld: library not found for -lSystem clang: error: linker command failed with exit code 1 (use -v to see invocation) Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed. Compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ Build flags: Id flags: The output was: 1 ld: library not found for -lc++ clang: error: linker command failed with exit code 1 (use -v to see invocation) ``` The commit just before this one still builds just fine.

Ray molenkamp commented

2021-02-25 16:36:15 +01:00

Update your svn libs, @sebbas was on that yesterday, i assume that was completed?

Ray molenkamp commented

2021-02-25 16:37:45 +01:00

Added subscriber: @sebbas

Sebastián Barschkis commented

2021-02-25 16:47:46 +01:00

I see you're on Apple M1. @hargettp the libs still have to be updated for that, so far the changes only considered x86.

As soon as I have access to an M1 machine again, I'll make the lib update.

@LazyDodo Yeah, I think it's time to make Apple arm a hard requirement for lib updates.

I see you're on Apple M1. @hargettp the libs still have to be updated for that, so far the changes only considered x86. As soon as I have access to an M1 machine again, I'll make the lib update. @LazyDodo Yeah, I think it's time to make Apple arm a hard requirement for lib updates.

Phil Hargett commented

2021-02-25 17:44:35 +01:00

Ah this makes sense @sebbas ; I ran make update before doing a fresh build, so missing ARM libraries would certainly do it.

Ah this makes sense @sebbas ; I ran `make update` before doing a fresh build, so missing ARM libraries would certainly do it.

Sebastián Barschkis commented

2021-03-03 15:07:40 +01:00

@hargettp ARM libraries have been updated, so after make update builds should work again!

@hargettp ARM libraries have been updated, so after `make update` builds should work again!

Phil Hargett commented

2021-03-03 16:38:09 +01:00

Worked like a charm and running again on Apple M1--thx @sebbas !

Brecht Van Lommel commented

2021-03-03 20:51:45 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Brecht Van Lommel closed this issue

2021-03-03 20:51:45 +01:00

Brecht Van Lommel commented

2021-03-03 20:51:45 +01:00

Great, let's consider this resolved then.

Thomas Dinges added this to the 2.93 LTS milestone 2023-02-07 18:46:38 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Cycles OSL - compiling shaders failing due "\" and "n" symbols combination inside include paths - Needs OSL update #83631