Blender Shader Language¶
Introduction¶
Historically, Blender's shaders were written in GLSL targeting only OpenGL.
With the addition of the Vulkan and Metal we needed to find a way to feed the same codebase to different backends. We opted to keep the GLSL syntax and make it compatible with the target API native language. This is for multiple reasons: * Lack of time to rewrite the codebase in another language. * Keeping compatibility with OpenGL 4.3 (no Spir-V extension). * Needed access to some MSL-only features.
As time went on, the complexity of our shader codebase increased. The lack of modern feature of GLSL is starting to make authoring of shaders and managing the shader codebase harder.
To bridge this gap, we started investing in tools. But quickly, our shaders tools became were very opaque and hard to use. Only a few people in the team had knowledge on how to use them and even less how they work.
For this reason, we are trying to convert these tools into our own shading language.
Targets¶
The goals of this shading language: * C++ compatible for host side code reuse. * Everything defined as code (no concept of implicit code injection). * Every code path statically checked.
What the language should be: * be easy to translate to the target shader languages (GLSL, MSL). * be translated at compile time. This mean translation should be done per file.
What the language should not be: * be directly compiled to Spir-V or other machine code. We do not want to write a compiler. * be 100% compliant with the C++ standard. It is not something that we have the capacity to do.
This language is based on the GLSL specification for historic reason. Features are expressed as C++ construct with some limitation to stick to the above rules.
Implementation¶
The shader files are translated at compile time into an intermediate language which doesn't have the C++ constructs.
Some metadata are extracted from each original file and a GPUSource is created for each of the file.
The intermediate language is even closer to GLSL and is also used to support user GLSL shader through from the python API.
At runtime this intermediate language is then compiled by the different backend using different compatibility macros.
flowchart LR
Source["Blender Shader Source"]
SourcePy["Python Shader Source"]
SourceMod["Intermediate Language"]
GPUSource
CompiledOGL["OpenGL GLSL Compiler"]
CompiledMTL["MSL Compiler"]
CompiledVLK["Vulkan GLSL Compiler"]
Source --> |Blender Compilation|SourceMod
SourcePy --> |pyGPU API|SourceMod
SourceMod --> |Runtime|GPUSource
GPUSource --> |Runtime Shader Compilation|CompiledOGL
GPUSource --> |Runtime Shader Compilation|CompiledMTL
GPUSource --> |Runtime Shader Compilation|CompiledVLKFeatures¶
In this section we explain the language's features and their differences with C++.
Include System¶
The #include directives act similarly to slang's __include.
It is a one time include of a file, with no propagation of preprocessor state.
This behavior is enforced when compiling shader files to C++ by always adding #pragma once to each header file and by putting all include statements before any other code.
Only the syntax #include "my_file.hh" is supported (no support for #include <my_file.hh> as it is against our codestyle).
It is forbidden to include C++ libraries in shader files.
For legacy reasons, most files currently includes shader info files *info.hh. These are omitted from the final shader source and are only there for compiling the shader using C++.
Types¶
Types use the same conventions as BLI math API.
Vectors are floatX,intX,uintX,shortX,ushortX,charX and ucharX where X is the component count between 2 and 4 included.
Smaller types like half, char and short can be used but are not given to be the same size on all implementation. Their use is forbidden in interface structs.
They are likely to be promoted to 32 bit integers.
Enum¶
Enums are supported when defined in namespace or global scope.
Underlying type should always be set.
All values must be explicitly set using integers literals. Do not forgetu suffix for unsigned literal.
enum axes : uchar { // Works
AXES_X = 1u,
AXES_Y = 2u,
AXES_Z = 3u,
};
enum axes : uchar {
AXES_X, // Doesn't work
AXES_Y,
AXES_Z,
};
Enum class are not supported but could be in the future.
Note that they are defined as global constant in GLSL and as native enum in MSL.
Reference¶
References in function scope are supported but have limitations:
* Their definition must not have side effect or function call.
* Array subscript index must be const qualified.
int &a = int_buffer[i++]; // Not ok, i++ has side effect
const int j = i++;
int &a = int_buffer[i]; // Works, j is const qualified
References inside function arguments are supported and are transformed into inout qualifiers.
Assert¶
The function assert(int) can be used just like in C.
Note that these only have effects if the cmake option WITH_GPU_SHADER_ASSERT is on.
Printf¶
printf can be used inside shader code to debug variables.
Note that the formatting options are limited to %u, %d, %f, %x without extra parameters.
The output is flushed only when printf_end is called on the host side.
The buffer is limited in the number of calls. So care must be taken to only call printf from a limited number of threads.
Strings support are only added for printf and they cannot be manipulated.
Bug
There is currently a bug preventing the use of formats starting with % (will result in crash).
Using a space before it will fix the issue.
Unroll¶
The [[gpu::unroll]] attribute can be added in front of for loops to manually unroll a loop.
Be aware that this is a lot more dumb than a real compiler option.
The number of iteration can be set manually for loop with conditions referencing external variables.
break and continue statements are forbidden inside an unrolled loop.
Be aware that this expansion happens at compile time and that the resulting copy pasted text is store in the final blender executable.
Namespace¶
Namespaces works like in C++ but are much more limited.
Automatic namespace elision is only supported inside the scope where the definition occurs.
namespace A {
void func() {}
void test() {
func(); // Works
}
} // A
namespace A {
void test2() {
func(); // Doesn't work
A::func(); // Works
using A::func;
func(); // Works
}
} // A
The syntax using namespace is not supported. All needed symbols from a namespace need to be mane visible manually.
The using syntaxes using X and using X = Y are allowed at both function and namespace scopes.
However, when used in namespace scope, it can only bring symbols from the same namespace.
namespace B {
void test() {}
} // B
namespace A {
void func() {}
} // A
namespace A {
using A::func; // Works
using B::test; // Doesn't work
} // A
Default Arguments¶
Default arguments are supported on functions.
These are implemented as function overloads and will expand to something similar as:Template¶
Templates must use explicit instantiation:
template<typename VecT> bool is_zero(VecT vec)
{
return all(equal(vec, VecT(0.0f)));
}
template bool is_zero<float2>(float2);
template bool is_zero<float3>(float3);
template bool is_zero<float4>(float4);
Template explicit instantiation must specify all parameters explicitly:
Templates having all arguments present in function signature must to be called with template argument deduction:
On the other side, templates not having all arguments present in function signature must to be called fully explicit:
template<typename T, int i> // i not in function arguments
void func(T a) {}
func<2>(2.0f); // Not OK
func<float, 2>(2.0f); // OK
Template calling templates are supported.
Template arguments can only be typename, integers or enums.
Template of template are not supported.
Template default arguments are not supported.
Templated types are not supported.
Variadic templates are not supported.
Implementation is currently done using macros, so be aware of different symbol types having the same names.