Migrate OpenGL Shaders to Vulkan. #89525

New Issue

Jeroen Bakker · 2021-06-29T11:07:03+02:00

Jeroen Bakker commented

2021-06-29 11:07:03 +02:00

Goal of this task is to convert all OpenGL shaders to Vulkan shaders. When using an OpenGL backend the syntax will be parsed and converted back to an OpenGL shader.

Numbers

Blender has

category	Number of shaders
gpu builtin shaders	62 static
draw manager	3 static
basic engine	2 static
image engine	2 static
overlay engine	172 static
grease pencil	17 static
workbench	169 static
Eevee (1)	83 static + material shaders

NOTE: There might be some python shaders or internally shaders that we haven't counted here.

NOTE: Overlay, workbench engine show a worst case scenario where each shader config generates different shaders. In reality only a subset would only create a different shader.

NOTE: Eevee2 isn't taken into account as it is still in development, but is expected to land before Vulkan port. As we want to perform the shader migration in master we should not convert any eevee shaders yet.

NOTE: Eevee2 shader layout is already organized to migrate to uniform to buffers.

Approach

The idea is to do this migration in master this way any changes in the shaders should be done in the Vulkan syntax and reducing the risk of branches to get out of sync.

Changes

See https://github.com/KhronosGroup/GLSL/blob/master/extensions/khr/GL_KHR_vulkan_glsl.txt for an overview of the changes.

There is a relax version https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GL_EXT_vulkan_glsl_relaxed.txt that is added for better compatibility with OpenGL shaders. It does not match our approach where vulkan GLSL syntax is leading.

NOTE: This list isn't complete and the implications still needs to be discussed.

Builtin Uniforms

Blender auto binds common matrices to shaders. We should check how many different combinations are actually using uniform mat4. Create structs for the common combinations and lazy initialize these combinations.
Draw manager already separates matrices into a viewBlock and a modelBlock. We could reuse a similar approach.

NOTE: See https://docs.google.com/spreadsheets/d/1JtE4lLJxg6faANZgz8PGFVQTm0a64wmrENiGxgNP0iI/edit?usp=sharing for some analysis on usage and grouping.

A common used configuration is

uniform mat4 ModelViewProjectionMatrix;
uniform mat4 ModelMatrix;

This could be converted to (is already done in draw manager but more complex as it supports instances)
Approach would be to extract the type info from the source layout(binding = 1) uniform <structurename> {

Grouping of the uniforms into common structs:

Group 0		257 shaders	push constant?
Group 1	MODEL, RESOURCE_CHUNK, RESOURCE_ID	80 shaders, mostly workbench and overlay	Yes
Group 2	COLOR	20 shaders	Yes
Group 3	MODEL, MODEL_INV	4 shaders	Yes
Group 4	MODEL, MVP, CLIPPLANES, COLOR, SRGB_TRANSFORM	58 shaders	No
Group 5	PROJECTION, VIEWPROJECTION, CLIPPLANES, SRGB_TRANSFORM	3 shaders	No
Group 6	MVP, NORMAL, COLOR, SRGB_TRANSFORM	1 shader	No
Group Fallback	Contains all builtin uniforms	In case there is no match with any groups mentioned above.	No

NOTE:Grouping is based on actual usage after linking. It could be that a shader needs to support multiple groups when it is used in multiple configurations.

CPP

struct ObjectMatrices {
  float ModelMatrix[4][4];
  float ModelInv[4][4];
};

Vulkan GLSL

layout(binding = 1) uniform ObjectMatrices {
  mat4 ModelMatrix;
  mat4 ModelInv;
};

OpenGL GLSL

layout(std140) uniform ObjectMatrices {
  mat4 ModelMatrix;
  mat4 ModelInv;
};

ID => Index
In Vulkan syntax the gl_VertexID has been renamed to gl_VertexIndex. These kind of changes will be send in for review one a by one basis for all shaders. These changes have low risk, are easy to do. See 44c875f59a as an example.

Same should be done for gl_InstanceID

Name Attachments

OpenGL queries binding location of shader elements based on its name. In vulkan this is done based layout keyword of the shader element see Description sets in the vulkan specification.

Unclear how to handle these yet. One idea is to preprocess the shader source to extract the names binding map so we can still support name lookup.

Builtin uniforms

In vulkan uniforms cannot be addressed individually. They should become part of a buffer. Precise details how to organize these buffers isn't clear yet.

To keep the structures up to date we should be able to share the code between GLSL and C/CPP.

Goal of this task is to convert all OpenGL shaders to Vulkan shaders. When using an OpenGL backend the syntax will be parsed and converted back to an OpenGL shader. ## Numbers Blender has | **category** | **Number of shaders** | | -- | -- | | gpu builtin shaders| 62 static | | draw manager | 3 static | | basic engine | 2 static | | image engine | 2 static | | overlay engine | 172 static | | grease pencil | 17 static | | workbench | 169 static | | Eevee (1) | 83 static + material shaders | NOTE: There might be some python shaders or internally shaders that we haven't counted here. NOTE: Overlay, workbench engine show a worst case scenario where each shader config generates different shaders. In reality only a subset would only create a different shader. NOTE: Eevee2 isn't taken into account as it is still in development, but is expected to land before Vulkan port. As we want to perform the shader migration in master we should not convert any eevee shaders yet. NOTE: Eevee2 shader layout is already organized to migrate to uniform to buffers. ## Approach The idea is to do this migration in master this way any changes in the shaders should be done in the Vulkan syntax and reducing the risk of branches to get out of sync. ## Changes See https://github.com/KhronosGroup/GLSL/blob/master/extensions/khr/GL_KHR_vulkan_glsl.txt for an overview of the changes. There is a relax version https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GL_EXT_vulkan_glsl_relaxed.txt that is added for better compatibility with OpenGL shaders. It does not match our approach where vulkan GLSL syntax is leading. NOTE: This list isn't complete and the implications still needs to be discussed. **Builtin Uniforms** Blender auto binds common matrices to shaders. We should check how many different combinations are actually using `uniform mat4`. Create structs for the common combinations and lazy initialize these combinations. Draw manager already separates matrices into a viewBlock and a modelBlock. We could reuse a similar approach. NOTE: See https://docs.google.com/spreadsheets/d/1JtE4lLJxg6faANZgz8PGFVQTm0a64wmrENiGxgNP0iI/edit?usp=sharing for some analysis on usage and grouping. A common used configuration is ``` uniform mat4 ModelViewProjectionMatrix; uniform mat4 ModelMatrix; ``` This could be converted to (is already done in draw manager but more complex as it supports instances) Approach would be to extract the type info from the source `layout(binding = 1) uniform <structurename> {` Grouping of the uniforms into common structs: | Group 0 | <empty> | 257 shaders | push constant? | | -- | -- | -- | -- | | Group 1 | MODEL, RESOURCE_CHUNK, RESOURCE_ID | 80 shaders, mostly workbench and overlay | Yes | | Group 2 | COLOR | 20 shaders | Yes | | Group 3 | MODEL, MODEL_INV| 4 shaders | Yes | | Group 4 | MODEL, MVP, CLIPPLANES, COLOR, SRGB_TRANSFORM |58 shaders | No | | Group 5 | PROJECTION, VIEWPROJECTION, CLIPPLANES, SRGB_TRANSFORM | 3 shaders | No| | Group 6 | MVP, NORMAL, COLOR, SRGB_TRANSFORM | 1 shader | No | | Group Fallback | Contains all builtin uniforms | In case there is no match with any groups mentioned above. | No | NOTE:Grouping is based on actual usage after linking. It could be that a shader needs to support multiple groups when it is used in multiple configurations. **CPP** ``` struct ObjectMatrices { float ModelMatrix[4][4]; float ModelInv[4][4]; }; ``` **Vulkan GLSL** ``` layout(binding = 1) uniform ObjectMatrices { mat4 ModelMatrix; mat4 ModelInv; }; ``` **OpenGL GLSL** ``` layout(std140) uniform ObjectMatrices { mat4 ModelMatrix; mat4 ModelInv; }; ``` **ID => Index** In Vulkan syntax the `gl_VertexID` has been renamed to `gl_VertexIndex`. These kind of changes will be send in for review one a by one basis for all shaders. These changes have low risk, are easy to do. See 44c875f59a as an example. Same should be done for `gl_InstanceID` **Name Attachments** OpenGL queries binding location of shader elements based on its name. In vulkan this is done based `layout` keyword of the shader element see Description sets in the vulkan specification. Unclear how to handle these yet. One idea is to preprocess the shader source to extract the names binding map so we can still support name lookup. **Builtin uniforms** In vulkan uniforms cannot be addressed individually. They should become part of a buffer. Precise details how to organize these buffers isn't clear yet. To keep the structures up to date we should be able to share the code between GLSL and C/CPP.

Jeroen Bakker self-assigned this 2021-06-29 11:07:03 +02:00

Jeroen Bakker commented

2021-06-29 11:07:03 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Jeroen Bakker commented

2021-06-29 11:07:03 +02:00

Added subscriber: @Jeroen-Bakker

hujin commented

2021-06-29 11:14:48 +02:00

Added subscriber: @2046411367

Rahul Chaudhary commented

2021-06-29 14:27:16 +02:00

Added subscriber: @RC12

Clément Foucault commented

2021-06-29 19:49:31 +02:00

Added subscriber: @fclem

Clément Foucault commented

2021-06-29 19:49:31 +02:00

Another thing to not forget is how does one updates a pyGPU shader to this version.

To me, one would not have to re-declare the builtin uniforms and just have to group uniforms into a uniform buffer.
I would prevent the use of push constant for now in order to avoid more complexity.

However, this means that UBO management from the pyGPU needs to be improved.

Another thing to not forget is how does one updates a pyGPU shader to this version. To me, one would not have to re-declare the builtin uniforms and just have to group uniforms into a uniform buffer. I would prevent the use of push constant for now in order to avoid more complexity. However, this means that UBO management from the pyGPU needs to be improved.

Chris Kohl commented

2021-06-29 21:36:43 +02:00

Added subscriber: @ckohl_art

Paul Larson commented

2021-06-30 02:55:30 +02:00

Added subscriber: @GeorgiaPacific

Gilberto Rodrigues commented

2021-07-06 06:30:10 +02:00

Added subscriber: @Gilberto.R

Basil commented

2021-07-26 04:57:59 +02:00

Added subscriber: @Defka

newin commented

2021-09-26 15:59:17 +02:00

Added subscriber: @newin

newin commented

2021-09-26 21:28:58 +02:00

I just spend a bit of time trying to figure out different options to support both Vulkan and OpenGL in the layout binding issue (which is kind of mixed with https://developer.blender.org/T89684).
The probably most ubiquitous way of doing this would be to wrap uniform declarations into macros that can be both parsed to extract those names and expanded into different forms depending on the target version.
Even though I've seen this fairly often it very often it tend to produce code quite unpleasant to read very fast + add a parser that can be the source of bug and add maintenance work and all. It certainly an option but I'm not a fan of this direction.

The other well-known way to handle that would be to do exactly the opposite: generate uniforms on the fly just before the shader compilation. It may be a good idea from a software design perspective (everything is automatically consistent + no parsing involved) in reality this is relatively impractical to have piece of the code missing when working on shaders. (Unreal does that a lot and that's always very annoying). That being said I've only looked at static shaders and not materials, maybe materials already do that and it would be possible to use that to all shaders.

The last would be to have a very simple set of "compatibility layer" macros (probably just a few #define in glsl_patch_default_get()) then use with the standard reflection features for opengl and add SPIRV-Reflect as a dependency for vulkan. The whole strategy would be to build a simple wrapper that would get/set those values in some generic way.
For game development I wouldn't advise the last (because of performance and portability reasons) but for blender I think it's both elegant and simple and the perf impact should be a very big deal.
Problem is: it will add a dependency of something under Apache Version 2.0 Licence and I have no idea if I can (or even should) do that.
Should I go ahead and add the dependency ?

PS: There is probably another option based on explicit binding/layout locations but as it would require OpenGL 4+ I haven't spend much time on this front. I'm sure OpenGL 3.3 as minimal has been chosen for a good reason.

I just spend a bit of time trying to figure out different options to support both Vulkan and OpenGL in the layout binding issue (which is kind of mixed with https://developer.blender.org/T89684). The probably most ubiquitous way of doing this would be to wrap uniform declarations into macros that can be both parsed to extract those names and expanded into different forms depending on the target version. Even though I've seen this fairly often it very often it tend to produce code quite unpleasant to read very fast + add a parser that can be the source of bug and add maintenance work and all. It certainly an option but I'm not a fan of this direction. The other well-known way to handle that would be to do exactly the opposite: generate uniforms on the fly just before the shader compilation. It may be a good idea from a software design perspective (everything is automatically consistent + no parsing involved) in reality this is relatively impractical to have piece of the code missing when working on shaders. (Unreal does that a lot and that's always very annoying). That being said I've only looked at static shaders and not materials, maybe materials already do that and it would be possible to use that to all shaders. The last would be to have a very simple set of "compatibility layer" macros (probably just a few `#define` in `glsl_patch_default_get()`) then use with the standard reflection features for opengl and add [SPIRV-Reflect](https://github.com/KhronosGroup/SPIRV-Reflect) as a dependency for vulkan. The whole strategy would be to build a simple wrapper that would get/set those values in some generic way. For game development I wouldn't advise the last (because of performance and portability reasons) but for blender I think it's both elegant and simple and the perf impact should be a very big deal. Problem is: it will add a dependency of something under Apache Version 2.0 Licence and I have no idea if I can (or even should) do that. Should I go ahead and add the dependency ? PS: There is probably another option based on explicit binding/layout locations but as it would require OpenGL 4+ I haven't spend much time on this front. I'm sure OpenGL 3.3 as minimal has been chosen for a good reason.

Emi Martinez commented

2021-10-11 21:10:49 +02:00

Added subscriber: @Emi_Martinez

Donovane commented

2021-10-11 21:28:52 +02:00

Added subscriber: @dodo-2

Jarrett Johnson commented

2021-11-01 07:44:56 +01:00

Added subscriber: @Jarrett-Johnson

Clément Foucault commented

2021-11-24 22:17:17 +01:00

@newin Thank you for your feedback. I did not thought about generating the interface at runtime. In fact, this is what we now are considering to do. This has benefits even if it makes shader harder to read:

Shaders are statically defined in their own structure, making it possible to compile to SpirV or check shader errors at compile time.
Making the interface abstract is much easier for backends that have non globals scope for their resources (DirectX, Metal).
It would remove a lot of complexity in shader managment.

We are already fragmenting our shaders to share code (i.e: stage interfaces) so I don't see fragmentation as a big issue.

The ugly side of this is that we would have to create a pyGPU API for addons to create shaders this way.

One major drawback is that it needs manual effort to write down the interface. But for simple internal shaders, it could be automated.

I've push an experimental branch called gpu-shader-descriptor where there is a proof of concept implementation of what I had in mind. Also see D13360 for discussion about the implementation.

@newin Thank you for your feedback. I did not thought about generating the interface at runtime. In fact, this is what we now are considering to do. This has benefits even if it makes shader harder to read: - Shaders are statically defined in their own structure, making it possible to compile to SpirV or check shader errors at compile time. - Making the interface abstract is much easier for backends that have non globals scope for their resources (DirectX, Metal). - It would remove a lot of complexity in shader managment. We are already fragmenting our shaders to share code (i.e: stage interfaces) so I don't see fragmentation as a big issue. The ugly side of this is that we would have to create a pyGPU API for addons to create shaders this way. One major drawback is that it needs manual effort to write down the interface. But for simple internal shaders, it could be automated. I've push an experimental branch called `gpu-shader-descriptor` where there is a proof of concept implementation of what I had in mind. Also see [D13360](https://archive.blender.org/developer/D13360) for discussion about the implementation.

Spundun Bhatt commented

2022-06-21 01:33:20 +02:00

Added subscriber: @Spundun-Bhatt

Spundun Bhatt commented

2022-06-21 01:33:20 +02:00

I was trying to figure out what's the work to be done for this task... I noticed that a lot of work has already gone into this with...

Is there anything else in scope for this task? I'm looking for starter projects that'd help me familiarize with the Vulkan API.

ps: Left some more details about my investigation of the work done here: #94975#1377420

I was trying to figure out what's the work to be done for this task... I noticed that a lot of work has already gone into this with... * [All commits with the word "GPUShaderCreateInfo"](https://developer.blender.org/search/query/81jFCPCnq7U./#R) * [All Differential Revisions with the word "GPUShaderCreateInfo"](https://developer.blender.org/search/query/SmM5u.pYJVzL/#R) Is there anything else in scope for this task? I'm looking for starter projects that'd help me familiarize with the Vulkan API. ps: Left some more details about my investigation of the work done here: #94975#1377420

Jeroen Bakker commented

2022-06-21 10:04:44 +02:00

@Spundun-Bhatt, yes we had to do the conversion already for the metal backend. Changes in eevee still needs to be done, but that will only be done for eevee-next. Will mark this task complete.

Hmmm... it can be a hard task to familiarize with Vulkan when introducing it in a project like Blender. The next step would be to implement the VKShader. That requires shaderc and other changes that are in the tmp-vulkan branch.

Note that the tmp-vulkan branch hasn't been updated for at least 6 months...

@Spundun-Bhatt, yes we had to do the conversion already for the metal backend. Changes in eevee still needs to be done, but that will only be done for eevee-next. Will mark this task complete. Hmmm... it can be a hard task to familiarize with Vulkan when introducing it in a project like Blender. The next step would be to implement the VKShader. That requires shaderc and other changes that are in the tmp-vulkan branch. Note that the tmp-vulkan branch hasn't been updated for at least 6 months...

Jeroen Bakker commented

2022-06-21 10:04:55 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Jeroen Bakker closed this issue

2022-06-21 10:04:55 +02:00

Aleksander Berg-Jones commented

2022-07-30 21:50:14 +02:00

Added subscriber: @bergjones

Aleksander Berg-Jones commented

2022-07-30 21:50:14 +02:00

After merging a relatively recent blender (within the past month) and tmp-vulkan I cannot get the gpu_shader_create_info_compile_all to work when compiling to spv. For example, the "vertex_out_interfaces_" portion of vertex_interface_declare gives

/* Interfaces. */
out flat_color_iface{
  flat vec4 finalColor;
};

when I need

layout(location = 0) out vec4 finalColor;

A similar situation for the "Push Constants" which for example give

/* Push Constants. */
uniform mat4 ModelViewProjectionMatrix;

gl_position = ModelViewProjectionMatrix * .....

when I need

layout (binding = 0) uniform UBO 
{
	mat4 projectionMatrix;
} ubo;

gl_position = ubo.ModelViewProjectionMatrix * .....

Maybe I may be missing something., but I cannot convert even simple shaders like 2d_flat_color. What I just did was in GPU_SHADER_CREATE_INFO added "vertex_out_VK" function and make it behave similar to vertex_in. I am working on making the push constants like MVP behave like UBO as well.

After merging a relatively recent blender (within the past month) and tmp-vulkan I cannot get the gpu_shader_create_info_compile_all to work when compiling to spv. For example, the "vertex_out_interfaces_" portion of vertex_interface_declare gives ``` /* Interfaces. */ out flat_color_iface{ flat vec4 finalColor; }; ``` when I need ``` layout(location = 0) out vec4 finalColor; ``` A similar situation for the "Push Constants" which for example give ``` /* Push Constants. */ uniform mat4 ModelViewProjectionMatrix; gl_position = ModelViewProjectionMatrix * ..... ``` when I need ``` layout (binding = 0) uniform UBO { mat4 projectionMatrix; } ubo; gl_position = ubo.ModelViewProjectionMatrix * ..... ``` Maybe I may be missing something., but I cannot convert even simple shaders like 2d_flat_color. What I just did was in GPU_SHADER_CREATE_INFO added "vertex_out_VK" function and make it behave similar to vertex_in. I am working on making the push constants like MVP behave like UBO as well.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Migrate OpenGL Shaders to Vulkan. #89525

Numbers

Approach

Changes