Design: DrawManager Colormanagement #71357

New Issue

Jeroen Bakker · 2019-11-05T11:58:43+01:00

Jeroen Bakker commented

2019-11-05 11:58:43 +01:00

This task will go over improvements I want to suggest to the Draw Manager color management. This ticket is in discussion, in the hope to get viewpoints of experts about the subject.

The scope of this ticket is limited to the draw manager as there are some isolated issues here. After the design is approved, the engineering plan can be written. In the engineering plan we will go on the how and code-technical implications.

Current Situation

DrawManager is used in many areas with other target colorspace needs than currently available. DrawManager has a flag do_colormanagement, what can either be on or off. When its on, it is up to the draw engines to do the correct expected color management actions. Most of the time this has been implemented with a lot of assumptions about colorspace of the input and output.

For example:

Workbench, Grease Pencil assumes that this flag can be used to detect the colorspace of the input and output buffers (Rec709 vs sRGB). Both don't use the scene color management settings for this, but fixed transform functions.
Overlay engines assumes that their input and output is always in sRGB.

Usages of DrawManager

Drawing of the 3d viewport to the display: the draw manager will render workbench, eevee, greasepencil, external render engines and additional overlays directly to the device output.
Viewport rendering: The draw manager will render workbench, eevee, greasepencil, external render engines and additional overlays to an render result. results are stored in Rec709.
Scene Strip: the draw manager will render workbench/eevee and greasepencil results are stored in an ImageBuffer in (sRGB during editing, during final rendering the Image Rendering is used).
Image Rendering: the draw manager will render workbench/eevee and greasepencil results is stored in an Image buffer in Rec709.
PyAPI: render.opengl

Issues

When doing viewport rendering the color management is done on the CPU what is typically slow.
When doing viewport rendering the overlay engine assumes that the input/output buffers are sRGB, but in fact they aren't, they are still scene Rec709.
When using a scene strip the buffer is not converted to sequencer, but to sRGB. With the default settings these are the same, but when users change it...
Rendering greasepencil results in different results in the different situations.
Movieclips/Images as Camera Background Images/Image empties do not respect color management settings of the image. They assume (as all overlay engines) that they input/output is in sRGB and are only able to handle Rec709 and sRGB images/movieclips.
...

Proposal

Generated GLSL color transfer functions by OpenColorIO configuration. common_colormanagement_lib.glsl. Make the functions OpenColorIO role based. this way it is more clear what function to use in what situation. The role based approach would also limit the configurations that needs to be handled by the shader. But some color management pre-processing needs to be done when images are not using a space defined by the role there in.
To be able to set a target OpenColorIO role, when drawing to an offscreen buffer. At the end of the drawing the offscreen buffer is then converted to the ColorSpace that is in the colormanagement configuration for that specific Role.
When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in the reference soace and only as a last step the view transform will be applied.
Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects. Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its own encoding to the scene linear space.
Make sure that when overlays are drawn, the input buffers are transferred to the expected color space (display), and transferred back afterwards. Note that color transfers are not reversible and that leads to other artifacts. This part still needs more thought.

This task will go over improvements I want to suggest to the Draw Manager color management. This ticket is in discussion, in the hope to get viewpoints of experts about the subject. The scope of this ticket is limited to the draw manager as there are some isolated issues here. After the design is approved, the engineering plan can be written. In the engineering plan we will go on the how and code-technical implications. **Current Situation** DrawManager is used in many areas with other target colorspace needs than currently available. DrawManager has a flag `do_colormanagement`, what can either be on or off. When its on, it is up to the draw engines to do the correct expected color management actions. Most of the time this has been implemented with a lot of assumptions about colorspace of the input and output. For example: * Workbench, Grease Pencil assumes that this flag can be used to detect the colorspace of the input and output buffers (Rec709 vs sRGB). Both don't use the scene color management settings for this, but fixed transform functions. * Overlay engines assumes that their input and output is always in sRGB. Usages of DrawManager * Drawing of the 3d viewport to the display: the draw manager will render workbench, eevee, greasepencil, external render engines and additional overlays directly to the device output. * Viewport rendering: The draw manager will render workbench, eevee, greasepencil, external render engines and additional overlays to an render result. results are stored in Rec709. * Scene Strip: the draw manager will render workbench/eevee and greasepencil results are stored in an ImageBuffer in (sRGB during editing, during final rendering the Image Rendering is used). * Image Rendering: the draw manager will render workbench/eevee and greasepencil results is stored in an Image buffer in Rec709. * PyAPI: render.opengl **Issues** * When doing viewport rendering the color management is done on the CPU what is typically slow. * When doing viewport rendering the overlay engine assumes that the input/output buffers are sRGB, but in fact they aren't, they are still scene Rec709. * When using a scene strip the buffer is not converted to sequencer, but to sRGB. With the default settings these are the same, but when users change it... * Rendering greasepencil results in different results in the different situations. * Movieclips/Images as Camera Background Images/Image empties do not respect color management settings of the image. They assume (as all overlay engines) that they input/output is in sRGB and are only able to handle Rec709 and sRGB images/movieclips. * ... **Proposal** * Generated GLSL color transfer functions by OpenColorIO configuration. `common_colormanagement_lib.glsl`. Make the functions OpenColorIO role based. this way it is more clear what function to use in what situation. The role based approach would also limit the configurations that needs to be handled by the shader. But some color management pre-processing needs to be done when images are not using a space defined by the role there in. * To be able to set a target OpenColorIO role, when drawing to an offscreen buffer. At the end of the drawing the offscreen buffer is then converted to the ColorSpace that is in the colormanagement configuration for that specific Role. * When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in the reference soace and only as a last step the view transform will be applied. **Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects.** Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its own encoding to the scene linear space. * Make sure that when overlays are drawn, the input buffers are transferred to the expected color space (display), and transferred back afterwards. Note that color transfers are not reversible and that leads to other artifacts. This part still needs more thought.

Jeroen Bakker self-assigned this 2019-11-05 11:58:43 +01:00

Jeroen Bakker commented

2019-11-05 11:58:43 +01:00

Added subscribers: @Jeroen-Bakker, @fclem

Evan Wilson commented

2019-11-06 05:55:45 +01:00

Added subscriber: @EAW

Sebastian Koenig commented

2019-11-06 11:25:10 +01:00

Added subscriber: @sebastian_k

item412 commented

2019-11-06 23:50:03 +01:00

Added subscriber: @item412

Sam Van Hulle commented

2019-11-07 14:06:24 +01:00

Added subscriber: @frameshift

Sam Van Hulle commented

2019-11-07 14:11:34 +01:00

Many many kudos for the write-up. This would tackle a huge amount of the current issues.

When doing viewport rendering the color management is done on the CPU what is typically slow.

But doing it on the GPU would be less accurate, you once told me. How inaccurate are we talking? Is it worth the trade-off?

Many many kudos for the write-up. This would tackle a huge amount of the current issues. > When doing viewport rendering the color management is done on the CPU what is typically slow. But doing it on the GPU would be less accurate, you once told me. How inaccurate are we talking? Is it worth the trade-off?

Michael Hermann commented

2019-11-12 22:05:44 +01:00

Added subscriber: @MichaelHermann

Sam Van Hulle commented

2019-11-19 16:26:53 +01:00

When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in REC709 and only as a last step the view transform will be applied.
-- Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects.
-- Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its color space to one of them (depends a bit on use).

Just to be on the safe side, I'd like to highlight some stuff in an attempt to prevent any hardcoding from being left behind.

When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in ~~REC709~~ the reference space and only as a last step the view transform will be applied.
- Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects.
- Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its ~~color space to one of them (depends a bit on use)~~ own encoding to the scene linear reference space.

I just like the word encoding because it implies more than just primaries. For instance, if I have a Filmic image, it can only be converted to the scene linear space by inverting that Filmic Log transform. Its primaries are still Rec. 709's, but its encoding is far from sRGB or BT.1886.

> - When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in REC709 and only as a last step the view transform will be applied. > -- Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects. > -- Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its color space to one of them (depends a bit on use). Just to be on the safe side, I'd like to highlight some stuff in an attempt to prevent any hardcoding from being left behind. - *When rendering and compositing the linear scene ref space (default REC709) must be used. This means that the overlay/mode engines will be drawing in ~~REC709~~ **the reference space** and only as a last step the view transform will be applied.* - *Note that this will also impact how Grease pencil blends colors and there will be rendering differences. These differences are currently also noticeable during final rendering of GP projects.* - *Images that are rendered in the background are currently fixed to REC709 or sRGB. other colorspaces are ignored. We should transform the image buffer from its ~~color space to one of them (depends a bit on use)~~ **own encoding to the scene linear reference space**.* I just like the word encoding because it implies more than just primaries. For instance, if I have a Filmic image, it can only be converted to the scene linear space by inverting that Filmic Log transform. Its primaries are still Rec. 709's, but its encoding is far from sRGB or BT.1886.

Jeroen Bakker commented

2019-11-19 16:37:45 +01:00

Thanks updated. The more I look into the subject the more issues I detect. Might be that I start with some small steps. As some stuff that OCIO provides is a bit hidden behind a wrapper. Need to find out if we actually want this wrapper still to be used. I don't like wrappers that limits possibilities.

Sam Van Hulle commented

2019-11-19 17:00:38 +01:00

In #71357#813086, @Jeroen-Bakker wrote:
The more I look into the subject the more issues I detect.

Ironically, that makes me very happy :D

I don't like wrappers that limits possibilities.

I get that, but something tells me they're there for a reason..?

> In #71357#813086, @Jeroen-Bakker wrote: > The more I look into the subject the more issues I detect. Ironically, that makes me very happy :D > I don't like wrappers that limits possibilities. I get that, but something tells me they're there for a reason..?

Anton Raves commented

2019-11-26 17:28:47 +01:00

Added subscriber: @Memento

Jeroen Bakker commented

2019-12-13 14:46:04 +01:00

I have added #72420 (Design: DrawEngine Color Management) as a sub-task for discussing a first step.

Jeroen Bakker commented

2020-06-15 10:35:13 +02:00

Changed status from 'Confirmed' to: 'Archived'

Jeroen Bakker closed this issue

2020-06-15 10:35:13 +02:00

Anton Raves commented

2020-06-15 12:34:33 +02:00

Removed subscriber: @Memento

Sam Van Hulle commented

2020-06-15 17:55:24 +02:00

@Jeroen-Bakker Could you elaborate on why this design is invalid? I'm not super familiar with the internals of Blender, but describing the issue might help people understand.

Jeroen Bakker commented

2020-06-15 20:39:49 +02:00

Hi Sam,

At the end we came up with a different design to solve this issue. the final design all draw engines work in linear scene ref space. drawing results into 2 images for each eye (render engine image and overlay engine image). when drawing to the screen these images are composited and converted to display space. This all is done in the GPUViewport. Also external render engines should now render in linear ref space when showing in the viewport.

Hi Sam, At the end we came up with a different design to solve this issue. the final design all draw engines work in linear scene ref space. drawing results into 2 images for each eye (render engine image and overlay engine image). when drawing to the screen these images are composited and converted to display space. This all is done in the GPUViewport. Also external render engines should now render in linear ref space when showing in the viewport.

Sam Van Hulle commented

2020-06-16 20:07:43 +02:00

@Jeroen-Bakker sounds great, thanks!

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Design: DrawManager Colormanagement #71357