Compositor backdrop handles alpha differently from image editor #80748

New Issue

Georg K · 2020-09-13T16:57:17+02:00

Georg K commented

2020-09-13 16:57:17 +02:00

System Information
Operating system: Linux-5.8.6-1-MANJARO-x86_64-with-glibc2.32 64 Bits
Graphics card: AMD VERDE (DRM 2.50.0, 5.8.6-1-MANJARO, LLVM 10.0.1) X.Org 4.5 (Core Profile) Mesa 20.1.7

Blender Version
Broken: version: 2.91.0 Alpha, branch: master, commit date: 2020-09-13 09:15, hash: f5ccf8727f
Worked: 2.90

Short description of error
Keying node alpha channel is not taken into account 100% when rendering.

On attached file:
2.90: background keyed out (not perfect because I left everything else on defaults)
2.91 alpha: background looks desaturated semi-transparent

Exact steps for others to reproduce the error

Enter compositor
add greenscreen source (for example https://www.vegascreativesoftware.com/fileadmin/user_upload/seo/keyword/guide_to_chroma_key_and_green_screen/vegas-pro-separate-subject-int.jpg)
add keying node (select a key color, everything else at default), connect viewer node
activate backdrop or look at viewer node output in image editor
F12 render and compare result to backdrop
or
Open attached file
F12 render
Keying node bug.blend

**System Information** Operating system: Linux-5.8.6-1-MANJARO-x86_64-with-glibc2.32 64 Bits Graphics card: AMD VERDE (DRM 2.50.0, 5.8.6-1-MANJARO, LLVM 10.0.1) X.Org 4.5 (Core Profile) Mesa 20.1.7 **Blender Version** Broken: version: 2.91.0 Alpha, branch: master, commit date: 2020-09-13 09:15, hash: `f5ccf8727f` Worked: 2.90 **Short description of error** Keying node alpha channel is not taken into account 100% when rendering. On attached file: 2.90: background keyed out (not perfect because I left everything else on defaults) 2.91 alpha: background looks desaturated semi-transparent **Exact steps for others to reproduce the error** - Enter compositor - add greenscreen source (for example https://www.vegascreativesoftware.com/fileadmin/user_upload/seo/keyword/guide_to_chroma_key_and_green_screen/vegas-pro-separate-subject-int.jpg) - add keying node (select a key color, everything else at default), connect viewer node - activate backdrop or look at viewer node output in image editor - F12 render and compare result to backdrop or - Open attached file - F12 render [Keying node bug.blend](https://archive.blender.org/developer/F8872906/Keying_node_bug.blend)

Georg K commented

2020-09-13 16:57:17 +02:00

Added subscriber: @georgK

Germano Cavalcante commented

2020-09-15 19:45:12 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Germano Cavalcante commented

2020-09-15 19:47:30 +02:00

Added subscribers: @Jeroen-Bakker, @mano-wii

Germano Cavalcante commented

2020-09-15 19:47:30 +02:00

It is indeed a regression.
@Jeroen-Bakker, I wonder if it has anything to do with the recent changes in the image editor.

It is indeed a regression. @Jeroen-Bakker, I wonder if it has anything to do with the recent changes in the image editor.

Georg K commented

2020-09-17 22:38:16 +02:00

Backdrop rendering looks normal in version: 2.91.0 Alpha, branch: master, commit date: 2020-09-17 20:18, hash: 9ee588cd4a, but F12 rendering still has artifacts.

Backdrop rendering looks normal in version: 2.91.0 Alpha, branch: master, commit date: 2020-09-17 20:18, hash: `9ee588cd4a`, but F12 rendering still has artifacts.

Sergey Sharybin commented

2020-10-13 12:36:58 +02:00

Added subscribers: @brecht, @Sergey

Sergey Sharybin commented

2020-10-13 12:36:58 +02:00

Keying node violates pre-multiplication of the compositing buffer: it sets alpha without multiplying color channels. The original motivation behind this was to be able to chain multiple keying nodes. But now it seems that this decision causes more problems than solutions, so probably need to ensure pre-multiplication of result at where it is expected to be.

However, "fixing" this specific node does not fix the issue in general: there are many more ways to "ruin" pre-multiplication in compositor, and having difference between compositor backdrop and image editor is rather confusing.

@brecht, proposal: add "Convert Premul" checkbox enabled by default, but not do-versioned for existing files, so that compatibility is not ruined. That would at least cover common usecases of this specific node. The backdrop drawing is still needs to be corrected, but that's a different story where I can not help.

Keying node violates pre-multiplication of the compositing buffer: it sets alpha without multiplying color channels. The original motivation behind this was to be able to chain multiple keying nodes. But now it seems that this decision causes more problems than solutions, so probably need to ensure pre-multiplication of result at where it is expected to be. However, "fixing" this specific node does not fix the issue in general: there are many more ways to "ruin" pre-multiplication in compositor, and having difference between compositor backdrop and image editor is rather confusing. @brecht, proposal: add "Convert Premul" checkbox enabled by default, but not do-versioned for existing files, so that compatibility is not ruined. That would at least cover common usecases of this specific node. The backdrop drawing is still needs to be corrected, but that's a different story where I can not help.

Brecht Van Lommel commented

2020-10-13 12:46:32 +02:00

I'm fine with a Convert Premul option on the keying node.

Sergey Sharybin commented

2020-10-14 16:23:13 +02:00

Added D9211 which sanitizes the keying node alpha behaviour.

@Jeroen-Bakker, @brecht, I would like you to make the decision about alpha rendering difference between image editor and compositor backdrop. To me it seems confusing at a least :(

Added [D9211](https://archive.blender.org/developer/D9211) which sanitizes the keying node alpha behaviour. @Jeroen-Bakker, @brecht, I would like you to make the decision about alpha rendering difference between image editor and compositor backdrop. To me it seems confusing at a least :(

Brecht Van Lommel commented

2020-10-14 18:10:57 +02:00

I guess it's just a bug to be fixed by @Jeroen-Bakker, to ensure the compositor backdrop draws the same way as the image editor?

Not sure there is any decision to take from my side.

I guess it's just a bug to be fixed by @Jeroen-Bakker, to ensure the compositor backdrop draws the same way as the image editor? Not sure there is any decision to take from my side.

Sergey Sharybin commented

2020-10-28 12:34:27 +01:00

The keying node now delivers properly premultiplied color. There is also Cryptomatte node which needs to be made to output proper color. There is #82154 about it.

I still believe root of this issue goes to the fact that image editor uses different drawing of alpha than the rest of related editor (compositor backdrop, movie clip editor).

@Jeroen-Bakker, is it something you think can be addressed for 2.92? Do you want to adjust projects tags?

The keying node now delivers properly premultiplied color. There is also Cryptomatte node which needs to be made to output proper color. There is #82154 about it. I still believe root of this issue goes to the fact that image editor uses different drawing of alpha than the rest of related editor (compositor backdrop, movie clip editor). @Jeroen-Bakker, is it something you think can be addressed for 2.92? Do you want to adjust projects tags?

Sergey Sharybin changed title from ~~Keying node bug~~ to Compositor backdrop handles alpha differently from image editor

2020-10-28 12:34:50 +01:00

Jeroen Bakker self-assigned this 2020-10-28 13:37:14 +01:00

Jeroen Bakker commented

2020-10-28 13:37:14 +01:00

Yes, I am planning to work on it for 2.92. I will merge my todo into this task.

Jeroen Bakker commented

2020-10-28 15:09:56 +01:00

Added subscriber: @fclem

Jeroen Bakker commented

2020-10-28 15:09:56 +01:00

Technical challenge

The backdrop is rendered between parent node tree and the current active node tree.

#80748.blend

Current layering in the draw code is:

parent of parent node tree
green overlay
parent node tree
green overlay
grid
backdrop
active node specific drawing
node tree

Everything is currently done in display space to an offscreen image.

Solutions

change the drawpix method to retrieve the backdrop in scene reference space using a custom shader to integrate the backdrop and transfer to display space. The complexity here is the transfer to display space as our OCIO wrapper doesn't support this. the alpha channel might be different when compared to the image editor.
use a gpu viewport the background is rendered to the overlay buffer (1-5), the backdrop (6) is rendered to the color buffer. During drawing of the backdrop the overlay texture is changed to simulate an alpha under. The rest (7-8) will be rendered on the overlay buffer. When copying the viewport to the display the overlay and color buffers are merged and transform to the display space. This will lead to some artifacts when looking through a transparent canvas of the node, but can be considered as cosmetic.

Additional benefit of the GPUViewport solution is that stereoscoptic compositing will also give the same result as in the 3d viewport or image editor as they shade the same color pipeline.

@fclem do you have any suggestion?

## Technical challenge The backdrop is rendered between parent node tree and the current active node tree. ![Screenshot from 2020-10-28 14-40-41.png](https://archive.blender.org/developer/F9117691/Screenshot_from_2020-10-28_14-40-41.png) [#80748.blend](https://archive.blender.org/developer/F9117693/T80748.blend) Current layering in the draw code is: 1. parent of parent node tree 2. green overlay 3. parent node tree 4. green overlay 5. grid 6. backdrop 7. active node specific drawing 8. node tree Everything is currently done in display space to an offscreen image. ## Solutions 1. change the drawpix method to retrieve the backdrop in scene reference space using a custom shader to integrate the backdrop and transfer to display space. The complexity here is the transfer to display space as our OCIO wrapper doesn't support this. the alpha channel might be different when compared to the image editor. 2. use a gpu viewport the background is rendered to the overlay buffer (1-5), the backdrop (6) is rendered to the color buffer. During drawing of the backdrop the overlay texture is changed to simulate an alpha under. The rest (7-8) will be rendered on the overlay buffer. When copying the viewport to the display the overlay and color buffers are merged and transform to the display space. This will lead to some artifacts when looking through a transparent canvas of the node, but can be considered as cosmetic. Additional benefit of the GPUViewport solution is that stereoscoptic compositing will also give the same result as in the 3d viewport or image editor as they shade the same color pipeline. @fclem do you have any suggestion?

Clément Foucault commented

2020-10-30 13:04:53 +01:00

The ideal solution would be to do something like the 3D view: At 6, draw the backdrop only using a holdout for it's alpha channel. This would let the render pass through the overlays and display correctly.

So your idea is good and yes the GPUViepwort would handle stereo correctly.

The ideal solution would be to do something like the 3D view: At 6, draw the backdrop only using a holdout for it's alpha channel. This would let the render pass through the overlays and display correctly. So your idea is good and yes the GPUViepwort would handle stereo correctly.

blender-admin commented

2020-11-24 13:18:57 +01:00

This issue was referenced by 278011e44d

This issue was referenced by 278011e44d433dc202b2ab8e7907e323fb23c82d

Jeroen Bakker commented

2020-11-24 13:23:32 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Jeroen Bakker closed this issue

2020-11-24 13:23:32 +01:00

Thomas Dinges added this to the 2.92 milestone 2023-02-08 16:16:03 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Compositor backdrop handles alpha differently from image editor #80748

Technical challenge

Solutions