Regression: Cryptomatte doesn't show all objects available #101601

New Issue

Loïc DAUTRY · 2022-10-04T22:21:31+02:00

Loïc DAUTRY commented

2022-10-04 22:21:31 +02:00

System Information
Operating system: Windows-10-10.0.22000-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.15

Blender Version
Broken: version: 3.3.0, branch: master, commit date: 2022-09-06 15:39, hash: 0759f671ce
Also tried 3.2.0 stable and 3.3.1 Release Candidate
Worked: 2.93.10 LTS (both legacy cryptomatte and new cryptomatte),

Short description of error
Cryptomatte is missing objects.

Expected (2.93.10 LTS):

Current (3.3.0):

It seems even worse with the Material output:

Note that this test scene is just the default cube duplicated around and materials duplicated too.

Exact steps for others to reproduce the error
cryptomatte_test.blend

With the test scene: hit the render button, and play with the different cryptomatte options, or use the legacy node. None of them show all four objects correctly on broken versions.
The issue seems identical with using a multilayer EXR rendered and saved from a working version like 2.93.10 LTS. So I guess the issue isn't the render nor data provided but either the cryptomatte itself or the data "reading" ?

Caused by 9dbea1db66

**System Information** Operating system: Windows-10-10.0.22000-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.15 **Blender Version** Broken: version: 3.3.0, branch: master, commit date: 2022-09-06 15:39, hash: `0759f671ce` Also tried 3.2.0 stable and 3.3.1 Release Candidate Worked: 2.93.10 LTS (both legacy cryptomatte and new cryptomatte), **Short description of error** Cryptomatte is missing objects. Expected (2.93.10 LTS): ![image.png](https://archive.blender.org/developer/F13614949/image.png) Current (3.3.0): ![image.png](https://archive.blender.org/developer/F13614952/image.png) It seems even worse with the Material output: ![image.png](https://archive.blender.org/developer/F13614956/image.png) Note that this test scene is just the default cube duplicated around and materials duplicated too. **Exact steps for others to reproduce the error** [cryptomatte_test.blend](https://archive.blender.org/developer/F13614961/cryptomatte_test.blend) With the test scene: hit the render button, and play with the different cryptomatte options, or use the legacy node. None of them show all four objects correctly on broken versions. The issue seems identical with using a multilayer EXR rendered and saved from a working version like 2.93.10 LTS. So I guess the issue isn't the render nor data provided but either the cryptomatte itself or the data "reading" ? Caused by 9dbea1db66

Loïc DAUTRY commented

2022-10-04 22:21:31 +02:00

Added subscriber: @L0Lock

Alaska commented

2022-10-05 00:21:05 +02:00

Added subscribers: @brecht, @Alaska

Alaska commented

2022-10-05 00:21:05 +02:00

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Alaska commented

2022-10-05 00:21:05 +02:00

For reference, the Cryptomatte does have all objects, it's just that one of the objects has an alpha of 0, making it invisible.

If you use this compositor setup, you will see the object again.

However, having cryptomatte objects that are invisible is annoying to work with. @brecht, do you think this is worth investigating? Or should it be left "as is" since the information is technically still there.

For reference, the Cryptomatte does have all objects, it's just that one of the objects has an alpha of 0, making it invisible. If you use this compositor setup, you will see the object again. ![Make invisible Cryptomatte objects visible.png](https://archive.blender.org/developer/F13615424/Make_invisible_Cryptomatte_objects_visible.png) However, having cryptomatte objects that are invisible is annoying to work with. @brecht, do you think this is worth investigating? Or should it be left "as is" since the information is technically still there.

Loïc DAUTRY commented

2022-10-05 00:36:25 +02:00

I didn't set this up in the shader and I didn't touch the object's display properties either, it's literally the default cube and default material duplicated three times. And all the cubes are present in the final render, and the alpha channel itself is fully white.
I'm not an expert in the subject, but that doesn't sound like a normal behavior to me.

I didn't set this up in the shader and I didn't touch the object's display properties either, it's literally the default cube and default material duplicated three times. And all the cubes are present in the final render, and the alpha channel itself is fully white. I'm not an expert in the subject, but that doesn't sound like a normal behavior to me.

Brecht Van Lommel commented

2022-10-05 18:48:45 +02:00

Changed status from 'Needs Developer To Reproduce' to: 'Needs Triage'

Brecht Van Lommel commented

2022-10-05 18:48:45 +02:00

I don't think the Pick output should have zero alpha where there are objects, it should work for picking objects without additional nodes.

Someone will need to bisect to find the cause of this change.

I don't think the Pick output should have zero alpha where there are objects, it should work for picking objects without additional nodes. Someone will need to bisect to find the cause of this change.

Alaska commented

2022-10-06 02:25:56 +02:00

I just thought I would document some quick findings from testing.

Somewhere between version 2.91 and 2.92, Cyrptomatte was expanded to make use of the alpha channel. Ever since that change, the cryptomatte output for this .blend file has been consistent for all newer versions of Blender.
So, the cube on the right when using Cryptomatte object has had an alpha of 0 ever since Blender 2.92.

The reason it looks different in Blender 3.2 and above is due to this bug: #98575 (Issues with HDRIs and render result pixel values that exceed half float limits)
Because the cube has an extremely high brightness (which turns black due to this bug), and thus turns invisible due to how Blender displays coloured 0 alpha pixels.

I just thought I would document some quick findings from testing. Somewhere between version 2.91 and 2.92, Cyrptomatte was expanded to make use of the alpha channel. Ever since that change, the cryptomatte output for this .blend file has been consistent for all newer versions of Blender. So, the cube on the right when using Cryptomatte object has had an alpha of 0 ever since Blender 2.92. The reason it looks different in Blender 3.2 and above is due to this bug: #98575 (Issues with HDRIs and render result pixel values that exceed half float limits) Because the cube has an extremely high brightness (which turns black due to this bug), and thus turns invisible due to how Blender displays coloured 0 alpha pixels.

Alaska commented

2022-10-06 06:11:02 +02:00

Upon further investigation, the culprit commit is 9dbea1db66. I've provided a "simple" fix here: D16165
But we're still going to run into issues where some objects are invisible due to the issue reported in #98575 turning the culprit objects black.

Upon further investigation, the culprit commit is 9dbea1db66. I've provided a "simple" fix here: [D16165](https://archive.blender.org/developer/D16165) But we're still going to run into issues where some objects are invisible due to the issue reported in #98575 turning the culprit objects black.

Brecht Van Lommel commented

2022-10-06 19:18:40 +02:00

Added subscriber: @Jeroen-Bakker

Brecht Van Lommel commented

2022-10-06 19:18:40 +02:00

CC @jbakker.

Richard Antalik changed title from ~~Cryptomatte doesn't show all objects available~~ to Regression: Cryptomatte doesn't show all objects available

2022-10-10 19:17:15 +02:00

Richard Antalik commented

2022-10-10 19:17:15 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Jeroen Bakker self-assigned this 2022-10-10 20:13:38 +02:00

blender-admin commented

2022-10-11 16:59:12 +02:00

This issue was referenced by 0048820df8

This issue was referenced by 0048820df8ee605d861a2dbc7f05823723dd5b36

Jeroen Bakker commented

2022-10-11 17:01:43 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Jeroen Bakker closed this issue

2022-10-11 17:01:43 +02:00

Jeroen Bakker commented

2022-10-11 17:08:30 +02:00

Changed status from 'Resolved' to: 'Confirmed'

Jeroen Bakker reopened this issue

2022-10-11 17:08:30 +02:00

Jeroen Bakker commented

2022-10-11 17:08:52 +02:00

Seems not to be working as expected.

Evan Wilson commented

2022-10-11 17:27:30 +02:00

Added subscriber: @EAW

Evan Wilson commented

2022-10-11 17:27:30 +02:00

In #101601#1428085, @brecht wrote:
I don't think the Pick output should have zero alpha where there are objects, it should work for picking objects without additional nodes.

@brecht To clarify, the pick is working fine. The name of the object appears when hovering over it, even when invisible, and is entered into the node when you click with picker.

@Alaska All the blocks have zero alpha: if that was the issue then none of the blocks would appear. The compositor is coded with the assumption that all images have associated alpha.

This is a color management issue. Setting the view transform to "Standard" or Raw" produces results identical to versions 3.0 or earlier.

@Jeroen-Bakker I think it is the fact that you set the OCIO View Transform GPU code generator to set 1e32 equal to infinity. The hash of the "invisible" block is around 1e35.

> In #101601#1428085, @brecht wrote: > I don't think the Pick output should have zero alpha where there are objects, it should work for picking objects without additional nodes. @brecht To clarify, the pick is working fine. The name of the object appears when hovering over it, even when invisible, and is entered into the node when you click with picker. @Alaska All the blocks have zero alpha: if that was the issue then none of the blocks would appear. The compositor is coded with the assumption that all images have associated alpha. This is a color management issue. Setting the view transform to "Standard" or Raw" produces results identical to versions 3.0 or earlier. ![Standard View Transform.png](https://archive.blender.org/developer/F13659146/Standard_View_Transform.png) @Jeroen-Bakker I think it is the fact that you set the OCIO View Transform GPU code generator to set 1e32 equal to infinity. The hash of the "invisible" block is around 1e35.

Jeroen Bakker commented

2022-10-11 17:30:36 +02:00

Yes, came to the same conclusion that OCIO view transform is rendering it black. Clamping shows Cube.003, but that would might break cryptomatte in some cases.
We could clamp it within the f16 range, so CM would not break.

Yes, came to the same conclusion that OCIO view transform is rendering it black. Clamping shows Cube.003, but that would might break cryptomatte in some cases. We could clamp it within the f16 range, so CM would not break.

Evan Wilson commented

2022-10-11 17:54:45 +02:00

@Jeroen-Bakker you might try removing the 1e32 = inf line of code. I am afk so I cannot verify, but I recall that commit having a comment referring to the “current” version of the OCIO library that hadn’t actually been merged at that point, and wondering if you had built the new version of the library on your own, or were possibly mistaken in what version you were using. The bug report in the OCIO tracker was closed with the claim that the fix for Mac GPU code generation was already in that version of the Library. So perhaps that line of code is no longer needed.

Jeroen Bakker commented

2022-10-11 17:58:13 +02:00

diff --git a/intern/opencolorio/gpu_shader_display_transform_frag.glsl b/intern/opencolorio/gpu_shader_display_transform_frag.glsl
index 92cf9b013fd..f4f5d1a5bcb 100644
--- a/intern/opencolorio/gpu_shader_display_transform_frag.glsl
+++ b/intern/opencolorio/gpu_shader_display_transform_frag.glsl
@@ -191,6 +191,7 @@ vec4 OCIO_ProcessColor(vec4 col, vec4 col_overlay)
 void main()
 {
   vec4 col = texture(image_texture, texCoord_interp.st);
+  col = clamp(col, -65535.0, 65535.0);
   vec4 col_overlay = texture(overlay_texture, texCoord_interp.st);
 
   fragColor = OCIO_ProcessColor(col, col_overlay);

This fixes the issue, in the end.
Better option would be to fix this when uploading the data to the GPU, but that might require a copy. Will check tomorrow if we could fix this in the image engine.

``` diff --git a/intern/opencolorio/gpu_shader_display_transform_frag.glsl b/intern/opencolorio/gpu_shader_display_transform_frag.glsl index 92cf9b013fd..f4f5d1a5bcb 100644 --- a/intern/opencolorio/gpu_shader_display_transform_frag.glsl +++ b/intern/opencolorio/gpu_shader_display_transform_frag.glsl @@ -191,6 +191,7 @@ vec4 OCIO_ProcessColor(vec4 col, vec4 col_overlay) void main() { vec4 col = texture(image_texture, texCoord_interp.st); + col = clamp(col, -65535.0, 65535.0); vec4 col_overlay = texture(overlay_texture, texCoord_interp.st); fragColor = OCIO_ProcessColor(col, col_overlay); ``` This fixes the issue, in the end. Better option would be to fix this when uploading the data to the GPU, but that might require a copy. Will check tomorrow if we could fix this in the image engine.

blender-admin commented

2022-10-12 08:47:12 +02:00

This issue was referenced by 0386737097

This issue was referenced by 0386737097716870236d5e480f8ae1ec13e0b7a7

blender-admin commented

2022-10-12 08:47:12 +02:00

This issue was referenced by 872a45f42b

This issue was referenced by 872a45f42bc5a582fac59e9cd24239ee25821bff

Jeroen Bakker commented

2022-10-12 08:56:43 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Jeroen Bakker closed this issue

2022-10-12 08:56:43 +02:00

Thomas Dinges added this to the 3.4 milestone 2023-02-07 19:01:34 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Regression: Cryptomatte doesn't show all objects available #101601