The values of the diffuse direct and indirect passes are pre-multiplied with the diffuse colour making it impossible to swap textures in post. #100056

New Issue

michael campbell · 2022-07-29T11:13:33+02:00

michael campbell commented

2022-07-29 11:13:33 +02:00

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96

Blender Version
Broken: version: 3.3.0 Alpha, branch: master, commit date: 2022-06-27 22:46, hash: 7a44f62bdb
Worked: (newest version of Blender that worked as expected)

Short description of error
The values of the diffuse direct and indirect passes are pre-multiplied with the diffuse colour making it impossible to swap textures in post.

As in other render engines, Cycles has separate diffuse lighting (direct/indirect) and diffuse colour passes. Unlike other engines however, cycles is altering the value of the lighting passes based on the colour of the texture. For example if the texture has black elements, then the diffuse direct and indirect passes are darkened in the same areas. In other engines the lighting passes are not affected at all by the value of the underlying texture, because this makes it impossible to perform the most basic of compositing tasks. The RS manual has a great explanation as to why this is important and how the image is recombined from the passes (basically the same as cycles regarding recombining of the raw (rs version of direct/indirect) and filter (rs version of colour pass). Check the integrated aov section of their manual, in particular the raw/filter workflow to see how the passes need to look.

To see how serious of an issue this is, open up the below blend file, which demonstrate how even the most common compositing task is unachievable as a result of the contamination. Firstly head to the compositor, and hit render. The node tree replaces the diffuse colour with a new texture, but because the texture data has affected the lighting passes, the brick material on the right hand side monkey is still evident. The material on the other monkeys doesn't have a texture connected to the base colour of the principled BSDF, so this works correctly.

An example of a situation in which this is important: You've just rendered a 3 week animation and the clients asks you to change the marble texture that appears throughout. Without the ability to swap out the texture in post, it's necessary to re-render the entire animation again. Even if using collections to mask out all but the floor, it's still a major nightmare.

diffuse passes bug.blend

**System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96 **Blender Version** Broken: version: 3.3.0 Alpha, branch: master, commit date: 2022-06-27 22:46, hash: `7a44f62bdb` Worked: (newest version of Blender that worked as expected) **Short description of error** The values of the diffuse direct and indirect passes are pre-multiplied with the diffuse colour making it impossible to swap textures in post. As in other render engines, Cycles has separate diffuse lighting (direct/indirect) and diffuse colour passes. Unlike other engines however, cycles is altering the value of the lighting passes based on the colour of the texture. For example if the texture has black elements, then the diffuse direct and indirect passes are darkened in the same areas. In other engines the lighting passes are not affected at all by the value of the underlying texture, because this makes it impossible to perform the most basic of compositing tasks. The RS manual has a great explanation as to why this is important and how the image is recombined from the passes (basically the same as cycles regarding recombining of the raw (rs version of direct/indirect) and filter (rs version of colour pass). Check the integrated aov section of their manual, in particular the raw/filter workflow to see how the passes need to look. To see how serious of an issue this is, open up the below blend file, which demonstrate how even the most common compositing task is unachievable as a result of the contamination. Firstly head to the compositor, and hit render. The node tree replaces the diffuse colour with a new texture, but because the texture data has affected the lighting passes, the brick material on the right hand side monkey is still evident. The material on the other monkeys doesn't have a texture connected to the base colour of the principled BSDF, so this works correctly. An example of a situation in which this is important: You've just rendered a 3 week animation and the clients asks you to change the marble texture that appears throughout. Without the ability to swap out the texture in post, it's necessary to re-render the entire animation again. Even if using collections to mask out all but the floor, it's still a major nightmare. [diffuse passes bug.blend](https://archive.blender.org/developer/F13325069/diffuse_passes_bug.blend)

michael campbell commented

2022-07-29 11:13:33 +02:00

Added subscriber: @3di

Brecht Van Lommel commented

2022-07-29 14:04:42 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2022-07-29 14:04:42 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Brecht Van Lommel closed this issue

2022-07-29 14:04:42 +02:00

Brecht Van Lommel commented

2022-07-29 14:04:42 +02:00

Supporting texture replacement is not a goal of the render passes implementation. The design choice was to optimize render times as much as possible, which means not computing lighting for cases where the texture is black.

michael campbell commented

2022-07-29 18:13:04 +02:00

Thanks. Might it be possible to make this optimisation optional? For large animations rendering a bit slower is a good trade-off if it means you can potentially avoid rendering twice.

michael campbell commented

2022-07-29 18:46:36 +02:00

Even Alex Honnold uses ropes when he thinks it might get windy.

Brecht Van Lommel commented

2022-07-29 18:53:07 +02:00

I don't think it's practical, it probably requires deep compositing and conflicts with OSL design of optimizing out zero weight closures.

It's not even clear to me that Cycles is unlike other renderers here. From the Redshift examples I'm guessing they do the same as Cycles, you can see the texture color affect the lighting pass near edges and DoF where it's difficult to cleanly separate them. And at least when I worked on Arnold it was no different either.

I don't think it's practical, it probably requires deep compositing and conflicts with OSL design of optimizing out zero weight closures. It's not even clear to me that Cycles is unlike other renderers here. From the Redshift examples I'm guessing they do the same as Cycles, you can see the texture color affect the lighting pass near edges and DoF where it's difficult to cleanly separate them. And at least when I worked on Arnold it was no different either.

michael campbell commented

2022-07-29 20:11:10 +02:00

Thanks. This image explains better (pay attention to how the diffuse raw pass is purely lighting , the textures data is limited only to the diffuse filter pass): https://docs.redshift3d.com/display/RSDOCS/Integrated+AOVs#IntegratedAOVs-DiffuseLighting:~:text=set%20to%200-,Raw,-Shading%20Elements

The important part is:

The diffuse filter AOV in conjunction with 'raw' lighting AOVs allows the user to tweak the lighting results separately from each material's diffuse color.

I get it's cool to speed up rendering by using the value of the textures to act as clamping though 👍

Thanks. This image explains better (pay attention to how the diffuse raw pass is purely lighting , the textures data is limited only to the diffuse filter pass): https://docs.redshift3d.com/display/RSDOCS/Integrated+AOVs#IntegratedAOVs-DiffuseLighting:~:text=set%20to%200-,Raw,-Shading%20Elements The important part is: > The diffuse filter AOV in conjunction with 'raw' lighting AOVs allows the user to tweak the lighting results separately from each material's diffuse color. I get it's cool to speed up rendering by using the value of the textures to act as clamping though 👍

Brecht Van Lommel commented

2022-07-29 20:51:06 +02:00

That sentence doesn't say anything about black elements in the texture, I guess you are assuming they can be replaced in other renderers without having actually verified if that's true.

Brecht Van Lommel commented

2022-07-29 20:51:51 +02:00

Either way, this behavior is very unlikely to change in Cycles, so not really worth discussing.

michael campbell commented

2022-07-29 21:09:36 +02:00

The image shows that the black areas of the textures aren't affecting the diffuse lighting passes. I have verified it's true, I wrote a multipass denoiser for redshift in houdini a few years ago.

No worries though, thanks for taking the time to respond.

The image shows that the black areas of the textures aren't affecting the diffuse lighting passes. I have verified it's true, I wrote a multipass denoiser for redshift in houdini a few years ago. No worries though, thanks for taking the time to respond.

Brecht Van Lommel commented

2022-07-30 01:12:41 +02:00

Ok, maybe the example from that documentation was not ideal since it seemingly has no black pixels in the diffuse filter AOV (besides the background), only near black. Or perhaps the hack is to add a tiny bit off diffuse to everything even when the texture is black.

michael campbell commented

2022-07-30 01:20:55 +02:00

Oh, are you saying that only completely black pixels in the colour pass lead to contamination of the direct and indirect lighting passes?

michael campbell commented

2022-07-30 01:24:49 +02:00

just checked and non blacks are also causing contamination:

On close inspection, it actually appears to be introducing noise to just the areas of the lighting passes where the colour data is present. The rest of the pass is completely clean.

just checked and non blacks are also causing contamination: ![image.png](https://archive.blender.org/developer/F13326009/image.png) On close inspection, it actually appears to be introducing noise to just the areas of the lighting passes where the colour data is present. The rest of the pass is completely clean.

michael campbell commented

2022-07-30 01:40:39 +02:00

even when the darkest brick texture component is set to a dark gray, it's still introducing the contamination:

even when the darkest brick texture component is set to a dark gray, it's still introducing the contamination: ![image.png](https://archive.blender.org/developer/F13326021/image.png)

Brecht Van Lommel commented

2022-07-30 02:31:38 +02:00

You can see the same effect in Redshift (but more subtle), the darker areas of the texture have more noisy lighting. That's the result of importance sampling and Russian roulette allocating fewer samples to such dark areas.

michael campbell commented

2022-07-30 02:40:15 +02:00

Naa, that's because it's a transmissive surface so is getting less diffuse samples. Check the black pip on the lemon, and also the black parts of the wood texture. Not even the slightest introduction of texture colour/value or noise on the light passes.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

The values of the diffuse direct and indirect passes are pre-multiplied with the diffuse colour making it impossible to swap textures in post. #100056