Using the alpha channel of an image texture corrupts RGB Channels in both EEVEE and cycles #95671

New Issue

tempdevnova · 2022-02-10T14:51:35+01:00

tempdevnova commented

2022-02-10 14:51:35 +01:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.07

Blender Version
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557

Short description of error
Using the alpha channel of an image texture affects RGB Channels in both EEVEE and Cycles when connecting the alpha channel to another arbitrary node.

Exact steps for others to reproduce the error

See attached .blend files for Eevee and for Cycles

Blender file that demonstrates that behavior in EEVEE:	The same behavior manifests itself a little differently in Cycles:
Plane_test_file_EEVEE.blend	Plane_test_file_cycles.blend

As it seems connecting the alpha channel somehow unpremultiplies the image.
All images are stored as openexr, the image alpha is therefore set as premultiplied.

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.07 **Blender Version** Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: `f1cca30557` **Short description of error** Using the alpha channel of an image texture affects RGB Channels in both EEVEE and Cycles when connecting the alpha channel to another arbitrary node. ![image.png](https://archive.blender.org/developer/F12860992/image.png) **Exact steps for others to reproduce the error** - See attached .blend files for Eevee and for Cycles |Blender file that demonstrates that behavior in EEVEE:|The same behavior manifests itself a little differently in Cycles: | -- | -- | |[Plane_test_file_EEVEE.blend](https://archive.blender.org/developer/F12860575/Plane_test_file_EEVEE.blend)|[Plane_test_file_cycles.blend](https://archive.blender.org/developer/F12860576/Plane_test_file_cycles.blend) As it seems connecting the alpha channel somehow unpremultiplies the image. All images are stored as openexr, the image alpha is therefore set as premultiplied.

tempdevnova commented

2022-02-10 14:51:35 +01:00

Added subscriber: @tempdevnova

Germano Cavalcante commented

2022-02-10 20:46:13 +01:00

Added subscribers: @fclem, @mano-wii

Germano Cavalcante commented

2022-02-10 20:46:13 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Germano Cavalcante commented

2022-02-10 20:46:13 +01:00

The way the alpha is "pre-multiplied" or otherwise affects the texture is a bit confusing to me, and I've seen related issues reported before. (#69106, #72677, #81474)

Since the other reports are closed and this is clearly still an issue for me, I'm confirming it.

(Maybe this also points to a possible optimization in the detection of unused nodes? Cc. @fclem)

The way the alpha is "pre-multiplied" or otherwise affects the texture is a bit confusing to me, and I've seen related issues reported before. (#69106, #72677, #81474) Since the other reports are closed and this is clearly still an issue for me, I'm confirming it. --- (Maybe this also points to a possible optimization in the detection of unused nodes? Cc. @fclem)

tempdevnova commented

2022-02-11 17:33:35 +01:00

After some extensive research this is what I've found:

https://blenderartists.org/t/textures-and-alpha-modes-oh-my/1161715

I did not test all cases but here is what I can confirm using openexr as file format:

Note: All textures used in the test files are premultiplied.

Cycles:
Alpha set to straight: this one works when alpha output of the same image texture node(otherwise it doesn't work) is used, but doesn't when alpha output isn't used.
Alpha set to premultiplied. unpremultiplies the image when alpha output of the same image texture node(otherwise it doesn't unpremultiply) is used, otherwise works as expected when alpha output isn't used.
Channel packed: Ironically seems to work for all cases.
None: Works for all cases, as alpha is ignored.

So my conclusions for this is:
Blender handles the image texture differently depending on whether the alpha output of the same image texture node is used.
When it isn't used it doesn't do anything meaning everything is fine, but when it is used, Blender for some reason unpremultiplies the image texture.
The big problem with this is, that the image effectly switches state from premultiplied to straight alpha, thereby potentially losing information, as premultiplies alpha can represent colors that straight alpha can't.
This would also explain why using channel packed works correctly, as it per definition leaves the individual channels alone.
This article perfectly explains why we should never use straight alpha in the rendering pipeline:

https://blender.stackexchange.com/questions/46151/why-should-i-never-ever-use-convert-premul-on-cycles-renders

This is further expanded by this article featuring Brecht van Lommel:

https://oiio-dev.openimageio.narkive.com/aaTdld9U/handling-unassociated-alpha#post3

(Under the assumption, that the Brecht in that article is the same Brecht we have at Blender, maybe getting him on board might resolve things quickly?)

Now for 3d rendering programs unpremultiplying an image is especially bad, in fact Blender can't even render straight alpha and is designed to use premultiplied alpha.
Here explained by Ton himself:

https://developer.blender.org/T24764

So, basically I think that removing the strange behaviour of Blender unpremultiplying the image under certain conditions will get rid of this bug, although i do have two questions:
Why change the behavior of nodes depending on connection status anyways?
Are there any other nodes, that behave similarily(also compositor, geometry nodes)?

And by the way:
I also noticed that the image previewer also incorrectly unpremultiplies the image, and when you hover over the image with your mouse it interestingly even unpremultiplies it twice, so there is that.

After some extensive research this is what I've found: https://blenderartists.org/t/textures-and-alpha-modes-oh-my/1161715 I did not test all cases but here is what I can confirm using openexr as file format: Note: All textures used in the test files are premultiplied. Cycles: Alpha set to straight: this one works when alpha output of the same image texture node(otherwise it doesn't work) is used, but doesn't when alpha output isn't used. Alpha set to premultiplied. unpremultiplies the image when alpha output of the same image texture node(otherwise it doesn't unpremultiply) is used, otherwise works as expected when alpha output isn't used. Channel packed: Ironically seems to work for all cases. None: Works for all cases, as alpha is ignored. So my conclusions for this is: Blender handles the image texture differently depending on whether the alpha output of the same image texture node is used. When it isn't used it doesn't do anything meaning everything is fine, but when it is used, Blender for some reason unpremultiplies the image texture. The big problem with this is, that the image effectly switches state from premultiplied to straight alpha, thereby potentially losing information, as premultiplies alpha can represent colors that straight alpha can't. This would also explain why using channel packed works correctly, as it per definition leaves the individual channels alone. This article perfectly explains why we should never use straight alpha in the rendering pipeline: https://blender.stackexchange.com/questions/46151/why-should-i-never-ever-use-convert-premul-on-cycles-renders This is further expanded by this article featuring Brecht van Lommel: https://oiio-dev.openimageio.narkive.com/aaTdld9U/handling-unassociated-alpha#post3 (Under the assumption, that the Brecht in that article is the same Brecht we have at Blender, maybe getting him on board might resolve things quickly?) Now for 3d rendering programs unpremultiplying an image is especially bad, in fact Blender can't even render straight alpha and is designed to use premultiplied alpha. Here explained by Ton himself: https://developer.blender.org/T24764 So, basically I think that removing the strange behaviour of Blender unpremultiplying the image under certain conditions will get rid of this bug, although i do have two questions: Why change the behavior of nodes depending on connection status anyways? Are there any other nodes, that behave similarily(also compositor, geometry nodes)? And by the way: I also noticed that the image previewer also incorrectly unpremultiplies the image, and when you hover over the image with your mouse it interestingly even unpremultiplies it twice, so there is that.

tempdevnova commented

2022-02-11 17:37:42 +01:00

Added subscriber: @brecht

tempdevnova commented

2022-02-11 17:37:42 +01:00

(Under the assumption, that the Brecht in that article is the same Brecht we have at Blender, maybe getting him on board might resolve things quickly?) @brecht

tempdevnova commented

2022-02-11 17:53:30 +01:00

Oh boy now to EEVEE:

So when opening my cycles test file and changing render engine to EEVEE reveals that everything except alpha mode none is broken.

The problem manifests itself a little differently than in cycles. In cycles only when adding or mixing the rgb image with the alpha the unpremultiplication takes place, but in EEVEE it apart from that also does it when the alpha output is just connected to any other node, even if it then doesn't even influence the original rgb image in any way(see first test file)

Oh boy now to EEVEE: So when opening my cycles test file and changing render engine to EEVEE reveals that everything except alpha mode none is broken. The problem manifests itself a little differently than in cycles. In cycles only when adding or mixing the rgb image with the alpha the unpremultiplication takes place, but in EEVEE it apart from that also does it when the alpha output is just connected to any other node, even if it then doesn't even influence the original rgb image in any way(see first test file)

tempdevnova commented

2022-02-16 17:45:52 +01:00

So, is there anyone who might be able to help?

Alexander Kovelenov commented

2022-11-01 09:54:01 +01:00

Added subscriber: @AlexKowel

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Using the alpha channel of an image texture corrupts RGB Channels in both EEVEE and cycles #95671