Blender applies color management to AOVs, effectively destroying them #83842
Closed
opened 2020-12-16 13:07:16 +01:00 by Ludvik Koutny
·
30 comments
No Branch/Tag Specified
main
manifold
blender-v4.4-release
npr-prototype
blender-v4.2-release
remote-asset-library-monolithic
blender-v3.6-release
blender-v4.3-release
temp-sculpt-dyntopo
blender-v3.3-release
brush-assets-project
pr-extensions-tidy-space
blender-v4.0-release
universal-scene-description
blender-v4.1-release
blender-v3.6-temp_wmoss_animrig_public
gpencil-next
blender-projects-basics
sculpt-blender
asset-browser-frontend-split
asset-shelf
blender-v3.5-release
blender-v2.93-release
sculpt-dev
bevelv2
xr-dev
v4.4.0
v4.2.8
v4.2.7
v3.6.21
v4.2.6
v3.6.20
v4.2.5
v3.6.19
v4.3.2
v4.3.1
v4.3.0
v4.2.4
v3.6.18
v4.2.3
v3.6.17
v4.2.2
v3.6.16
v4.2.1
v3.6.15
v4.2.0
v3.6.14
v3.3.21
v3.6.13
v3.3.20
v3.6.12
v3.3.19
v4.1.1
v3.6.11
v3.3.18
v4.1.0
v3.3.17
v3.6.10
v3.6.9
v3.3.16
v3.6.8
v3.6.7
v3.3.14
v4.0.2
v4.0.1
v4.0.0
v3.6.5
v3.3.12
v3.6.4
v3.6.3
v3.3.11
v3.6.2
v3.3.10
v3.6.1
v3.3.9
v3.6.0
v3.3.8
v3.3.7
v2.93.18
v3.5.1
v3.3.6
v2.93.17
v3.5.0
v2.93.16
v3.3.5
v3.3.4
v2.93.15
v2.93.14
v3.3.3
v2.93.13
v2.93.12
v3.4.1
v3.3.2
v3.4.0
v3.3.1
v2.93.11
v3.3.0
v3.2.2
v2.93.10
v3.2.1
v3.2.0
v2.83.20
v2.93.9
v3.1.2
v3.1.1
v3.1.0
v2.83.19
v2.93.8
v3.0.1
v2.93.7
v3.0.0
v2.93.6
v2.93.5
v2.83.18
v2.93.4
v2.93.3
v2.83.17
v2.93.2
v2.93.1
v2.83.16
v2.93.0
v2.83.15
v2.83.14
v2.83.13
v2.92.0
v2.83.12
v2.91.2
v2.83.10
v2.91.0
v2.83.9
v2.83.8
v2.83.7
v2.90.1
v2.83.6.1
v2.83.6
v2.90.0
v2.83.5
v2.83.4
v2.83.3
v2.83.2
v2.83.1
v2.83
v2.82a
v2.82
v2.81a
v2.81
v2.80
v2.80-rc3
v2.80-rc2
v2.80-rc1
v2.79b
v2.79a
v2.79
v2.79-rc2
v2.79-rc1
v2.78c
v2.78b
v2.78a
v2.78
v2.78-rc2
v2.78-rc1
v2.77a
v2.77
v2.77-rc2
v2.77-rc1
v2.76b
v2.76a
v2.76
v2.76-rc3
v2.76-rc2
v2.76-rc1
v2.75a
v2.75
v2.75-rc2
v2.75-rc1
v2.74
v2.74-rc4
v2.74-rc3
v2.74-rc2
v2.74-rc1
v2.73a
v2.73
v2.73-rc1
v2.72b
2.72b
v2.72a
v2.72
v2.72-rc1
v2.71
v2.71-rc2
v2.71-rc1
v2.70a
v2.70
v2.70-rc2
v2.70-rc
v2.69
v2.68a
v2.68
v2.67b
v2.67a
v2.67
v2.66a
v2.66
v2.65a
v2.65
v2.64a
v2.64
v2.63a
v2.63
v2.61
v2.60a
v2.60
v2.59
v2.58a
v2.58
v2.57b
v2.57a
v2.57
v2.56a
v2.56
v2.55
v2.54
v2.53
v2.52
v2.51
v2.50
v2.49b
v2.49a
v2.49
v2.48a
v2.48
v2.47
v2.46
v2.45
v2.44
v2.43
v2.42a
v2.42
v2.41
v2.40
v2.37a
v2.37
v2.36
v2.35a
v2.35
v2.34
v2.33a
v2.33
v2.32
v2.31a
v2.31
v2.30
v2.28c
v2.28a
v2.28
v2.27
v2.26
v2.25
Labels
Clear labels
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Asset datablocks, libraries, browser and shelf
Interest
Audio
Interest
Automated Testing
Interest
BlendFile
Interest
Blender Asset Bundle
Interest
Code Documentation
Code comments, Python/RNA API descriptions
Interest
Collada
Interest
Compatibility
Backward and forward compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
FBX
FBX I/O related topics
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline & IO
Interest
Platforms & Builds
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
USD
Interest
UV Editing
Interest
Undo
Interest
User Interface
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Wayland windowing on Unix
Interest
Workbench
Interest
glTF
glTF format I/O topics
Interest: X11
Xorg/X11 windowing
Legacy
Asset Browser Project
Archived
Legacy
Blender 2.8 Project
Archived
Legacy
Milestone 1: Basic, Local Asset Browser
Archived
Legacy
OpenGL Error
Archived
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Related to security, see policy: https://developer.blender.org/docs/handbook/bug_reports/vulnerability_reports/
Module
Animation & Rigging
Module
Asset System
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline & IO
Module
Platforms & Builds
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
Windows
Platform
macOS
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Archived
Type
Report
Archived
Type
To Do
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Hoshinova
James-McCarthy-4
Sebastian-Herholz
casey-bianco-davis
gandalf3
Blendify Aaron Carlisle
quantimoney Aditya Y Jeppu
Alaska Alaska
angavrilov Alexander Gavrilov
frogstomp Aleš Jelovčan
amelief Amélie Fondevilla
elubie Andrea Weikert
Andy_Goralczyk Andy Goralczyk
ankitm Ankit Meel
Anthony-Roberts Anthony Roberts
antoniov Antonio Vazquez
aras_p Aras Pranckevicius
Arnd Arnd Marijnissen
bartvdbraak Bart van der Braak
mont29 Bastien Montagne
blender-bot Blender Bot
bnagirniak Bogdan Nagirniak
BClark Brad Clark
brecht Brecht Van Lommel
BrianSavery Brian Savery (AMD)
ideasman42 Campbell Barton
CharlesWardlaw Charles Wardlaw
CharlieJolly Charlie Jolly
Chris_Blackbourn Chris Blackbourn
lateasusual Chris Clyne (Lateasusual)
ChrisLend Christoph Lendenfeld
HobbesOS Cian Jinks
fclem Clément Foucault
cmbasnett Colin Basnett
Kdaf Colin Marmond
dfelinto Dalai Felinto
pioverfour Damien Picard
DanielBystedt Daniel Bystedt
pepe-school-land Daniel Martinez Lara
zanqdo Daniel Salazar
Mets Demeter Dzadik
erik85 Erik Abrahamsson
EAW Evan Wilson
filedescriptor Falk David
fsiddi Francesco Siddi
GaiaClary Gaia Clary
DagerD Georgiy Markelov
mano-wii Germano Cavalcante
zazizizou Habib Gahbiche
HooglyBoogly Hans Goudey
Harley Harley Acheson
weasel Henrik D.
Hjalti Hjalti Hjálmarsson
howardt Howard Trickey
nirved-1 Hristo Gueorguiev
mod_moder Iliya Katushenock
brita Inês Almeida
JacquesLucke Jacques Lucke
Jason-Fielder Jason Fielder
JasonSchleifer Jason schleifer
Jebbly Jeffrey Liu
Jeroen-Bakker Jeroen Bakker
deadpin Jesse Yurkovich
neXyon Joerg Mueller
eliphaz John Kiril Swenson
guitargeek Johnny Matthews
Brainzman Jonas Holzman
JoniMercado Jonatan Mercado
JorgeBernalMartinez Jorge Bernal
JosephEagar Joseph Eagar
JoshuaLeung Joshua Leung
Bone-Studio Juan Gea
jpbouza-4 Juan Pablo Bouza
JulianEisel Julian Eisel
JulienDuroure Julien Duroure
JulienKaspar Julien Kaspar
kevindietrich Kévin Dietrich
lone_noel Leon Schittek
LucianoMunoz Luciano Muñoz Sessarego
LukasStockner Lukas Stockner
LukasTonne Lukas Tönne
LunaRood Luna Rood
MaiLavelle Mai Lavelle
EosFoxx Marion Stalke
Baardaap Martijn Versteegh
scorpion81 Martin Felke
mendio Matias Mendiola
Matt-McLin Matt McLin
MetinSeven Metin Seven
wave Michael B Johnson
Michael-Jones Michael Jones (Apple)
makowalski Michael Kowalski
pragma37 Miguel Pozo
nrupsis Nate Rupsis
jesterking Nathan Letwory
nathanvegdahl Nathan Vegdahl
PrototypeNM1 Nicholas Rishel
nickberckley Nika Kutsniashvili
Sirgienko Nikita Sirgienko
OmarEmaraDev Omar Emara
pablovazquez Pablo Vazquez
PaoloAcampora Paolo Acampora
PascalSchon Pascal Schön
pmoursnv Patrick Mours
muxed-reality Peter Kim
lichtwerk Philipp Oeser
P2design Pierrick PICAUT
PratikPB2123 Pratik Borhade
Limarest Ramil Roosileht
farsthary Raul Fernandez Hernandez
LazyDodo Ray molenkamp
iss Richard Antalik
rjg Robert Guetzkow
salipour Sahar A. Kashi
Sayak-Biswas Sayak Biswas
Sean-Kim Sean Kim
sherholz Sebastian Herholz
sebastian_k Sebastian Koenig
ZedDB Sebastian Parborg
sebbas Sebastián Barschkis
Sergey Sergey Sharybin
IRIEShinsuke Shinsuke Irie
sidd017 Siddhartha Jejurkar
SietseB Sietse Brouwer
SimonThommes Simon Thommes
SonnyCampbell_Unity Sonny Campbell
Stefan_Werner Stefan Werner
Lockal Sv. Lockal
dr.sybren Sybren A. Stüvel
ThomasDinges Thomas Dinges
Ton Ton Roosendaal
BassamKurdali Ursula kurdali
Vasyl-Pidhirskyi Vasyl Pidhirskyi
WannesMalfait Wannes Malfait
wbmoss_dev Wayde Moss
weizhen Weizhen Huang
leesonw William Leeson
xavierh Xavier Hallade
jenkm Yevgeny Makarov
ChengduLittleA YimingWu
gfxcoder jon denning
Clear assignees
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#83842
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce GTX 1080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.79
Blender Version
Broken: version: 2.91.0, branch: master (modified), commit date: 2020-11-25 11:06, hash:
b50598bc78
Worked: probably never
Short description of error

Blender applies color management to AOV output, effectively destroying the data. When user uses non standard view transform, such as filmic, which is in fact default now, and then wants to output the AOVs at once using compositor, so they don't have to be manually saved from framebuffer one by one, the filmic color transform is applied to AOVs saved this way, and they end up being useless.
While regular image save dialog has an option to prevent this, called "Save As Render"

...the compositor File Output node does not have have this option, and defaults to wrong behavior out of the box, as it would likely be more often used for saving AOV passes, rather than color corrected beauty passes.
Only workaround is to disable Blender-wide color management completely, making it in turn useless, because workflow of constantly turning it off and on whenever user wants to output AOVs is just not acceptable.
Exact steps for others to reproduce the error
Result: Texture AOV has different color tones to the original texture due to applied color mapping
Expected: Color mapping is never ever applied to AOVs
Added subscriber: @Rawalanche
Added subscriber: @rjg
@Rawalanche as far as I'm aware the file output node doesn't have any knowledge about the source of the data you're passing into it and thus can't / doesn't treat the input from AOVs differently. You would want to use OpenEXR as output format as this stores the values as is. Besides the view transform being applied on other formats, it's generally not advisable to use PNG for AOVs, even when setting the view transform to RAW, as all values have to be mapped to integer values in the [0, 255] range. This means you will always be losing information / precision in the process when using PNG and won't get the exact same results back as the original AOV when loading the saved image into Blender or another application.
That does not apply when you use AOVs for example to generate texture sets for games, where EXR is (often incompatible and/or harder to edit in image editing software) overkill. PNG is nothing more than a data container, so there is absolutely no excuse for making assumptions of whether or not user wants to store color managed data within it. I guess that when you look at the render layers, and see "BaseColor", "Roughness" and "Normal", you can already tell what I am trying to achieve. I am trying to bypass Blender's unusable texture baking tools using AOVs, but even when trying to output those, Blender throws sticks under my feet.
Let me put it differently. Ask yourself if there is any excuse to not have the "Save As Render" switch present in the File Output compositor node, when it's present in all other places where Blender outputs texture files.
@Rawalanche it's a missing feature as the node currently has no way of knowing that you don't want the view transform to be applied.
Inability for user to prevent certain image formats from being saved with color transform walks a line between missing feature and a bug.
There's a difference between new feature as in user being able to do something they were not able to do before, and new feature as in being able to use existing feature without problems.
Added subscriber: @lichtwerk
Note: even EXR has its issues with multiple layers, see #68239 (Cryptomatte affected by Color Space change in OpenEXR multilayer.)
@lichtwerk From my understanding this is a different issue, as this happens when loading an existing multi-layer image and selecting the color space that is then indiscriminately applied to all layers. The problem in this ticket is just that the file output node has no option to disable the view transform for non-color data (except for when using OpenEXR as output format).
Yep, they are different issues, was just noting that it is always necessary to be able to specify which data is being colormanaged and which is not [but admit I was a bit off-topic here]
Added subscriber: @brecht
Changed status from 'Needs Triage' to: 'Archived'
I've added this issue to the list in #68926 (Color Management Improvements) now.
We recommend using OpenEXR for digital intermediates, which avoids this problem.
Also note that when saving to PNG, color management is in fact needed. It needs to write sRGB per the PNG spec, and because 8bit can not store linear colors without significant data loss. What you want in this case would be an option to skip the artistic choices like Filmic, but still convert linear to sRGB.
As I wrote, my intention for it was to output PBR texture sets. You can not make assumptions about what user wants to use certain image format for. sRGB is something most engines have direct option to remove on load. I mean most normal maps for games come in 8bit PNGs. EXRs are unacceptable for regular game textures, for many reasons. They are harder to edit in average image editing packages, some engines do not even properly support them, and they don't even display preview thumbnail in Windows file explorer, due to how exotic of a format they are, so that managing a texture libraries becomes complete hell.
Of course I want to convert linear to sRGB. Yes, PNG is dependent on that. But exactly as there is a need to write sRGB per the PNG spec, there is also nothing in the PNG specs saying other color transforms should be written too, so doing this directly, without user's choice is actually against PNG spec. Lastly, the "Save As Render" checkbox already does exactly that, it still applies sRGB to PNG, just not the transforms which are meant for view only.
What are you doing is that you are treating both image format essential standardized transform like sRGB and random, artistic, non standardized transform like Filmic (at least Blender's implementation is non standard) as the same thing, and applying them both forcibly, without user's choice. When you load 8bit file, Unreal Engine has a simple checkbox to counter sRGB transform for linear data saved in 8bit image files:

But there's nothing to remove Blender's arbitrary filmic transform.
And saying to use EXR for game image textures to solve this issue is like telling person stuck in a highway traffic they should have used a helicopter instead to avoid the problem.
This is LITERALLY just about adding the already existing "Save As Render" checkbox to the File Output compositor node. So please do not just clean this up into some old Color Management Improvements task where it will rot for several years before the task will eventually be forgotten.
@Rawalanche We recognize that there is a need for this so it has been added to the ToDo task. It will be worked on when developers have time to look into this, just as it would with a bug report. While we understand that you consider this a very important issue that is best solved right away, we still need to organize the development and prioritize tasks.
In the meantime, until this feature is added, I would suggest you set the View Transform to Raw and the file output format to PNG. This should give you the desired result for your intended use case with the AOVs. The current downside is that this applies globally and your render / composite will also be affected by this setting.
Yes, I am aware of the workaround, that's what I am doing, but currently, for this particular use case of baking PBR texture sets, I want to be able to do both outputting of the correct data as well as viewing what I am working on in a color managed environment at the same time. Users should not be forced to make a choice between one of these two things or have to do constant chores of turning color management off and on all over again depending of if they are about to save the output. So that's why this is important.
This issue was referenced by
27b78c9c94
Added subscriber: @MartinZ
This comment was removed by @MartinZ
Feel free to comment if you have a constructive suggestion for a better name and description, developer.blender.org is not the place to rant.
Well, I would gladly do that, but I am afraid it seems I have no way of knowing what it does exactly like most users without deep understanding how Blender's code works and ability to program in C. I am guessing it disables color transforms set in Color Management settings and outputs raw values, is this what it does? In that case why not call it "Disable color transforms" ? Or if it applies other transforms, why not say "Apply x color transforms" where x is those color transforms...
@MartinZ This has been partly discussed in D9921.
This should get addressed in the near future.
Sounds better. Of course "Standard view transform" is just as meaningless, if the standard is not defined. This is another big issue. What is standard? Why not specify the standard just there? Is it sRGB IEC61966-2.1? Or is it any other standard? Why hide it?..
@MartinZ The "Standard" in this context refers to the view transform named "Standard" not a specific specification standard. Technically this would depend on the OCIO config used. For Blender without any modifications this would be sRGB as defined by the
srgb.spi1d
lookup table when using an sRGB display (and a different one when using a different color space for display).If we were to make larger changes, like renaming the "Standard" view transform, this should rather be planned and addressed in context of #68926 and not just by a single patch without design task.
The OpenColorIO design is have the display device and view transform as independent choices, so that you can switch display devices and keep other color management settings the same. That means "sRGB" should not be in the view transform name.
That makes sense. As long as it's not "Save as Render" with the current tooltip it's way better. I think "Standard" might not be the best choice for the name and it would make more sense to just call it Display Device Transform directly if it is, but what do I know. Even if it is a bit confussing and named "Standard" at least it makes a whole lot more sense than what it is right now.
Added subscriber: @EAW
Just a note: OCIO is starting a UX working group to
Hopefully it will help users of multiple programs to have a more consistent CM user experience.
From my experience, the output without any color transform is generally referred to in the industry simply as "Linear", so that's what I would call it. "Save as Render" label indeed makes absolutely no sense. Render just means rendered picture, and rendered picture can be color managed or not, depending on how the user intends to use it.