Page MenuHome

Cryptomatte - File Output node is messing up the multilayer exr (metadata), so blender cryptomattes cannot be read properly in external compositors
Confirmed, LowPublicTO DO

Description

System Information
Operating system: Windows-10-10.0.17134 64 Bits
Graphics card: GeForce GTX 1080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 419.35

Blender Version
Broken: version: 2.80 (sub 71), branch: blender2.7, commit date: 2019-05-20 22:03, hash: rBe78770039397
Worked: (optional)

Short description of error
Cryptomatte - File Output node is messing up the multilayer exr, so blender cryptomattes are not working in the compositing app Blackmagic Fusion 9.

Exact steps for others to reproduce the error
Open the attached blend-file.
Hit render animation.
Two files are generated within the tmp folder.
One from the compositor file output node (Cubes_Cryptomatte_test_FileOutput_0001.exr)


and one from the core output in the output properties (Cryptomatte_Cubes_test_0001.exr).

As you can see on the filesize of these multilayer exrs, they are not the same, but should?!
If you try to use the cryptomatte channels in these files in the free compositing app Blackmagic Fusion 9 only the cryptomattes frome the one, which wasn´t save though the file output node, works correct.

Is the file output node in the blender compositor changing something in the multilayer exr?


In Fusion Cryptomatte_Cubes_test_0001.exr - The Channel IDs are recognized by Fusion.

In Fusion Cryptomatte_Cubes_test_FileOutput_0001.exr - The Channel IDs are not recognized by Fusion.

The error message whithin Fusion.

Event Timeline

fusion cryptomatte plugin is trash generally... i had problems with cryptomatte from clarisse and also from houdini with fusion.. if the project is a bit bigger it simply don't work anymore.

Yes maybe the plugin in trash but I think the fileoutput in blender should be the same, if you use a fileoutput node or not.
Comparing the metadata or the exr-header may show what is different amongst the two outputs.
Otherwise if you have many renderlayers you are not able to render them into different files if you want to use cryptomatte in another compositing app.
Look into this thread https://blenderartists.org/t/cryptomatte-is-here-amazing/1132013/
People have problems using cryptomattes from blender cycles in all the other compositing apps (Nuke, After Effects, Fusion).
Sorry, but the blender compositor is no alternative if you want to make highend compositing.
Today I testest cryptomattes from blender in After Effects. With the free ProEXR 2.0 Plugin it works fine but I need a node base compositing software.
I also tested Natron. The cryptomatte implementation in Natron works, but if you use aspecific nodes like glow together with the cryptomatte keyer, it crashes.
There are sample files from the cryptomatte inventor Psyop, which you can download from their githup page together with the cryptomatte plugins for nuke and fusion and they work fine.
So something is wrong with the blender cryptomatte implementation, maybe not following the standard as thought by Psyop?

yeah i agree that meta data shuld have VFX standard and be same in both case. However i don't agree about compositings apps in general.. For me, the best is Nuke.. after that is Blender and very last is Fusion, that on big projects just chokes and generally performance is bad and anoying. it is mostly designed to work with TV stuff. not CGI. However it do have neat color gradeing tools.

Nuke maybe the best, its industry standard I know and I would love to work with it, but my company can´t afford to pay nearly 4000€ per license.
Fusion 9 is free and Fusion 16 Studio is only 320€.

Nuke is worth the money. Working in fusion is slow as hell, i wouldn't be suprsied if nuke artist would do job 2 times faster and better.

Please keep on topic. Cluttering this site with non related information will not make us solve issues faster.

Not sure, but might be related to T63270?

My problem is as mentioned in the description, that if I use a file output node in blender the cryptmatte informations in the multilayer exr change, so they can´t be read by the compositing app Fusion.
If I render without an output node, everything works fine in Fusion.
So my assumption is that the file output node changes IDs or metadata?!

Jeroen Bakker (jbakker) lowered the priority of this task from 90 to Low.May 23 2019, 3:49 PM

Crypto matte meta data related to the naming of the layers. The compositor does not support meta data of render layers, or any meta data at all.
We should come up with a good design if we want to support using blender compositor for this.

Will put this as a TODO as it is currenlty not designed or implemented to support this.

If you have viewlayers, the file output node is the only way, to render them to different multilayer exr´s.
Otherwise every viewlayer and pass ends up in one multilayer exr. If you have to rerender one viewlayer, you are produceing new content not able to delete old renderings wihtin the multilayer exr.
In Blender it is then also not possible, to write only cryptomatte to a seperate exr-file, because without the file output node the combined pass is always written to file.
For me the FileOutput Node is very important when rendering viewlayer! Otherwise you have to render every viewlayer after each other, setting new output settings after each rendering. Thats not very efficient!
Is it not possible to make the fileoutput node the same behaviour than the regular output?

I second this. I'm facing this problem in a project we're currently working on and again I have to find a workaround to solve something that doesn't work as it should in Blender.

Same problem on our side. This behavior of Metadata not being attached correctly to OpenEXR does not only affect File Output nodes it seems. Saving from the Image Editor to OpenEXR (MultiLayer) seems to show the same problem. Is the Metadata attached at all? Can someone confirm this?

FYI we are using Nuke for compositing, so the app on the other side doesn't seem to matter.

If you want proper cryptomate IDs you can´t use the file output node. The cryptomates within images saved from the Image Editor work in Fusion 9/16 and in Photoshop (Exr-IO-Plugin). I tested it a few minutes ago. Sorry I can´t test it on Nuke.
Maybe the problem with the file output node will be adressed some day, because so you are forced to render every viewlayer in one multilayer-exr if you want to use cryptomattes, but I have given up hope!

The developer of the exr io plugin has said that the issue is that blender is outputting the cryptomattes as images instead of data. Should be a really quick fix if this can get some attention from a developer.

https://devtalk.blender.org/t/cryptomatte-in-photoshop-issue-exr-io-plugin/8286/2

From my point of view cryptomatte in Blender is working fine.
The problem in the moment is the file output node, that is not able to store them in the correct way like the main output does.

yeah, and it's a real pain the file output node doesnt have a save button too. Otherwise you could set up the masks in blender's compositor using the cryptomatte node, plug the mask output into a file node, and then press save on the file output node.

oh actually, just realised I can do this by plugging the mask output into a viewer node and saving it from the image editor.

Workaround to the file output node problem is to set up the multilayer exr in the properties panel, manually save the blender compositor output using the image editor. Then in the 3rd party software load in the exr from the properties panel, and load in the composited image saved from the image editor. Then you can use the properties cryptomatte to make further adjustments to the file blender composited image externally.

I am frustrated. I try to integrate Blender into my work process. The inteldenoiser is wonderful. Cryptomatte is great. But I don't get both in a MultiLayer EXR file. The Denoiser can only be reached via compositing. And a functioning cryptomatte can only be obtained from the file output.
I eat my hair XD

See https://devtalk.blender.org/t/cryptomatte-metadata-missing-in-2-8/5546/33
Brecht writes: "The Blender compositor indeed does not preserve metadata with the file output node, that’s a limitation currently, there is no type of metadata node socket for passing along that information from one node to another."

Up on this topic, I have the same issues here, and it's all about metadata. T72673

The file output node is pretty important when we are using "classic professionnal" software in compositing (as @Marcus Papathoma (machieb) said, it's the only simple way to render only one view layer at time ...).

No promess, but we may have someone in our team who would look at it ...

Philipp Oeser (lichtwerk) renamed this task from Cryptomatte - File Output node is messing up the multilayer exr, so blender cryptomattes are not working in the compositing app Blackmagic Fusion 9. to Cryptomatte - File Output node is messing up the multilayer exr (metadata), so blender cryptomattes cannot be read properly in external compositors.Jan 16 2020, 11:24 AM

@Jeroen Bakker (jbakker), @Sergey Sharybin (sergey): not sure this should see some love in T69190: VFX & Video Module? Seems to be pretty bad for a bigger pipeline?

perhaps each input should silently receive the metadata if present. I'm guessing it's ouput from the cryptomatte output in order for it to work in the compositor (unless the cryptomatte node is accessing it from a known location rather than it's inputs.

Semyon (Semyon) added a comment.EditedJan 16 2020, 9:52 PM

sorry, im not specialist - you find solution this trouble? How can in animation now make cryptomatte export?

Having the same issue here in a studio pipeline.

Having the same issue here in a studio pipeline.

use the output section instead of the file output node, works in external softwares then. File output node is a bit useless.

you'll need to render it as an animation though, so set the frame range to just one frame.

Having the same issue here in a studio pipeline.

use the output section instead of the file output node, works in external softwares then. File output node is a bit useless.

Sorry, didnt understand how did it. I need to up Cryptomatte, set setting on scene-output. Enough? But in Nuke I dont see any information. Settings in screeshot. Look right? File exr attached too

I just did a render from the Output section with Cryptomatte activated and I get the same Fusion fail lacking the manifest metadata as the poster above.

Yesterday on a different scene I did successfully get a Cryptomatte pass to work. No idea what is different.

Update the cryptomatte plugin for nuke/fusion I think is the answer:

https://github.com/Psyop/Cryptomatte

Update the cryptomatte plugin for nuke/fusion I think is the answer:
https://github.com/Psyop/Cryptomatte

This could solve a part of the problem because indeed the Nuke/fusion could be well improved, but the main issue here is the fact that there are no metadata at all in the openexr files out of file nodes output, and those are very usefull in complexe compositing

Having the same issue here in a studio pipeline.

use the output section instead of the file output node, works in external softwares then. File output node is a bit useless.

Unfortunatelly, we actually can't use the main blender output with openexr multilayer, because all renderlayers are merged into one and only one file, which means that if you separated characters and background into 2 different layers, if you want to render one layer, you have to render both at the same time, which have a heavy cost, and which can't be done often, in solo project either in production.

Actually the simpliest way to use the main output is to Split All aovs and layers into separate files, but that means easly dozen different image sequences per shot..

You could use the 'Use for Rendering' checkbox or 'Render Single Layer' checkbox in the render layer settings to only render an individual layer.

Alternatively (I haven't tried this), buy you could try duplicating the scene as a linked copy, enable one view layer in one scene, and the other in the other scene, then set the compositor up to render both scenes. I'm thinking this might result in 2 exr's because both scenes will have their own output properties.

On a scene I'm currently trying to debug the Cryptomatte EXRs aren't working even if the render is coming from the Output pane. The Cryptomatte does work in the Blender compositor, but in Fusion I get the error about the missing manifest.

On a scene I'm currently trying to debug the Cryptomatte EXRs aren't working even if the render is coming from the Output pane. The Cryptomatte does work in the Blender compositor, but in Fusion I get the error about the missing manifest.

That's what I was talking about : blender's cryptomatte reader is working only with visual values (which is simple but effective) whereas in Nuke and fusion, the cryptomatte plugin need a manifesto which is writtent in the metadatas of the exr file.

In blender, by using the main output, those metadatas are written in the exr, in the file output, they are not.

Right. In my case though, even out of the Main Output it's still missing the manifest.

Updated to the latest version of Cryptomatte 1.2.5 and still no luck over here getting a Cryptomatte out of Blender. Still says it's missing the manifest.