Page MenuHome

EEVEE: Cryptomatte
Confirmed, NormalPublicDESIGN

Authored By
Jeroen Bakker (jbakker)
Sep 22 2020, 4:41 PM
Tokens
"Love" token, awarded by Draise."Cup of Joe" token, awarded by filibis."Love" token, awarded by KINjO."Mountain of Wealth" token, awarded by Brandon777."Love" token, awarded by 295032."Love" token, awarded by Raimund58.

Description

Research how to get Cryptomatte inside EEVEE

Cycles

This section explains the current implementation in Cycles.

Cryptomatte Layers
Cycles supports 3 different kind of cryptomatte layers:

  • Material
  • Object
  • Asset (in Blender this maps to a collection)

The names of the render passes are Crypto[Object/Material/Asset]{99}

Weights and Hashes

The bsdf transparency is used as weight.
For hashes the name of the object, material or asset is hashed using util_murmur_hash3 and converted to a float using util_hash_to_float.

Coverage

When using Cycles CPU the cryptomatte result is sorted and flattened coverage.cpp. The flattening makes sure that samples that points to the same hash only occupying a single slot. The sorting process sorts the items based on its weight. This could be an optimization so highest coverage are first in the list.

EEVEE

In EEVEE transparency is done as a pre-pass this updates the depth buffer to the objects we want to get the data from. This allows to output a single float per sample/layer as we don't need the weight. Every number of samples we can download the texture to CPU RAM and update the Coverage (number of occurances / number of samples). After the final sample has completed we can store the hashes with the highest coverage in the RenderResult.

Pros:

  • No need to do material evaluations a simple depth test would be sufficient.
  • Needs up to 3 floats per object+material combination
  • We could use gl_Layer in the geometry shader to store the result in a 3d texture
    • Texture depth is (number of render samples * number of layers).
    • When render samples are large, we could download the texture to CPU and start processing it in the background. A different texture will be bound to the GPU so the rendering can continue.
  • Shading groups only need to be setup once and can be reused between the different samples.
  • Feature complete (and compatible) with Cycles.

Cons:

  • Not possible to support Viewport previews of the cryptomatte render pass as it needs CPU processing.
  • Rendering overhead as downloading the textures floods the PCI bus. This can slow down final rendering.
  • Blend Alpha isn't supported.
NOTE: Eventually we didn't use a 3d texture due to the OpenGL Pipeline performance penalty. It was faster to do it in a 2d texture and download the texture for every sample.

Additional

There was a request to access the asset, material and object cryptomatte hash from bpy. When implementing this project it can be one of the first patches. BKE will have a method to find the asset/object and material cryptomatte hash (BKE_cryptomatte). RNA read only properties are added to the object/material to calculate the hashes for the specific object/material.

Revisions and Commits

Event Timeline

Jeroen Bakker (jbakker) changed the subtype of this task from "Report" to "Design".
Jeroen Bakker (jbakker) updated the task description. (Show Details)
Jeroen Bakker (jbakker) changed the task status from Needs Triage to Confirmed.Fri, Nov 13, 8:42 AM
Jeroen Bakker (jbakker) changed the status of subtask T82573: EEVEE: Store Cryptomatte hashes names mapping from Needs Triage to Confirmed.
Jeroen Bakker (jbakker) changed the status of subtask T82576: Cryptomatte: Add RNA to extract the hashes in python scripts from Needs Triage to Confirmed.