Page MenuHome

Baking system overhaul: Move baking settings from render settings into BakePasses
Needs ReviewPublic

Authored by Lukas Stockner (lukasstockner97) on May 4 2018, 11:23 PM.
Tokens
"Pterodactyl" token, awarded by Grische."Like" token, awarded by Fracture128."Burninate" token, awarded by Thane5."Love" token, awarded by andruxa696."Love" token, awarded by iWaraxe."Like" token, awarded by belich."100" token, awarded by Frozen_Death_Knight."Love" token, awarded by bnzs."Like" token, awarded by momotron2000."Love" token, awarded by ambient."Love" token, awarded by mino159."Love" token, awarded by fiendish55."Love" token, awarded by Schamph."Love" token, awarded by brilliant_ape."Love" token, awarded by symstract."Yellow Medal" token, awarded by amonpaike."Party Time" token, awarded by madminstrel."Love" token, awarded by kioku."Love" token, awarded by erickblender."Love" token, awarded by TheRedWaxPolice."Love" token, awarded by DotBow."Love" token, awarded by rawalanche."Love" token, awarded by juantxo."Love" token, awarded by jonathanl."Love" token, awarded by wo262."Like" token, awarded by billreynish."Love" token, awarded by LazyDodo.

Details

Summary

The current baking workflow has some issues:

  • The way to specify which image should be used is cumbersome and unintuitive
  • Baking multiple channels has to be done by baking, changing settings and image, baking again etc.
  • When baking two objects sharing a material, the image has to be changed before each baking pass.
  • Bake types are hardcoded, other rendering engines have no way of specifying their own

Therefore, this patch introduces a system that allows to persistently store the combinations of Object, Image, Material and Settings that should be used for baking.

This is handled by the BakePass, a new type that replaces the Bake settings that used to be part of the rendering settings.
Each object holds a list of bake passes, and each bake pass holds an image and the settings that should be used to bake that image.
When running the Bake operator, it iterates over all bake passes of the currently selected object and executes them.

This way, the user can configure which images should be generated from each object and rebake quickly. Especially with the trend of baking PBR maps for e.g. game engines, this workflow is a major use case of baking.

Previously, it was possible to specify a different image for each material. To keep that functionality, the user can optionally select a material, in which case the baking pass will be limited to that material. Then, by adding a pass per image/material-pair, you get the same functionality.

In order to get rid of the hardcoded types, the Blender-side baking code doesn't care about the type anymore - the engine just provides the pixels and some additional info (for example, setting is_normal enables the normal space conversion).

This patch is not really finished yet, here's a list of ToDos:

  • Polish the UI
  • Versioning code - do we want to create a BakePass from the old settings? If yes, some versioning has to be done in Cycles, which means that we need to keep the old data in RNA :/
  • Share renderer session (currently afaics the scene is synced and a Cycles session is started for each pass, it would be better to keep the session for all passes)
  • Setting verification (used to be part of the Blender side, has to be moved into Cycles)
  • Also supporting the new way of specifying images in the case of Multires Baking
  • More Testing

Diff Detail

Repository
rB Blender
Branch
arcpatch-D3203 (branched from master)
Build Status
Buildable 2980
Build 2980: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I didn't look at the code, but from the description this sounds great!

@Lukas Stockner (lukasstockner97) Hey, I hadn't seen this patch before. This sounds really nice.

If you need help with the UI portion, let me know.

Updated to latest master.
Or rather, manually transferred the changes to latest master - turns out that almost a year of 2.8 development is kinda tricky to merge.

The UI code is significantly improved now - only things that actually are Cycles-specific are handled by the Cycles code. Still, on the design level improvements are certainly possible.

I've also updated the ToDos, I hope that Phabricator picks up the change.

Hi,

this is absolutely amazing! Blender's pretty much ideal choice for any degree of content creation for game development, except one pitfall - baking, which you have now decided to tackle. I am eternally grateful.

One thing I am curious about. Aside from lack of proper workflow, which you are addressing, and performance dependent on tile size, which will likely be addressed too, does this patch also enable support to bake things like metalness or roughness texture? You do mention the PBR trend, which is indeed very important, and Blender is already onboard of the bandwagon with the Principled BSDF shader, but the issue right now, aside from the aforementioned ones, is that crucial PBR channels are missing. Does this patch add support for them?

Thanks again for the time you are putting into this! :)

Also, is there any way to get a build, for testing purposes? (Without a need to download all the heavy machinery required to build this patch myself of Windows... ? :) )

I love this! A couple of thoughts:

With nodes being so open ended, you can have different PBR and NPR models other than Principled, or sometimes you want to bake parts of your nodetree, and while having hardcoded bake passes for Principled is nice, an extra more generic solution would be to have a Bake Passes node as a way to support arbitrary channels being baked all at once.

Currently you have to use the Emission pass bake, and if you want to bake another part of your node tree, you have to plug it to Emission shader, and bake again. In your new baking system, you save having to change images per material, which is great, and you save having to bake each pass individually, among the supported passes, but for arbitrary channels, you still have to change things per material and bake again per channel.
Currently I take advantage of the RGB channels and I use a nodegroup to switch the input of the Emission shader on multiple materials at once😂 but it's not enough

I also wonder about baking multiple objects and to lowpoly. Currently "Selected to Activate" limits you to one baking target, the active one, and you lose your pair every time you deselect. Baking should have the ability to specify which collection to bake onto which target object, and to bake multiple pairs of collections+targets at the same time, with options to exclude shadows from other collections or not

I've made some improvements and fixes to the code.

Notable changes:

  • "Selected to Active" was a remaining case where viewport state was used for baking, so I replaced it with an option to specify a collection. When one is provided, shading is baked from its members to the active object.
  • Cycles now shows an option to override the sample count. This is quite important for baking - for example, when baking normals and Combined shading, the shading needs a lot more samples.
  • Added an option for specifying which UV layer should be used for each pass

Smaller changes:

  • The option for the target image now shows a button for creating a new image. The UI alignment is really hacky, I'd appreciate any advice.
  • Baking multiple objects in one go by selecting them is no longer supported. Could be added back if it's really needed.
  • Only allocate pixel_array_high when needed, saves some memory

Fixes:

  • The enable/disable option is no longer ignored
  • Fix filtering by material
  • No longer crash when baking without a selected image
  • Don't hide the type setting for certain types
  • Various cleanups

More TODOs:

  • Hide the baking options/operator when the current engine doesn't support baking (Eevee)
  • When multiple passes are executed, passes shouldn't paint their margin over results of previous passes
  • (Longterm) Revive the old Cycles AOV patch and add support for baking AOVs
  • (Longterm) Combine with D3108
  • (Longterm) Don't sync the entire scene in Cycles when the selected type doesn't need it (e.g. normals)

@Ludvik Koutny (rawalanche):
Roughness should be supported in master already, the others aren't though.
This patch does not add any new types, but with it we can easily add them in the future without touching Blender's code or worrying about bitflags (adding roughness took up the last available bit, so now's the time for an overhaul, similar the the RenderPasses a while back).
I also just pushed the updated version to the experimental buildbot, builds should be up soon at https://builder.blender.org/download/experimental/ (check the date, should be Feb 25th).

@Wo!262 (wo262):
The option to bake any number of custom outputs definitely is something I want to look into in the future, that's what the AOV ToDo in the latest update is about. As far as I'm aware, it shouldn't be too hard.
Also, thanks for the Selected to Active reminder, I've implemented the collection approach now.

@Ludvik Koutny (rawalanche):
Roughness should be supported in master already, the others aren't though.
This patch does not add any new types, but with it we can easily add them in the future without touching Blender's code or worrying about bitflags (adding roughness took up the last available bit, so now's the time for an overhaul, similar the the RenderPasses a while back).
I also just pushed the updated version to the experimental buildbot, builds should be up soon at https://builder.blender.org/download/experimental/ (check the date, should be Feb 25th).
@Wo!262 (wo262):
The option to bake any number of custom outputs definitely is something I want to look into in the future, that's what the AOV ToDo in the latest update is about. As far as I'm aware, it shouldn't be too hard.
Also, thanks for the Selected to Active reminder, I've implemented the collection approach now.

I am a little disappointed that the ability to bake multiple objects at once is gone. This will require a bit more of manual work when baking. You will select object, click bake, wait some minutes, then have to go back, select another one, click again, wait again, etc... The issue with workflows that require lots of manual steps interrupted by long pauses is that almost no one can keep the attention, so you will switch from blender to web browser example, while waiting for bake to start baking another object, and then you forget about it, and waste even more time.

Regarding UI, someone already mentioned it above - How about a concept of "pass through" bake nodes. A node, which you could connect for example between roughness map and roughness input of your Principled shader, and this one would then have file path UI element to specify output path and file name of the given bake texture. Bake button would then parse through all nodes of this type on all selected objects, and would bake whatever is incoming into these bake nodes. At the same time, for rendering itself, these nodes would act only as a pass through (reroute), and would do nothing.

Ok, I did some testing of the experimental build:

Good:
It appears that performance is no longer as much affected by the tile size. The rule that the larger tile size means faster bake still applies. 1024x1024 tiles still bake way faster than 32x32 tiles, but the difference is not nearly as big as it is with official 2.8 branch.

Bad:
In official 2.8, the Cycles samples amount affects the quality of AO maps. The more samples, the cleaner AO map is. In this experimental branch, that is not the case. Amount of Cycles samples has no effect no AO quality whatsoever. This means I have to go through every single AO node in every single material slot of every single object and increase AO samples from 16 to something very impractical for actual rendering, like 128. Then, once bake is done, I have to again go through every single AO node in every single material slot of every single object and decrease AO samples back to 16 so that rendering performance remains reasonable.

EDIT:
Now I am confused. So for example if the AO map is in the Emission shader, then Cycles samples do have effect on AO quality. If the AO map is in the roughness slot of Principled shader, then Cycles samples have no effect on AO quality :|

More improvements:

  • Baking multiple objects at once by selecting them is supported again
  • The progress bar works as expected now
  • The margin filling of a pass doesn't overwrite results of previous passes anymore

Also, some internal changes (e.g. cleaned up IMB_filter_extend, renamed BakeAPIPass to BakeTask).

@Ludvik Koutny (rawalanche) Good point about baking multiple objects - to be honest, I'm not really sure why I just removed it in the first place, it was broken in the previous version of the patch but fixing it wasn't hard.

Regarding the arbitrary output - I don"t really like the idea of having baking settings inside a shader node. My idea was to have a separate output node in which you can set a name and connect stuff to it. Then, the workflow would be to add such nodes for everything you want to output and to add a BakePass for each one. The neat part about that is that it works with normal rendering as well - you could add render passes which contain that information from each object whose shader contains the matching node.

Regarding tile size and AO samples - are you sure that these are affected by this patch? This only changes stuff on the Blender side, I don't see how it could impact either of those things...

@Ludvik Koutny (rawalanche) Good point about baking multiple objects - to be honest, I'm not really sure why I just removed it in the first place, it was broken in the previous version of the patch but fixing it wasn't hard.
Regarding the arbitrary output - I don"t really like the idea of having baking settings inside a shader node. My idea was to have a separate output node in which you can set a name and connect stuff to it. Then, the workflow would be to add such nodes for everything you want to output and to add a BakePass for each one. The neat part about that is that it works with normal rendering as well - you could add render passes which contain that information from each object whose shader contains the matching node.
Regarding tile size and AO samples - are you sure that these are affected by this patch? This only changes stuff on the Blender side, I don't see how it could impact either of those things...

Hey, yes, you are right, regarding AO samples, your patch behaves exactly the same as master. It was my mistake. Regardless, it would be really nice if AO map sampling could behave consistently, so that the render samples in baking process affect AO quality. Having to manually bump AO samples in every material on every object and then manually return them to original values would be quite painful workflow. So if it's something you could "patch in" as well, that would help tremendously :)

Regarding the nodes. I did not expect the nodes to actually contain all the baking settings. Just output path and the image size. But I do realize it's still probably way too far from your idea of how baking should be handled.

I finally had time to look at this more. A few notes:

  • Cycles vs Eevee: Right now this also shows when Eevee is enabled, but I believe Eevee is not supported?
  • The baking now exists inside the Object Properties, which generally seems confusing and quite messy.
    • It's the first thing you see now when you open Blender
    • Avoiding this could be tricky. I suppose we could make it only appear if you enable Baking from inside the Render Properties, or something like that.
    • At the very least, the parent panel could be closed by default and renamed to just 'Bake'
  • The layout could use some work
    • The Bake operator button appears inside the bake pass, but I assume it bakes all passes? The relationship between these items is somewhat confusing
    • The output image is not in the Output sub-panel
    • Generally we could organize subsections in Input, Method and Output to follow the logical flow here.
    • Why is Use Cage dependant on Bake From Collection? They seem unrelated to me.
    • We should use property split for the Image, Material and UV Map fields so the alignment is correct.

Update to latest master (and therefore clang-format), no functional changes.

Another rebase to get to the newwest master, also squashed to a single commit now.

Rebase to latest master and two crash fixes.

could someone look into this T53498 it would be useful if cycles used the target meshes custom normals for baking normal maps.

Update to address feedback from @William Reynish (billreynish).

I finally had time to look at this more. A few notes:

  • Cycles vs Eevee: Right now this also shows when Eevee is enabled, but I believe Eevee is not supported?
  • The baking now exists inside the Object Properties, which generally seems confusing and quite messy.
    • It's the first thing you see now when you open Blender
    • Avoiding this could be tricky. I suppose we could make it only appear if you enable Baking from inside the Render Properties, or something like that.
    • At the very least, the parent panel could be closed by default and renamed to just 'Bake'

Good points. I changed it so that it's named "Baking" and it's closed by default. Also, it now only appears for Cycles, so it's invisible in the default startup anyways.

  • The layout could use some work
    • The Bake operator button appears inside the bake pass, but I assume it bakes all passes? The relationship between these items is somewhat confusing

True, I changed that.

  • The output image is not in the Output sub-panel
  • Generally we could organize subsections in Input, Method and Output to follow the logical flow here.

Hm, okay. Originally I intended to have the basic options at the top, but I see why this structure might be better. I changed it for now.

  • Why is Use Cage dependant on Bake From Collection? They seem unrelated to me.

Honestly, I don't know either, but it's like that in the current code as well (for Bake From Selection).

  • We should use property split for the Image, Material and UV Map fields so the alignment is correct.

I guess this is obsolete now that they're in separate panels.

It's a good first step to improve the bake system. Anyway, I think that the main problem of the actual implementation of baking is that it is by object. It's is a error, because we don't need bake only one object, I need to bake 10-20 objects that use same materials but they are different objects. Because other way I'm obligated to use a proxy object to bake. Similar problems if I'm baking a lightmap ¿How we bake a lightmap? joining all objects in one? it's not practical.

So, the base of the system must be not ligated to the object. The bake must be separate "nodes" that make reference to what objects must to bake (with a list of objects, a collection or maybe materials) and made all this process automatic. And must to allow that objects will bake only with same name objects, or maybe other solution, to avoid problems baking.

Other problem of the actual system, not this patch, is that the system apply the border dilation in each bake, not at the end of the bake. It make taht the dilation can overwrite other islands pixels, depending of the baking order.

It's a good first step to improve the bake system. Anyway, I think that the main problem of the actual implementation of baking is that it is by object. It's is a error, because we don't need bake only one object, I need to bake 10-20 objects that use same materials but they are different objects. Because other way I'm obligated to use a proxy object to bake. Similar problems if I'm baking a lightmap ¿How we bake a lightmap? joining all objects in one? it's not practical.
So, the base of the system must be not ligated to the object. The bake must be separate "nodes" that make reference to what objects must to bake (with a list of objects, a collection or maybe materials) and made all this process automatic. And must to allow that objects will bake only with same name objects, or maybe other solution, to avoid problems baking.
Other problem of the actual system, not this patch, is that the system apply the border dilation in each bake, not at the end of the bake. It make taht the dilation can overwrite other islands pixels, depending of the baking order.

That doesn't make any sense. Yes, we should be able to bake multiple objects at once, without manually clicking bake on each, but the setup of all the baking needs to be individual per object depending on materials and UVs.

It's a good first step to improve the bake system. Anyway, I think that the main problem of the actual implementation of baking is that it is by object. It's is a error, because we don't need bake only one object, I need to bake 10-20 objects that use same materials but they are different objects. Because other way I'm obligated to use a proxy object to bake. Similar problems if I'm baking a lightmap ¿How we bake a lightmap? joining all objects in one? it's not practical.
So, the base of the system must be not ligated to the object. The bake must be separate "nodes" that make reference to what objects must to bake (with a list of objects, a collection or maybe materials) and made all this process automatic. And must to allow that objects will bake only with same name objects, or maybe other solution, to avoid problems baking.
Other problem of the actual system, not this patch, is that the system apply the border dilation in each bake, not at the end of the bake. It make taht the dilation can overwrite other islands pixels, depending of the baking order.

That doesn't make any sense. Yes, we should be able to bake multiple objects at once, without manually clicking bake on each, but the setup of all the baking needs to be individual per object depending on materials and UVs.

that dont have any sense. if I want to bake the same AO map in 500 objects I must to configure in each object all the passes, options and file path?

In other software and Blender addons that have tried to tackle this, they use a design called Jobs. In these jobs you specify the passes and objects (could be a collection) that should be baked, to which UV map, as well as cage relationship. Then you may bake one or all jobs. The less you have to set up per object and per material, the better. Then it's up to the user if they need a special case of individual bakes per material or per object. You don't have to design a UI where you select one UV map for each object either, instead, in the Job you select just one UV map index or name, and it's up to the user to make sure the objects don't overlap in UV space, they can always create more jobs.

As for passes, and this goes in hand with the AOV design task, it would be nice to have a series of premade passes that don't have to be set up per material with the AOV output node. Otherwise we're in the same place. Basically the same things that the Geometry node gives you, like Position, but global instead, as well as other PBR things like roughness.

It's a good first step to improve the bake system. Anyway, I think that the main problem of the actual implementation of baking is that it is by object. It's is a error, because we don't need bake only one object, I need to bake 10-20 objects that use same materials but they are different objects. Because other way I'm obligated to use a proxy object to bake. Similar problems if I'm baking a lightmap ¿How we bake a lightmap? joining all objects in one? it's not practical.
So, the base of the system must be not ligated to the object. The bake must be separate "nodes" that make reference to what objects must to bake (with a list of objects, a collection or maybe materials) and made all this process automatic. And must to allow that objects will bake only with same name objects, or maybe other solution, to avoid problems baking.
Other problem of the actual system, not this patch, is that the system apply the border dilation in each bake, not at the end of the bake. It make taht the dilation can overwrite other islands pixels, depending of the baking order.

That doesn't make any sense. Yes, we should be able to bake multiple objects at once, without manually clicking bake on each, but the setup of all the baking needs to be individual per object depending on materials and UVs.

that dont have any sense. if I want to bake the same AO map in 500 objects I must to configure in each object all the passes, options and file path?

Depends on workflow. If you are baking let's say GI from Cycles for Eevee then it needs separate workflow, but if you are baking PBR map sets for game engines, then yes. Generally speaking, the problem right now is that these two very different workflows are mixed into one tool.

Note that the patch already will bake the passes on all selected objects.

As for the argument of having the same passes on a lot of objects - hm, yeah, I can see why that would be useful. I'm not sure how to do this properly - it would be possible to add an option for "use the bake passes of the currently active object for all selected objects", but that's starting to drift back into the obscure unintuitive workflow that we have right now. Maybe add an option in the Input panel for "Bake *To* Collection"?

It's a good first step to improve the bake system. Anyway, I think that the main problem of the actual implementation of baking is that it is by object. It's is a error, because we don't need bake only one object, I need to bake 10-20 objects that use same materials but they are different objects. Because other way I'm obligated to use a proxy object to bake. Similar problems if I'm baking a lightmap ¿How we bake a lightmap? joining all objects in one? it's not practical.
So, the base of the system must be not ligated to the object. The bake must be separate "nodes" that make reference to what objects must to bake (with a list of objects, a collection or maybe materials) and made all this process automatic. And must to allow that objects will bake only with same name objects, or maybe other solution, to avoid problems baking.
Other problem of the actual system, not this patch, is that the system apply the border dilation in each bake, not at the end of the bake. It make taht the dilation can overwrite other islands pixels, depending of the baking order.

That doesn't make any sense. Yes, we should be able to bake multiple objects at once, without manually clicking bake on each, but the setup of all the baking needs to be individual per object depending on materials and UVs.

that dont have any sense. if I want to bake the same AO map in 500 objects I must to configure in each object all the passes, options and file path?

Depends on workflow. If you are baking let's say GI from Cycles for Eevee then it needs separate workflow, but if you are baking PBR map sets for game engines, then yes. Generally speaking, the problem right now is that these two very different workflows are mixed into one tool.

actually the main software for bake, substance, marmoset, or xnormal. bake all objects at same time.

Note that the patch already will bake the passes on all selected objects.
As for the argument of having the same passes on a lot of objects - hm, yeah, I can see why that would be useful. I'm not sure how to do this properly - it would be possible to add an option for "use the bake passes of the currently active object for all selected objects", but that's starting to drift back into the obscure unintuitive workflow that we have right now. Maybe add an option in the Input panel for "Bake *To* Collection"?

i think that bake jobs is the better option. in that jobs you indicate.

  • the lowpoly objects (maybe with collections or other)
  • The highpoly objects (with other collection)
  • The UV map
  • The Material Filter
  • The passes

And inside the passes

  • If the objects must to use multires to bake (not like now that you indicate it out)
  • if the objects must to bake only same name meshes

Probably the option to implement that is inside Image editor or a new editor. I know that it's strange tell that, but I don't see why bake must be inside properties of an object.

i think that bake jobs is the better option. in that jobs you indicate.

  • the lowpoly objects (maybe with collections or other)
  • The highpoly objects (with other collection)
  • The UV map
  • The Material Filter
  • The passes

And inside the passes

  • If the objects must to use multires to bake (not like now that you indicate it out)
  • if the objects must to bake only same name meshes

Probably the option to implement that is inside Image editor or a new editor. I know that it's strange tell that, but I don't see why bake must be inside properties of an object.

IMHO baking is complex enough that it doesn't lend itself well to traditional UI design. I wrote an addon called Meltdown (no longer maintained) that tried this, but to achieve the flexibility I needed from the baking system as an artist, the UI had to be extremely complex and arcane. I wasn't very happy with that.

But baking could be a perfect fit for nodes.

Each nodegraph would constitute a job to be completed together and specify however many source, cage and target meshes or collections, and however many passes, material overrides and outputs the user desires.

Bake graphs could then be fired from the traditional Render panel, or straight from the node editor.

This would maximize flexibility while eliminating UI clutter from arcane and rarely needed functionality.

In fact, someone over on Blenderartists has created a node-based baking UI over here:
https://blenderartists.org/t/addon-bake-wrangler-b0-6-node-based-baking-tool-set-updated/1187732

While this addon is extremely incomplete, it's very promising and looks like a good place to start.

i think that bake jobs is the better option. in that jobs you indicate.

  • the lowpoly objects (maybe with collections or other)
  • The highpoly objects (with other collection)
  • The UV map
  • The Material Filter
  • The passes

And inside the passes

  • If the objects must to use multires to bake (not like now that you indicate it out)
  • if the objects must to bake only same name meshes

Probably the option to implement that is inside Image editor or a new editor. I know that it's strange tell that, but I don't see why bake must be inside properties of an object.

IMHO baking is complex enough that it doesn't lend itself well to traditional UI design. I wrote an addon called Meltdown (no longer maintained) that tried this, but to achieve the flexibility I needed from the baking system as an artist, the UI had to be extremely complex and arcane. I wasn't very happy with that.
But baking could be a perfect fit for nodes.
Each nodegraph would constitute a job to be completed together and specify however many source, cage and target meshes or collections, and however many passes, material overrides and outputs the user desires.
Bake graphs could then be fired from the traditional Render panel, or straight from the node editor.
This would maximize flexibility while eliminating UI clutter from arcane and rarely needed functionality.
In fact, someone over on Blenderartists has created a node-based baking UI over here:
https://blenderartists.org/t/addon-bake-wrangler-b0-6-node-based-baking-tool-set-updated/1187732
While this addon is extremely incomplete, it's very promising and looks like a good place to start.

Yes, I though in something like taht, But I didn't want to talk about of a alternative so different.