The purpose of this task is to describe the main uses cases and functionality that should be provided by a Multires sculpting system, as well as to discuss a plan on how to implement that in Blender.
The main use case of Multires
The problem Multires solves
When you are working on any art medium (this also applies for painting), you should always try to work from the general shapes to the details. In case of digital sculpting, this means that you should always try to work with the minimum number of vertices that represent the level of detail you are trying to add to the model.
This is also true for the current workflow we have implemented (The Voxel remesher). By default, it provides a low poly mesh (about 10K vertices), that you should deform trying to get main shape of the object you want. After that, you increase the resolution of the remesher until you have enough vertices to represent the next level of detail you want to work on, and you continue from there.
The reasons you want to work this way are:
- Performance: It is faster (more responsive, without lag) to deform a mesh with 10K vertices than a mesh with 25M vertices. If you use a big brush size needed to modify the object proportions on 25M vertices at the same time, it is going to lag. (Unrelated note, but we should not try to optimize Sculpt Mode to support these kind of operations as they don’t make any sense with a Multires system in place).
- (Most important) Noise: If you sculpt with more vertices than needed for the size of details and shapes you are trying to get, you are going to end with a lot of uncontrolled shape noise produced by the tools. This is not a specific problem with Blender brushes and tools, this happens in any software and in every sculpting system. If you use the Draw brush to add some volume on a big area directly onto a 25M vertices mesh, you are going to need need to clean manually all small uncontrolled imperfections that big stroke stroke produced if you want that to be the final surface. If you start sculpting from a sphere with 1M vertices, you are going to spend more time cleaning shape noise than sculpting or the sculpt is just going to look bad. This is why nobody works this way. You don’t generate unintended shape noise if there aren’t enough vertices to represent it.
That is the main problem Multires sculpting solves. With Multires, you are able to choose at any moment the level of detail that is appropriate for the change you want to make in you model, solving both the performance and the noise issues. This gives an artist a huge amount of freedom as you know you can always go back to tweak low frequency details (proportions, general volumes) after sculpting mid/high frequency details (wrinkles, skin alphas...) if you consider that they don’t look right, without loosing any work.
Without this, the rest of the sculpting workflows (the Voxel remesher, Dyntopo) are the same as sculpting with real clay or marble, you can only sculpt bigger first shapes, then mid details and them the small details. If it does not look right, there is not way to fix it, you need to delete what looks wrong and start again. This is why those workflows are often used for concepting general shapes, and you want two switch to Multires as soon as you can. Sculpting a complex model without using Multires, on any software, is extremely (and unnecessarily) hard and it requires an insane amount of skill from the artist to workaround the limitation. It is like sculpting without using undo. It is too risky to create a high poly sculpt with projected alphas without this feature.
In order to solve this problem, the main thing Multires needs to guarantee is that when you sculpt shapes on your model in the appropriate resolution level, all the other levels are updated accordingly to reflect the changes you did. This is what people often refer as ¨detail propagation¨.
Multires Design
When designing a sculpting Multires system that provides the previously mentioned functionality, We need to take the following points into consideration:
- All Multires subdivision levels contain valid information (sculpted shapes, mask, colors...) that should never be lost or damaged.
- The user wants to modify any subdivision level, at any time, without restrictions (there should not be limits of how much the mesh should be deformed or “not allowed modifications”).
- The user wants to delete and rebuild higher/lower subdivision levels in he/she considers that the shape information that level contains is not longer relevant.
For example, if the user wants to do a high poly sculpt, he/she switches to the higher level subdivision possible that the computer allows to create the high frequency detail projecting alphas. When that is done, the user may want to switch to a lower level to tweak proportions, or to a mid level to tweak small volumes with a soft brush. All the work the user put into applying the alphas should be propagated with the lower subdivision level changes with the Multires functionality and the model should look fine.
If, for some reason (a bug or an design issue in the implementation that did not consider point 1), switching and modifying lower levels may introduce mesh artifacts in the higher ones (the ones that contains the alpha projections and high frequency details), it will destroy all the work done previously by the user in that levels.
In that case, the user is again limited to not going back to select the mesh resolution needed for the modifications he/she wants to make in the mesh. A Multires system like this is not solving the intended problem and there is no reason to use it. It provides the same functionality as any of the other sculpting workflows (Voxel, Dyntopo), but even more limited as the user can’t modify the base topology to adapt it to a new shape. It does not have any reason to exist.
If you project requires you to use a fixed predefined topology in your model (for animation, for example), subdividing the mesh manually and deforming it with a cage or a datatransfer provides the same functionality. You can also get a lower subdivision level by duplicating the model and using unsubdivide if you need it for texture baking or animation. This is not the main problem Multires should be trying to solve.
Any unwanted mesh artifact in the detail propagation, no matter how small it is, invalidates all Multires sculpting features and makes it not functional for its intended use case, as it leads to irreparable data corruption.
Current Multires Modifier
Considering the previously design and use case description as valid Multires sculpting system:
- Multires in 2.81 does not support switching levels. This invalidates the feature completely and makes it not functional, but prevents the users to corrupt the mesh data if they use it. This is like not having Multires sculpting at all.
- Multires in 2.80 creates artifacts and vertex explosions when modifying any level, creating irreparable mesh data corruption. Releasing Blender with Multires in this state and without any visible UI warning could have destroyed the projects of many users in an irreparable way. Luckily, most users were not using it to create their models in the previous versions because it was also not working as intended.
- Multires in 2.79 and before creates a visible grid pattern on higher levels when modifying lower ones. This introduces unwanted shape noise and effectively destroys valid sculpted information, making it not functional for its intended use case. Users are limited to work considering of how much they can deform lower levels by the amount of noise they are predicting Multires will produce in the higher ones. This is not something that should be happening in a production ready implementation of the feature.
The most advances tools Blender has right now that were added in the last releases (Quadriflow, Elastic deform, IK Pose brush...) are designed to work with a Multires system:
- The main purpose of Quadriflow is to produce subdividable geometry for a Multires. When you are done sculpting with Dyntopo/Voxel remesher, where you can’t have the freedom to choose the level of detail you want to work on, you can quickly generate a base mesh, subdivide it and reproject the details back. That way you can tweak again general proportions and low frequency detail.
- These brush tools are designed to tweak proportions/big shapes of models, which is something you most likely want to do after sculpting more detail to have an idea of how your model looks like. Multires solves that problem.
- These brushes can’t be used directly on high poly meshes. These tools needs to have disabled some sculpt mode optimizations in order to work, so they are much slower than regular brushes. In fact, if you try to use the IK pose brush directly on a high poly mesh, most likely you are going to run out of memory. If you don’t, it is going to create terrible artifacts between each segment. In order for these brushes to work as intended on high poly meshes, you need to use Multires to switch to a lower level, use the brush, and then switch to a higher level again to get the intended result.
Multires implementation reference
SculptGL contains a valid and functional Multires sculpting implementation. You can try it by sculpting different details at different levels and switching between them, as well as creating and deleting subdivision levels:
https://stephaneginier.com/sculptgl/
This implementation of Multires provides:
- Detail propagation that works across all levels. You can choose the desired level you want to sculpt on and that deformation is propagated correctly to all subdivision levels. There are not visible artifacts of any kind.
- You can both create and delete levels in both directions (higher levels and lower levels) with subdivide/delete higher and reverse/delete lower.
These features are the minimum and required a Multires implementation in Blender should provide, as they are pretty much the definition of a Multires sculpting system. Any other Multires related feature or optimization (being part of the modifier stack, baking, shape keys, layers, exporting, reprojection, tool compatibility, Quadriflow...) should always come later. There should not be a design decisions that prioritize any other Multires feature or optimization without having those minimums perfectly working, bug free and functional.
Proposal
Given the fact that, as described here, Multires was never a functional feature for its intended purpose and:
- The code is extremely hard to read and maintain. I still can’t understand what it is doing in some places or what was the reasoning to make it work that way in the first place.
- New features are planned that require Multires support (masking tools, vertex colors, visibility operators). Adding Multires support for those features should not be a project bigger than the feature itself. This was the case with most of the 2.81 tools, and probably with the future planned features (Sculpt vertex colors or visibility management, for example)
- We already have and sculpt tools abstraction layer in place. Making changes on the underlying data structures of sculpt mode should be easier than before
- The original Multires comes from before 2.5. The basics of how it should works were never correct and a lot of features and modifications were added during the years. We are now trying to build state of the art sculpting features on top of a broken foundation that has more than 10 years.
Some options are:
- Replace Multires modifier with a simple implementation based on the SculptGL code (something we know it works) and start again from there. We can keep the current Multires modifier for compatibility, but we need to evaluate if supporting the features and fixes described in this task on the current code (and make it work exactly as the SculptGL Multires) is going to take more time than implementing something based on the SculptGL code from scratch. We also need to look that the new features that are planned and evaluate how hard they are going to be to add to the current Multires code (colors, per face data...) knowing that the SculptGL implementation already supports them.
- Try to add this functionality to the current Multires modifier. In that case, the priority should always be supporting the functionality described in this task first, and the bring all the rest of the features back. In that case, it would be needed better documentation and code organization to be able to integrate new features into it easily.
