This tasks is to design a plan to fix the issues related to having different datalayer for vertices and Multires data so we can use it to fix existing bugs and implement new features.
Currently, the data editable from Multires are coordinates, per element visibility, sculpt mask and face sets. Each one of these data as a different solution for this:
- Coordinates are being applied to the base mesh from the grids manually by using the Apply Base operator. When modifying the base mesh, that changes the subdivision limit surface, updating the result as coordinates in grids are stored as displacement in the CD_MDISP datalayer. This is not the standard design of a Multires sculpting software, but this approach works for all the expected use cases and it allows for some non-standard functionality (like sculpting the base mesh while previewing the displacement or sculpting in deformed meshes).
- Element visibility is stored in the ME_HIDE flag in MVerts->flag for meshes and in a bitmap which is part of CD_MDISP for grids. As far as I know, there is nothing that keeps this data in sync.
- The Sculpt Mask is stored in the CD_PAINT_MASK for vertices and in the CD_GRID_PAINT_MASK for grids. The main sync point is in BKE_sculpt_mask_layers_ensure were the current state of CD_PAINT_MASK is copied into a new CD_GRID_PAINT_MASK datalayer when grid mask data does not exist, and it is never updated again. When applying Multires with CD_GRID_PAINT_MASK, the mask data is also not copied to the new base mesh.
- Face Sets are stored in CD_SCULPT_FACE_SETS for base mesh polys and Multires reads them directly based on the index of the face the grid is. This makes them to be always in sync, but they are not editable per element in Multires.
This is affecting future development in the following ways:
- As the paint mask is not synced from grids to the base mesh, there is not possible to implement features like mask extract from Multires as there is no CD_PAINT_MASK datalayer in the base mesh after applying the modifier to extract the new mesh.
- The Sculpt Vertex Colors Multires implementation includes a CD_GRID_PROP_COLOR datalayer and uses a similar approach as masks. If we replicate the same code to sync it with CD_PROP_COLOR, were are going to have the same issues, but for this feature those are not acceptable. Vertex Colors should be always in sync between the base mesh and grids without requiring anything from the user. Also, there can be multiple CD_PROP_COLOR datalayers, which is a feature that does not exist in the sculpt mask datalayer.
- If we want to implement sculpt layers for displacement, that will require to add another datalayer to store them. I'm still not sure how they will be stored (coordinates, displacement relative to the layer 0...). Implementing this feature on top of the current design means dealing with the base mesh sync issue for multiple layers plus always running the Multires smooth propagation for all of them each time the base layer changes.
- There are also some features that are trivial to implement (like PBR vertex painting using a datalayer that contains color + PBR values at the same time) that are also limited by not having mesh/multires data unified.
The mesh/grids datalayer management project should try to provide a common solution to all these problems. Ideally, it should be possible to use the same vertex datalayer transparently from the tools and the Multires code, having all these abstracted by another system that keeps the mesh and grids data always in sync.