Smooting shape key vertices with sculpt collapses masked vertices to object origin
Closed, ArchivedPublic


When in sculpt mode and using the smooth tool on a shape key causes all masked vertices to collapse to object origin.

Attached is a cube with subdivisions and masked vertices. Unmask to see unaltered mesh vertices. Mask to hide selected vertices. Use the smooth tool on cube. Unmask to see bug result.

Tested on PC (laptop) and Mac.


To Do

Brecht, can you check if this relates to 32943? Otherwise we can ask Nicholas too :)

It's unrelated, the other report was a simple error in the VBO drawing. I'll assign to Nicholas but will take a look into it to see if I can fix.

I can't see anything collapsing in this file, testing with latest svn on Mac. One issue here is that the shape key being sculpted has weight 0.0, so brushes do nothing. That's not a bug, set the weight to 1.0 and it will work.

The second issue I see, which is a bug, is that masks don't show at all there are shape keys. The hidden faces are also not drawn hidden in that case.

Hi Brecht,

Maybe it is my terminology that is confusing? Mask is not the proper term here. I should have said hidden vertices. If you click on the cube with the smooth tool in sculpt mode then change to edit mode to unhide the vertices all hidden vertices have moved to the origin of the model.

I see the brush weight for smooth tool in the uploaded file is set to 0.5 (default setting) so not sure why you are seeing a value of 0.0 - I am sure this is a bug as I have tested on both mac and pc and get the same result.

If you are saying that this isn't a bug then I guess I will just have to live with the fact that I can not use smooth tools on shape keys.

I was referring to the value in the shape keys panel, not the brush weight, the shape key value is 0.0 so the shape keys is not actually going to be visible.

If all the vertices go back to the origin that's a bug, but it's not happening when testing it here.

Apologies, yes the Shape Key is set to 0.0. In fact the bug is more evident if the Basis key is selected before applying smooth.

Hi Brecht,

I have posted a video of the bug in action


The bug may be that those faces draw at all. without the shape key the faces are hidden and can't be painted onto.

Sculpt code is doing paint_is_face_hidden() checks when calculating faces to paint onto, but then drawing all faces because the resulting mesh is a CDDM from the modifier stack.

How about ignore the hidden face check when '(cddm->pbvh && cddm->pbvh_draw)' is false?
... or have CDDM be able to skip hidden faces too.

This bug is fixed in r56465.
There is still the problem that hidden faces cant be painted onto. added todo.

added as TODO.