Page MenuHome

Delta Mush Modifier
Closed, ResolvedPublic

Description

Hi all,

(Campell Barton, I picked you since you are a modifiers module owner, I have no real idea who is best to review this)

I have put together a modifier implementing a deformation cleanup / relaxation method originally presented by Rhythm and Hues at siggraph 2014 (http://dl.acm.org/citation.cfm?id=2614144 if you have access behind the paywall, otherwise the video on http://www.fxguide.com/featured/voodoo-magic/ is a good overview).

The basic idea of this modifier is to smooth the mesh in a rest position, and record the changes in vertex position in each vertex's local tangent space.
After this, each evaluation of the modifier applies the same smoothing and then the original surface detail is reconstructed by re-applying these rest pose deltas in each vertex's tangent space. The effect of this is to smooth out deformations created by other modifiers etc (e.g. armature), while retaining fin surface detail.

For a comparison of the effect:
Without:


With:

Smoothing only:

The modifier panel is based on the laplacian smoothing modifier (since it uses the same smoothing), and looks like:

Note that the UV map is for generating consistent tangents, as UV coordinates are required for tangent calculation.

I put this modifier together largely as a way of learning my way around parts of blender's codebase, so I sure there's plenty of stuff I could be doing more efficiently etc.

Regardless of whether this gets accepted or not I am interested in continuing get involved in blender, particularly around modifiers/deformation tools for character skinning.

Regards,
Jack

Details

Differential Revisions
D1183: Implement the delta mush modifier.
Type
Patch

Event Timeline

Jack Simpson (sazerac) set Type to Patch.
Jack Simpson (sazerac) created this task.
Jack Simpson (sazerac) raised the priority of this task from to Needs Triage by Developer.
Julian Eisel (Severin) triaged this task as Normal priority.

MUCH better since the switch to standard smoothing, but still some glaring issues that I'm having trouble pinpointing. Pinning of boundary verts is a big one. I don't think this needs to be as complicated as setting up vertex groups, just a checkbox should work. The two other main issues involve the smoothing and reprojection. Smoothing seems to be happening universally, which causes mesh shrinkage. This is especially evident on the ears/fingers/toes/etc. on human models. The original implementation avoids this, I believe, by weighting the smoothing based on distance between verts compared to their rest positions. I haven't looked at the code closely to see if anything like this is happening yet, but currently the modifier seems to have trouble maintaining volume. The final issue is related to the smoothing algorithm itself. The mesh explodes at higher values. I'd suggest putting a "soft" clamp at 0 and 1 for the Lambda, while still allowing manual values greater than that (although I don't know why you'd ever want negative smoothing).

On the plus side, it's very fast, and already quite useful in a number of cases. This will massively simplify rigging and skinning when it (fingers crossed!) makes it into master.

On further testing it would appear that the issue is somewhere in the reprojection of points, as the smooth modifier by itself doesn't have the same issue with maintaining volume.

Hey! Some idea for your mush after giving it a try (blender python and houdini vex) and talking with Matt:
Basically, things should go this way:

  • laplacian smooth deformed mesh
  • compute tangent space
  • apply inverted delta (inverted tangent space)

Seems something went wrong cause you lose mesh volume in the actual implementation :-)
Here i get best result with a medium lambda (0.5) and something btw 30/60 iteration of smoothing (depending on the mesh resolution).

About pinPoint, non manifold edge could be pin by default (according to the video) but a vertex weight sounds good enougth.
About their 'weighted distance' option, i tried many things but didn't manage to get the exact same result as shown in their video. But even without this, modifier could still be very usefull!

Good luck and have fun with it :)

Boris

bobizib, thanks for the link to the paper, I'll have a read and see about trying that smoothing alg.

The stuff I think is from lingering errors in the tangent basis calculation. I noticed them while fiddling with it late last night as well. From you guys' tests, it appears they are much more prominent with higher smoothing iterations.

I'll have a shot at adding boundary pinning for non-manifold meshes and fix the tangent stuff tonight.

Here's the file I've been testing on, if you're interested. It has lots of surface details that need to be preserved. The modifier as it exists actually handles this mesh fairly well. It still showcases some of the volume shrinkage though. No non-manifold points to test with on this one unfortunately, but I'll gladly whip up some other test cases if you need them.

Cheers!

Didn't get as far as I hoped with changes tonight, so no new patch.

I had a look at (and test of) the face center based tangents and they don't seem to work properly, which I suspect is because they all always point approximately to the face center, which leads to poor results when averaging them around a vert.

I've started looking at the smoothing implementation / pinning, but that may take a little longer.

I had a chat with m9105826 over IRC and I think I might try changing the modifier to store the absolute positions of the rest shape rather that the deltas to see what its like. This would make it easier to tweak on the fly, but take a bit of a performance hit. Possibly I could also cache the deltas and only update them from the cached rest positions when other modifier params change.

About face center tangent, you don't have to average it, just pick the first met tangent for a vertex (first time it is found in a face loop) and ignore it next time it is met.
Not sure it can works this way through C / blender mesh things, but works great in python.

have fun !

Thanks, I just had a quick shot with that and initial tests look good, no spinning around normals of the deltas so far.

I also have a rough implementation done for storing the original positions as well as the deltas and It seems to work, so I'll try and get an updated patch tonight, there are a few bugs I still need to deal with and I don't have the non-manifold pinning done yet.

Did a big tidyup and a bunch of improvements.

Still no boundary pinning (I will get to it eventually), but it should be possible to do it with vgroups now.

Testing it will probably not work on old files as I have changed the modifier data a fair bit.

Smoothing seems to be messed up now. Sections still aren't smoothed even after hundreds of iterations. Could be a problem with either deltas or the new smoothing algorithm, but it looks like it's more likely the new smoothing.


Update the patch with boundary pinning.

@Campbell Barton (campbellbarton), other than code review, whats the process for getting this accepted?

Pinning works as exactly as expected. Speed is really nice too. From an artistic point of view this seems good to go in my opinion, it's already changed the way I'll skin in Blender in the future.

@Jack Simpson (sazerac), re: process of getting accepted.

If the functionality is generally useful and fits with Blender's design, then its normally accepted.

In this case the code seems generally fine (while not being quite ready for master).
There are a few design issues - storing old settings and continually comparing with new is weak and would like to try avoid. I need to check on binding more too.

Since your still making some changes and getting user feedback I'd like to get the patch cleaned up (resolve any obvious issues).

Then get some more user feedback to see how well the final modifier works.

Thanks. Yeah I'm planning on making a thread on BA when I got the code fairly stable and sorted out.

@Campbell Barton (campbellbarton) just looking through your changes on the new branch, looks good, and looks like I have lots of reading on C optimisations to do at some point.

One thing I did experiment with is using squared distance rather than distance for the neighbour weighting in the smoothing iterations. It looks like the results aren't as good though, it seems to distort the mesh a bit more at some bends. ~20% speedup though.

Also, one very minor suggested change in the BA thread was to make the smoothing only toggle apply to the unbound state: http://hastebin.com/oyamajuzew.coffee

@Jack Simpson (sazerac), could we discuss details on patch/implementation in D1183?
I didn't notice your reply here, but have some question about smoothing method in the differential.

Not sure its necessary to have both design task and differential, but this can stay open to get user feedback I suppose.