Page MenuHome

Bendy bone segment deformation interpolation
ClosedPublic

Authored by Alexander Gavrilov (angavrilov) on Apr 3 2019, 11:06 AM.

Details

Summary

I implemented interpolation between bendy bone segments. It eliminates the stair-step-looking deformations and vertex jumping when using shape keys with bendy bones.

This shows bendy bones with interpolation:

And this shows how it works currently in master branch:

It's a minor change in a single source file limited to a single static function (with 2 small static functions added to improve clarity).

Diff Detail

Repository
rB Blender

Event Timeline

Sam (Sam200) created this revision.Apr 3 2019, 11:06 AM

This seems like a good idea. My only concern is the performance impact, would be good to test how playback speed is affected in a real world rig. Probably it's ok.

Alexander Gavrilov (angavrilov) requested changes to this revision.EditedApr 3 2019, 7:51 PM

I don't really like these uninterpolated ends, but I'm not sure it's possible to do anything about this without rethinking this whole idea of segments, and re-doing the B-Bone construction math.

Basically, if you are going the fully interpolated way, you'd probably want to make it so that the transformation is also continuous at the transition between bones in a chain. To do that, instead of building N 'segments' like sub-bones, you basically want to sample the bezier transformation at N+1 points (including both ends of the bone), and interpolate between them all. This will however require changing everything, including B-Bone drawing...

source/blender/blenkernel/intern/armature.c
944

There is a whole library of matrix math functions, I'm sure there's something suitable there for this.

This revision now requires changes to proceed.Apr 3 2019, 7:51 PM

Actually, I thought more about it and it may be possible to hack in interpolating until the ends without full overhaul of segments at the cost of memory (two more matrices to store per B-Bone instead of just one), by basically computing and storing the end matrices derived directly from the handles in addition to the current segments. The drawing code then can use the middle segment matrices, while deformation can blend in the end matrices in those 0.5 areas. The question is how well it would work with such hacks, and whether it is worth it.

This seems like a good idea. My only concern is the performance impact, would be good to test how playback speed is affected in a real world rig. Probably it's ok.

The performance impact should be comparable to having one additional bone assigned to each vertex within the bendy bone affected vertices or even less. In other words, quite small.

I don't really like these uninterpolated ends, but I'm not sure it's possible to do anything about this without rethinking this whole idea of segments, and re-doing the B-Bone construction math.
Basically, if you are going the fully interpolated way, you'd probably want to make it so that the transformation is also continuous at the transition between bones in a chain. To do that, instead of building N 'segments' like sub-bones, you basically want to sample the bezier transformation at N+1 points (including both ends of the bone), and interpolate between them all. This will however require changing everything, including B-Bone drawing...

The bone ends are already interpolated with adjacent bones outside this function, as part of this bones overall vertex weighting.

There is a whole library of matrix math functions, I'm sure there's something suitable there for this.

I have looked at the math library, and I couldn't find a simple linear matrix interpolation. The only blending or interpolation functions there are for matrices are those that break up the transformation and interpolate rotation differently. They are significantly slower than a simple linear matrix blend and are not consistent with how interpolation is done between bones, as far as I understand (haven't looked at that code yet). That is, if "Preserve volume" option in armature modifier is not checked.

Perhaps a simple matrix blend function should be added to the math functions instead?

I don't really like these uninterpolated ends, ...

Ok, I see what you are saying. It would be better if the entire length of bone would be interpolated by the spline. For that I would suggest calculating transforms between segments instead of for each segment, and interpolate that.

... To do that, instead of building N 'segments' like sub-bones, you basically want to sample the bezier transformation at N+1 points (including both ends of the bone), and interpolate between them all. This will however require changing everything, including B-Bone drawing...

I don't really like these uninterpolated ends, ...

Ok, I see what you are saying. It would be better if the entire length of bone would be interpolated by the spline. For that I would suggest calculating transforms between segments instead of for each segment, and interpolate that.

I'm so dumb sometimes, you are saying the exact same thing.

... The question is how well it would work with such hacks, and whether it is worth it.

Here I subdivided the bendy bone into two bones to test how the transition looks between bones, to see whether the lack of interpolation at the end segments is visible.

What you see here is 3 bones, the "segment" at the bottom is a regular one-segment bone. The next 5 segments are the next bone, and the upper 5 segments are another bone.

I can't even see where the transition is. There is no visible lack of interpolation.

Here's one with fewer segments per bone.

The uneven deformation is due to the segmentation itself, but the transition between the two bones is still not distinguishable.

I added linear_blend_m4_m4m4 function to matrix math functions, because it was missing and others might need it in the future.

As I said before, the functions that already exist don't do a simple linear interpolation, which is exactly what is needed here to keep it consisted with the way bones are blended.

Sam (Sam200) marked an inline comment as done.Apr 4 2019, 9:59 AM

For those interested, I'm currently working on adding interpolation at the end points by caching two additional matrices per b-bone, one per endpoint, following Alexander's idea.

I'm getting close.

I already figured out where I need to make the changes and that they won't break other code.

I'll post again when I'm done.

Note that Armature modifier isn't the only place that uses B-Bone shape - there is the Armature constraint which by definition must produce the same transformation. Copy Transforms may also benifit from access to smoothed shape rather than raw segments.

I have also just committed a clean up change that will cause a merge conflict with this.

Note that Armature modifier isn't the only place that uses B-Bone shape - there is the Armature constraint which by definition must produce the same transformation. Copy Transforms may also benifit from access to smoothed shape rather than raw segments.

I will have a look. Hopefully it calls on the same functions, but if not, I can can make the appropriate changes there too.

I have also just committed a clean up change that will cause a merge conflict with this.

That's ok. I will deal with that before updating my diff.

Bendy bones are now interpolated over their full length.

The latest result:

I managed to keep the changes local to armature.c (besides the added matrix math function). The diff is a bit confusing to decipher so here's a summary of most important changes:

    • In the bPoseChannel_Runtime struct, the members bbone_deform_mats and bbone_dual_quats have additional two elements allocated. These are the start and end point deforms. This change should not affect most code that uses these arrays. Appropriate copy functions were updated.
  • I had to refactor some function implementations to prevent code duplication. Basically, these functions performed some calculations as an intermediate step, which I wanted access to. So I broke them up where necessary.

As to updating constraints such as Armature constraint and Copy Transforms:
Apparently, as currently implemented, they treat bendy bones the same as regular bones. The spline and segments are simply ignored. So, making that code recognize bendy bones can probably be treated as a separate issue.

I forgot to take scaling into account for the endpoints. Will post again with update.

FYI, I started investigating if I can implement this the proper way with N+1 nodes between segments.

Sam (Sam200) updated this revision to Diff 14639.EditedApr 6 2019, 4:23 AM

Everything looks to be working now.

The updated diff is taken from the current master branch.

I would love some feedback.

Edit:
By the way, in the image you see, all 3 armatures have a small bone on the bottom and a long, 5-segment bendy bone on top.

FYI, I started investigating if I can implement this the proper way with N+1 nodes between segments.

Yeah, that would certainly be better.

So I got this kind of transformation with just 3 segments, sharp turn at one end, and zero easing at the other one.

This changes B-Bone deformation to use matrices computed between
segments, with tangents computed from the real bezier curve.
This data is properly used in constraints too.

Sam (Sam200) added a comment.EditedApr 6 2019, 2:03 PM

This changes B-Bone deformation to use matrices computed between
segments, with tangents computed from the real bezier curve.
...

Nicely done!

In your image, why isn't there any blending between the adjacent bones? Are there simply no weights assigned to the bones?

Also, I think your code may have the same bug I ran into (and only partially solved). Namely, add_weighted_dq_dq function produces messed up scaling if some of your dual quats have member scale_weight set to 0 and some don't. It's set to 0 if the dual quat was generated from a unit scale matrix.

The work around that I found for this is to manually set scale_weight to 1.0 and scale to a unit matrix for all dual quats that have scale_weight equal to 0.0. That is if you know that some other dual quats are scaled already.

Really though, add_weighted_dq_dq function and DualQuat struct needs to be looked at to fix this issue.

Edit: I forgot to mention that I already submitted a bug report about that function. It really needs to be looked at.

In your image, why isn't there any blending between the adjacent bones? Are there simply no weights assigned to the bones?

Because all vertices are each assigned to only one bone, the whole point being to show how the B-Bone deformation works in its pure form.

Edit: In the bottom left corner it shows zero easing - I decided to make that case ignore the handle and align to the actual curve like before, because setting easing to 0 is used to 'disable' B-Bones, e.g. in rigify. Top right is an active handle, so the interpolation within the B-Bone matches the shape up with the end as well as it can.

Alexander, I just ran your code and I can't reproduce that dual quat weighting bug. I don't know what you did differently to avoid it, but everything is looking good.

My test scenes look exactly the same with your code as they did with mine. I love it.

Did you also update the python api where a script gets access the segment matrices? It's needed for armature export scripts, which I was writing just before I got into this bendy stuff.

Did I mention already that I'm excited about this new bendy bone interpolation. It's so much nicer. :)

Alexander, I just ran your code and I can't reproduce that dual quat weighting bug. I don't know what you did differently to avoid it, but everything is looking good.

Well, one thing that comes to mind is that instead of trying to interpolate between segments and then add the result to the global accumulator, I refactored the code so that it can simply add both segments to the accumulator with appropriately multiplied weight. I.e. now it works absolutely identically to there simply being two bones instead of two segments.

Did you also update the python api where a script gets access the segment matrices? It's needed for armature export scripts, which I was writing just before I got into this bendy stuff.

The API now returns matrices for the N+1 nodes between segments, because it's based on accessing what is cached in Runtime and that changed.

This looks so good. :)

I like your implementation much more too. Exporting bendy bones by converting them into regular bones will produce one fewer bone with your implementation than it would with mine.

Thanks for taking the time to do this right.

@Brecht Van Lommel (brecht): So, what is the plan about this? This would technically be a backward compatibility breaking change (slightly different transformation from B-Bones), so possibly it should go in asap, if breaking compatibility is to be avoided after 2.80?

@Alexander Gavrilov (angavrilov), you can commit this.

I did not review the code in detail, but functionality and overall structure seems fine.

This revision is now accepted and ready to land.Apr 13 2019, 1:57 PM
This revision was automatically updated to reflect the committed changes.