Page MenuHome

Quaternion SLERP/SQUAD Rotation Modes
Needs RevisionPublic

Authored by Jack Andersen (jackoalan) on Apr 20 2016, 8:52 AM.

Details

Reviewers
Joshua Leung (aligorith)
Group Reviewers
Animation
Summary

This adds two new rotation modes to Objects and PoseBones ('QUATERNION_SLERP' and 'QUATERNION_SQUAD'). Also provides a potential solution for T45473

Objects and PoseChannels maintain a cache of 4 interpolation quaternions during animation playback, providing enough context to turn fcurve time-instances into quaternion interpolation knots.

Additionally, a bpy.ops.graph.sample_quaternions operator is available to 'export' the computed quaternion keyframes into their source fcurves.

Comparison video, demonstrating dynamic long-case correction:
https://vimeo.com/164798991

Diff Detail

Repository
rB Blender

Event Timeline

Jack Andersen (jackoalan) retitled this revision from to Quaternion SLERP Pose Interpolation.Apr 20 2016, 8:52 AM
Jack Andersen (jackoalan) updated this object.
Jack Andersen (jackoalan) set the repository for this revision to rB Blender.
Jack Andersen (jackoalan) updated this revision to Diff 6452.

Fixes crash when user-editable keyframes aren't available (fcurve baked)

Solution feels a bit weak, though not sure theres a really elegant way to handle this.

source/blender/blenkernel/intern/armature.c
2088

Should check this is <4,
also could early-exit once all 4 are found.

2089

This seems like it could easily be a performance bottleneck, it will also ignore fcurve interpolation modes AFAICS.

It should do a binary search, however best choose if this would even be accepted.

Jack Andersen (jackoalan) updated this revision to Diff 6455.

Much more resilient state validation, only applied on ob->recalc & OB_RECALC_TIME passing with several failsafe cases to original FCurve interpolation

Use binarysearch function to discover keyframes

Jack Andersen (jackoalan) marked 2 inline comments as done.Apr 20 2016, 1:46 PM

Yea, SLERP is definitely the sort of algorithm that ignores any sort of component-wise interpolation. Overriding the fcurve interpolation entirely at the right point of update cycle is necessary to gather the curves together.

It makes for a much more sane interpolation over longer keyframe-spans. I think it's a good mode to have, one that Maya has had for years.

Exit filter loop when 4 quaternion components found

More efficient bail when we don't have a full set of quaternion curves

Joshua Leung (aligorith) requested changes to this revision.

As a "general purpose" solution (vs a localised in-production hack), I have several key concerns with this approach:

  1. This assumes that rotation animation only results from FCurves in the active action. However, the NLA may also contain keyframes for these channels.
  2. There are also FModifiers which act upon the FCurves to modify how the curve (including the times/values of the keyframes) will get evaluated. Even if you're not worried about users trying to amplify the results, or add noise, even the simple problem of repeating the FCurve will end up requiring the "Cycles FModifier" (which works by redirecting time when evaluating the curve, so that it ends up repeating).
  3. This only accounts for bones. Objects too can have quaternion interpolation. (Admittedly, it may not used that often, but from experience it's better to not make those assumptions).
  4. Drivers which use the raw property values instead of the final posed transforms (i.e. using "Single Property" targets instead of "Transform Channel") will probably not see these values
  5. In its current state, there are all kinds of limitations like this, but users will not be aware that these limitations will apply when using this option.

Other issues/queries that this brings up:

  1. Have you tested this on rigs with drivers, to make sure that everything still works properly
  2. What about action constraints?
  3. What about for armature ghosting and for motion path calculation?
  4. Does this still cope/degrade gracefully if animators for whatever reason tweaked the curves so that they no longer had keyframes on the same frames?
  1. Right now, animators lose all control over timing of how the animation transitions from one orientation to another. For example, it would be interesting looking into whether being able to control the rate of interpolating between A and B (instead of restricting it to be strictly linear, and throwing away any tweakability from there). Of course, to do that, it would be better to have a single curve controlling interpolation (i.e. the 0-1 factor used for doing slerp between and b).
This revision now requires changes to proceed.Apr 20 2016, 2:04 PM

Yea, honestly I haven't considered all the possibilities for animation sources.

Currently, the fcurve values themselves are all obtained via evaluate_fcurve when user-editable keyframes are present on all curves of the current action. A better solution would be to resolve keyframe-times via the NLA.

I'll do some testing and explore other options for controlling the 'ease' of the SLERPed curve

From a user perspective, I'd prefer (strongly) to see this on a per bone/object bases, perhaps close to where the rotation mode option is (UI wise) . Having it as an armature wide setting feels like a production kludge, aka 'old blender' ;)

A half-baked idea I've been toying around with is that perhaps we will have to look into allowing keyframes to store "extra data", instead of forcing everything into a strict 1 FCurve = 1 Float Value = 1 Number Property Affected.

So, in essence, you could have a single FCurve for "quaternion rotation" (i.e. all 4 quaternion rotation components in one go). We would probably add a "SLERP" rotation mode to make sure that this way gets used, and to keep backwards compat with files saved using the old per-element quaternion stuff. The FCurve would still have normal BezTriple (Keyframes) and FSamples - these would be there for controlling the interpolation/timing behaviour. Alongside this, you'd then have a separate array of the 4 quaternion components (one set per keyframe), which is what the user sets in the UI.

You'd also probably want to have another copy of this, to be stored alongside the "curval" float, to store the interpolated value, which will then be written to the file? Or perhaps, it might be better to instead store some kind of reference/index to the before/after quaternions to use, so that things like FModifiers can "just work" on the normal keyframes, and interpolate/play around with those as they like. Going with this second option, we'd then only perform the actual slerping in the animsys_write_rna() -> we'd take the value from the FCurve, along with the quat values to interpolate between, do the deed, and then write out this quaternion as a whole into the relevant RNA components.

A similar approach could then be used for things like strings and pointers - by making this "extra data" storage general, and able to store arbitrary stuff (we just need to tell it how big each one of these elements is in the array - sizeof(FCurveQuatData), sizeof(FCurveStringData), sizeof(FCurvePointerData)) something similar could happen there too. In the case of strings and pointers though, the FCurves might end up making less sense (as the y-values are effect useless - unless we somehow coop them to indicate in some way whether the pointer values are equal, etc.?). And these kinds of extra data could of course be edited via the Properties Region -> Active Keyframe pane would probably be an ideal candidate for that.

What is murkier however is going through all the various tools in Blender which use/modify keyframes, and make sure that they still do a good job (instead of causing the keyframes and extra data to get out of sync when they modify the curves). That would probably end up being the bulk of work required if we decide to do something like this.

Also, there's the issue of how drivers would then be handled.... In those cases, we might have to only allow for direct "driven value <= source value" mappings, as anything else wouldn't make any sense.

Anyway, that's just a general overview of an option that we could consider. There are still probably lots of smaller details I've overlooked which would end up causing us some grief.

You'd also probably want to have another copy of this, to be stored alongside the "curval" float, to store the interpolated value

I like that idea a lot, made a mockup image to better understand the various control cases

Extending the FCurve with arbitrary data buffers sounds like the best way store it. That would also enable objects to be transformed that way.

If adding auxiliary data to FCurves is too invasive, a simpler option would be a single animatable float property on top of the existing rotation_quaternion array.

Also add a new "SLERP Quaternion" rotation mode which could reveal a fifth animatable value in the UI. Keyframes of this property could be treated as a clock signal, sampling all 4 quaternion fcurves at each curval keyframe.

I like this idea, since it's compatible with existing quaternion data and just a matter of switching rotation mode and (optionally) animating the curval channel

I have a couple of unrelated thoughts about this.

First:
SLERP is not typically useful for most animation in the same way that LERP isn't typically useful for e.g. location animation: it's not smooth. It can still, of course, be useful in the blocking process, but would rarely be used for final animation. I strongly recommend that SQUAD also be implemented, as it is roughly the SLERP equivalent of spline interpolation. Then in the animation process, SLERP can be used during some stages of initial blocking, and the user can then switch to SQUAD to have smooth interpolation and control over e.g. ease-in-ease-out.

Second:
I like the idea of having more sophisticated interpolation modes for quaternion rotations. But I think if we go this route, we need to think more generally.

Fundamentally, SLERPing (and SQUADing) introduces an inter-dependency between the quaternion channels, turning them into something of an indivisible single unit. Is there a way that we can generalize this concept to other animation in Blender? For example, maybe there are weird interpolation modes where it would be useful to similarly treat e.g. translations as a single unit rather than three channels. How can we handle this in a way that generalizes, and also doesn't break existing workflows?

My initial thought is that rotations already have alternate representations in Blender: the various Euler's, axis-angle, and quaternion. Couldn't this be just another rotation "mode" added to that list? Maybe "linked quaternion" or "unified quaternion" or something like that? Or maybe we could go the other way and use "quaternion" for the new mode and "split quaternion" for the old. Then within the new mode the user can set the curve's interpolation mode to constant, SLERP, or SQUAD interpolation.

Personally, I prefer that to SLERP/SQUAD just being additional interpolation modes on the current quaternion implementation. Because a lot more than just interpolation is changing: the nature of the curves are changing. This would make it explicit that, just like when switching between euler and axis-angle, the nature of what you're working with is changing at a more fundamental level: you won't be manipulating individual channels anymore, you'll be manipulating a single indivisible unit that wholly represents the rotation.

There are still a lot of problems that my suggestion doesn't solve, of course (most or all of which Aligorith has listed above). But I'm hoping this makes sense as a foundation for how users access the functionality. It keeps a clean split between old/new behavior, and it can easily generalize to other similar situations if they arise in the future.

(As a final side-note: for the love of god don't get rid of the current quaternion workflow. It is easily among my favorite things about animating in Blender rather than e.g. Maya/XSI. I am super happy to see someone working on SLERP because it absolutely will be useful, but I strongly, strongly feel that it should be added as an additional feature, not as a replacement. This is very much a trade-off kind of situation. Although SLERP/SQUAD are more perfect mathematically, in practice they do not universally make for a better animation workflow or user experience.)

Jack Andersen (jackoalan) updated this revision to Diff 6523.

OK, after careful study of anim_sys, NLA, and relationship with scene updating, this patch is much more integrated with Blender's timing model.

In addition to SLERP interpolation, SQUAD has also been implemented with new quaternion math to support it in blenlib. interp_qt_qtqtqtqt is the entry to the SQUAD interpolation, exposed to Python as mathutils.Quaternion.squad.

Objects and PoseChannels now carry a lazily-allocated quaternion cache for gathering 4 surrounding F-Curve/Driver evaluated quaternions. These caches are invalidated with the object's AnimData recalc.

The listener for the NLA editor now sets the recalc flag on active AnimData, ensuring the quaternion cache is invalidated appropriately.

bpy.ops.graph.sample_quaternions is a new operator directly based on "Sample Keyframes". Instead of sampling via F-Curve evaluation, it samples using the cached interpolation context.

Jack Andersen (jackoalan) marked an inline comment as not done.Apr 25 2016, 12:23 PM
Jack Andersen (jackoalan) updated this revision to Diff 6525.

had one of the cache tests disabled for debugging, it's re-enabled now

Remove redundant cache-tests where value has already been evaluated

Correct a shadowed variable warning

Fix pose-channel access in NLA evaluation pass

Much less hacky handling within NlaStrip stack.

Interpolation knots are now cached for each NlaStrip, tracked via the strip's name in a GHash map.

Animated influence is interpolated against identity using SLERP.

NlaStrip blending is implemented like so:

  • Replace: override with higher quaternion
  • Add: multiply higher quaternion with lower quaternion
  • Subtract: multiply inverse of higher quaternion with lower quaternion
  • Multiply: same as 'Add'

Transition strips are interpolated with direct SLERP.

Quaternion cache invalidation is now handled in response to NC_ANIMATION, ND_KEYFRAME notifiers.

Notifiers that are responsible for modifying keyframes now include the active object pointer as the note reference, so the exact cache can be invalidated.

Fix in SQUAD quadrangle calculation

Cache now handles 2-quaternion updates correctly

Keyingsets now recognize new rotation modes

Auto-keying functionality confirmed to work in-between interpolation evaluations

Fix SQUAD discontinuities in sample_quaternions operator

Move quaternion cache invalidation notifier handler to catch events from auto-keyframing

Proper handling of PoseChannel recalc

Here's a 3-mode quaternion comparison showing the modes so far (purposefully with long-case quaternion intervals):

https://vimeo.com/164798991

Minor change to make GHash reserve more effective

More portable rounding function in sample quaternions operator exec.

Since actions-to-objects is a one-to-many relationship, keyframe invalidation now includes all Object datablocks as candidates, subject to Action matching.

Ensure quaternions in sample_quaternions operator are normalized before interpolation

Jack Andersen (jackoalan) retitled this revision from Quaternion SLERP Pose Interpolation to Quaternion SLERP/SQUAD Rotation Modes.May 3 2016, 11:50 PM
Jack Andersen (jackoalan) updated this object.

The requested changes have been made, could we get another review?
@Joshua Leung (aligorith) @Campbell Barton (campbellbarton)

NlaStrip blending is implemented like so:

  • Add: multiply higher quaternion with lower quaternion

D1929 adds a code path for gathering quaternions on a per-strip basis. It's specifically designed to sample explicitly-keyed timed points surrounding the evaluated strip-time.
After interpolating with SLERP/SQUAD, the proposed quaternion math is implemented for blending tracks.
This same code path could be made to evaluate direct nlerp via the fcurves, instead of the 2x or 4x evaluations for SLERP/SQUAD

Any chance this could be considered for 2.79, before the long stretch to 2.8? @Joshua Leung (aligorith)

Any news on this?

It seems weird to me that NLA blending does the wrong thing with quaternion rotations, but quaternion rotations are the default mode for pose bones. It seems like a big hole in Blender's animation functionality.

(Edit: Sorry for marking this as "needs revision", I didn't mean to set that)

This revision now requires changes to proceed.May 4 2017, 5:19 PM