Retiming

On user level retiming is used to change playback speed of strips.

In summary, this is done in 2 ways: - Linear interpolation for changing speed or stopping playback - Ramp curve to make transition between 2 speeds

Retiming data

Retimed strip range, also called segment is deliminated by retiming keys SeqRetimingKey. Data is owned by Strip.

Member strip_frame_index is retiming key position in number of frames from the strip start. This value is in time domain of strip content.

Member retiming_factor represents a frame of content. This value has range from 0 to 1, where 0 is first content frame and 1 is last content frame.

Member flag defines key type, remaining members are used as backup to simplify transition rebuilding.

Note: strip_frame_index of last retiming key defines strip length.

Time domains

While linear interpolation sounds very simple, reality is bit more complicated. Strip length and offsets are stored in content time domain. This is to preserve playback speed of movies, when scene frame rate is changed, Strip::media_playback_rate is used to scale strips to correct proportions.

Since retiming key position is relative to strip, their position must also scale with the strip.

In this image you can see a sample strip in 60fps timeline which retimes a 30fps movie.

Middle layer - "Strip in content domain" is what is stored in .blend file, while bottom layer - "Strip in timeline domain" is presented to the user and interacted with.

This means, that there is a lot of code converting one domain to another, which may easily be a source of bugs.

In addition, this is why strip_frame_index is not plain integer. It would be sometimes impossible to place retiming key to exact timeline frame, or retiming could produce different result, if timeline frame rate changes.

Retiming key types

Default type

Is evaluated by linear interpolation between 2 retiming factor values.

Freeze frame

These are same as default type, but both key have same retiming factor value. However having specific type allows the code to differentiate this key type from default one.

Speed transition

Speed transitions always consist of 2 retiming keys between 2 retimed segments. These are evaluated by finding Y value of circular arc tangential to 2 lines defined by neighboring segments.

This makes it possible to resize transtion while frame mapping of neighboring segments remains unchanged and at the same time the transition itself remains smooth.

However this also means, that changing speed of any neighboring segment requires transition to be rebuilt. For this, original_strip_frame_index and original_retiming_factor are used.

Evaluation

Once value of retiming factor is evaluated from retiming data, this is simply multiplied by content length to get content frame.

This image is an example of strip with 2 linear segments and speed transition. Dashed line signifies how normal playback at 1x speed would be evaluated.

Sound retiming

Sound retiming requires special considerations, because sound playback is handled by external library - Audaspace. This library does not support scheme with frame interpolation, but it requires a buffer with playback speed defined on per-frame basis. For this functions BKE_sound_set_scene_sound_pitch_constant_range and BKE_sound_set_scene_sound_pitch_at_frame are used to make this as fast as possible.

RetimingRangeData is a helper struct that holds vector of RetimingRange which defines a speed for each retimed segment of sound strip. In case of speed transition, speed of each frame is evaluated. Audaspace then further interpolates playback speed between individual "samples" it feeds to actual sound device buffer.

Therefore sound retiming is not exact and there could be minor deviations.

Meta strip retiming

It is possible to retime meta strips while strips inside of meta strip are retimed.

For image producing strips, this works simply by first evaluating meta strip retiming to its content frame range following by another retiming evaluation of child strip.

With sound strips this is not possible and RetimingRangeData produced by meta must be combined with RetimingRangeData produced by sound strip.