Video Sequencer Cache
User level design
On user level cache system should follow zero configuration principle: the video playback and tweaking should be as fast as possible without user spending time on fine-tuning per-project settings.
The only settings which user should be interacting with are in the User Preferences:
- In-memory cache size limit
- Disk cache size limit
- Disk cache folder
These settings are set once in the User Preferences and are used by all sequencer projects. The default values are based on the minimal hardware requirements.
Levels of caching
For best editing and playback performance multiple levels of cache are needed.
The most important ones are:
- Cache of strip input (colormanaged using general Blender color space rules)
This allows to faster, without lag, move image strip in time, adjust strip settings like transform, by avoiding need to re-read file on every modification. Lets call this cache level STRIP_INPUT.
- Cache of the final sequencer result.
This allows to have realtime playback after the sequencer got cached. Lets call this cache level SEQUENCER_FINAL.
The simplified flow of the image from disk to the artist is presented in the following diagram:
Need to think about whether having strip output cache is helpful. If the stack rendering is fast, having extra levels of cache will have negative affect due to less final frames fitting into the memory.
Cache levels are to utilize reference counting as much as possible. For example, when having single Image Strip without modifications set up in the strip the final sequencer frame in the SEQUENCER_FINAL cache is to reference the image from STRIP_INPUT cache. This allows to minimize memory footprint, playback performance for the story boarding type of tasks performed in the sequencer.
The following example visualizes cache frame referencing in the following scenario:
- Sequencer have single Image Strip using HappyFroggo.png as an input. The strip has length of 4.
In Blender terms, the cache contains a single copy of ImBuf created for HappyForggo.png. All the sequencer cache entries are referencing this ImBuf for lowest possible memory footprint.
In a typical video editing scenario an artist views the sequencer result in a rather small area of the screen layout:
This behavior can be exploited in the following way: the sequencer processing and caching can happen in the lower resolution. This is something what current proxies design is solving, but does in the fully manual manner.
There is a possibility to make proxies behavior more automatic, by performing downscale on an image after it was read from disk, but before it gets put to the STRIP_INPUT cache. Default behavior could be something like:
- Use closest power-of-two downscaling (similar to mipmaps)
- The target resolution is 50% of the window resolution, but no more than 1080p.
In order to support workflows where an artist needs to investigate in a close-up manner the final result, there will be a display option Best Quality (but defaulting to Best Performance). This could fit into existing Proxy Display Size menu.
In the future more automated input resolution selection is possible to be implemented. For example, it is possible to automatically switch to the
Best Quality mode when zoom-in is detected.
Image scaling with a power-of-two scale factor can be implemented very efficiently using threading and vectorization.
On a performance aspect, for image sequences such scale down will be an extra computation, which will only pay off if effects/transformation is used.
For the movie files, this step will actually make things faster because it is required to convert color spaces (happening in sws_scale), which is not
threadable. The scale down will be done together with color space conversion, which is expected to give better performance compared to the current state of the sequencer playback.