Page MenuHome

Implemented basic pre-fetching in Sequencer
Needs ReviewPublic

Authored by Guillaume M (mathers) on Jan 31 2019, 4:38 PM.

Details

Summary

Leverages the ImBuf proxy (implemented in imbuf/intern/moviecache.c despite it being used both in the sequencer and movie clip editor) to prefetch frames.

This is done by putting the next N frames in a queue, and a thread renders them one at a time. Note that this may block the main render thread, since only one simultaneous render is possible.

When the max size of the cache is reached, the frame that is the farthest from the playhead is dropped. For now, this patch continues to pre-render frames payhead to playhead+prefetch continuously, even if the last frames keep getting dropped.

Therefore, prefetch_frames should be set low enough in the options so that all these frames fit in the cache. To keep patches small, I'll improve this in further patches once this is accepted.

To test this, change the movie cache size and prefetch frames options in the user prefs (System tab)

Based on 25772c9e1d2d5239e1f7f83f6ea0c4d7d1f1df4a with D4262 applied

Diff Detail

Event Timeline

Slow down a bit :)

I am already working on this in D3934
I can use some help there if you want to help

@Christopher_Anderssarian was right... There's lack of communication.
Sorry, I can't keep up and I am already paying more attention to Blender then to my work :) Hope, that my boss won't read this...

@Guillaume M (mathers) Just a heads up, this patch is also a bit outdated - I will update it later today. but it will be snapshot during refactoring.

I was using this repo to test latest version: https://github.com/sobotka/blender/branches
branch vse-cache+fix

however when I resumed work on this, there were commits that were in conflict, so I said screw it and discontinued repo.
I can create repo on my github.

Depends of course, if you want to collaborate on this, then we can choose how and who will do what.

Hi @Richard Antalik (ISS), I would be glad to collaborate on this, and I'm sorry your patch was not merged before it got outdated. I think one of the reasons is it was very long (4 kloc !) and hard to review.

I'm hoping that by providing small incremental updates to the Sequencer, I can keep my work merged regularly into the main tree.

I really admire your work in this patch, but honestly I doubt I can build and test it on my machine in a reasonable time, and its size makes it hard to review.

For the sake of clarity I think we should coordinate on T60912. Just say when you're starting to work on a small task (500 loc max) and create an independent well-documented diff where we can review, comment and hopefully merge quickly.

My patch is not outdated it should apply cleanly and partial review is still pretty much possible.
You can try to rewrite it from scratch - I have no power to stop you.

I have to point out, that there are dead ends and wrong ideas in development process and Blender has a release cycle. We should not end up in situation, where during release of major/minor version the situation will be worse then in previous version. By doing incremental changes, this situation can happen if we are not extremely careful. That's why in case of big rewrites we work on branches that if functional and "bug-free", they will be merged in master branch.

By doing incremental changes, this situation can happen if we are not extremely careful. That's why in case of big rewrites we work on branches that if functional and "bug-free", they will be merged in master branch.

Is there a branch that I can checkout in order to test D3934 and review individual commits ?

By doing incremental changes, this situation can happen if we are not extremely careful. That's why in case of big rewrites we work on branches that if functional and "bug-free", they will be merged in master branch.
Is there a branch that I can checkout in order to test D3934 and review individual commits ?

There is this github repo, but initial commit is of course large patch applied. Then my local commit messages suck, and I use it more like save feature...

As core devs don't have too much free time now(ha, they will never have), I will ask brecht if this can be discussed in monday meeting on IRC.
I will request review of (detailed) principle of operation, changes to other modules then VSE (mainly moviecache) and how do they want this patch to be prepared for final review/merge.
Also present current unresolved challenges - mainly a memory leak that I am unable to identify and I can use some guidance in scene copy procedure as we will probably need full copy for rendering scene strips.

Plan was to create this detailed ducumentation this week anyway so I will start working on that right away.

My patch is not outdated it should apply cleanly and partial review is still pretty much possible.

Sorry I mush have confused this patch with D3597.

There is this github repo, but initial commit is of course large patch applied. Then my local commit messages suck, and I use it more like save feature...

Which repo ? sobotka/blender on branch vse-cache ?

Which repo ? sobotka/blender on branch vse-cache ?

That, or use patch D3934: VSE cache with frame prefetching (WIP)
I guess that it is better to use full names for linking tasks and diffs in most cases