Page MenuHome

Video Sequencer: Very low framerate
Closed, ResolvedPublic

Description

System Information
Operating system: Windows-10-10.0.16299 64 Bits
Graphics card: GeForce GTX 970/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 390.65

Blender Version
Broken: version: 2.81 (sub 4), branch: master, commit date: 2019-08-30 09:58, hash: rBf033fa0fbcfb
Broken: version: 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: rBf6cb5f54494e
Half-Broken: version: 2.79 Stable commit date: 2017-09-11 10:43, hash: 5db8ac9
Worked: version: 2.76 Stable commit date: 2015-10-11 06:55, hash: 48f7dd6

Short description of error
On recent releases the video sequencer is very slow at playing and scrubbing. Much more than older releases were.
Here I provide a benchmark:
2.81: average 12-15fps
2.80: average 12-15fps
2.79: average 20-24fps
2.76: stable 24fps

Exact steps for others to reproduce the error

  • Download the attached file:
  • Unzip
  • Inside the folder, you have 4 .blend files named as their respective blender versions
  • Open each one with the correct blender version.
  • Play to see the framerate difference.
  • You must also scrub in the timeline to see the speed difference between the versions.

NB: I've done the framerate benchmarks with the four versions of blender opened simultaneously.

Developer Note
Let's use this task to focus on the difference between 2.79 default VSE settings and 2.80. The regression from 2.76 to 2.78 likely involve too many changes to track down. But the VSE default settings is something quite tangible.

Event Timeline

Félix (Miadim) updated the task description. (Show Details)

confirmed. The sequencer performance is going worse version to version

I believe I notice this in the past, but the issue was more a different between the default settings we have in 2.79 and 2.80.
For instance if I open your 2.79 file in 2.80, its performance is similar in 2.79 and 2.80.

Could you run in your system and confirm if you get similar performance results in both 2.79 and 2.80 with the exactly same .blend?

Hello!
To go further with this test, I've modified the 2.76 file, I've set the frame rate to 120 fps. The I reset to factory default, I open the file, I hit play and I screenshot in the middle of the sequence (to avoid the cache effect). I only test the 2.76 file and not the other anymore.
Here's the results :
2.77 : 60 fps
2.79 : 30 fps, but the scrubbing (especially backward) is verrrrry sluggish.
2.80 : 30 fps, but the scrubbing (especially backward) is verrrrry sluggish. I've tried with ou without the cache, the first read frame rate stays the same.

One more thing. If I try to recreate the test file with blender 2.80 from scratch, I've got the same very low frame rate as Félix (19 fps see attached screenshot). 2.80 seems to "corrupt" file made from scratch with it.
We might have two different issues there.

Félix (Miadim) added a comment.EditedSep 17 2019, 5:44 PM

@Dalai Felinto (dfelinto) I think I can tell I have same performances with the 2.79.blend file on Blender 2.80 and 2.79b.
I doesn't manage to reach a stable 24fps and is around 19-22fps.
Also, with 'Cache Final' enabled, the playback framerate is slower.

Félix (Miadim) added a comment.EditedSep 17 2019, 5:54 PM

I made some other tests and I found out that the performance drop happened between 2.78a (24fps stable) and 2.78b (low: around 20fps -> no real difference against 2.80).

Dalai Felinto (dfelinto) lowered the priority of this task from Needs Triage by Developer to Confirmed, Medium.Sep 18 2019, 1:23 AM

@Richard Antalik (ISS) can you take a look at this? Not the regression from 2.76 to 2.78. But the difference between the default VSE settings in 2.80 and 2.79 that makes a video "by default" to play slower in 2.80. It must be one of the default settings that are different.
Maybe the reporter (@Félix (Miadim)) can help nailing this down as well? You should be able to do it without any coding.

@Richard Antalik (ISS) can you take a look at this? Not the regression from 2.76 to 2.78. But the difference between the default VSE settings in 2.80 and 2.79 that makes a video "by default" to play slower in 2.80. It must be one of the default settings that are different.

Last time I checked, 2.80 was actually a bit faster then 2.79... I can check again just to be sure.

As for slow performance:
Short answer: Use proxies (and/or prefetching [preferably, and report bugs :}])
And disable AV sync, that feature could be called random frame player...

Long answer:
Narrowing this down is quite hard, because playback rate in is affected by UI and other stuff unrelated to VSE pipeline.

If scrubbing is area mostly affected, that implies regression in FFMPEG implementation, which I have suspected to be sub-optimal (read not perfect). I am not very familiar with that code. I have looked at some data structs we use and stepped through frame retrieval a few times, but that's it. If I commit to resolve this bug it would take me probably a month to find most significant performance issues and fix them. Knowing how my planning worked in the past, multiply that number by 2 and we have rough estimation.

I would suggest perhaps thinking of audit from FFMPEG dev if we don't have free senior staff to look at this and this is super hot issue.
Personally I didn't consider this a hot issue, because we can deal with this by using proxies (and disk cache in future). If you would do anything more than simple cuts, you will need features like disk cache anyway until we have faster image processing, which I do consider to be much more important.

Another feature that may help with playback speed(not much scrubbing though) is prefetching, as it decouples rendering to a separate thread, thus eliminating non-VSE influence on performance. This feature is fresh though, and last time I checked there were some bugs (that weren't there during development obviously...)

With prefetching enabled and render size set to 50% I was able to achieve 24 FPS without proxies. This is on very poor machine and debug build.
It should be noted, that performance boost is just a side effect of prefetching, it's main purpose is performance consistency.

Thanks for the reply @Richard Antalik (ISS).

Last time I checked, 2.80 was actually a bit faster then 2.79... I can check again just to be sure.

Please do. The comparison is to load the same file in 2.79, then in 2.80 without any other change and just playback. Now if you save the 2.79 file and open it in 2.80 it should play fast, which is why I'm suspecting some of the default options.

Did test with sample video, all default settings:

2.77: 18 FPS
2.79: 18 FPS
2.80: 35 FPS

with provided blend file:

2.77: 60 FPS
2.79: 60 FPS
2.80: 60 FPS

@Dalai Felinto (dfelinto)
I can tell, that default 2.80 speed is lower than "theoretical maximum" because of D4737: Sequencer: use Alpha Over blend mode by default
I didn't knew at the time, that it will cause regression. Question is, that if we would even want to revert such change - ease of use vs performance.

There is a quite hacky way to improve performance, by checking, if output of the sequence can be transparent - check for codec, opacity, transformation, modifiers and things like that.

There is a quite hacky way to improve performance, by checking, if output of the sequence can be transparent - check for codec, opacity, transformation, modifiers and things like that.

This is per strips per frame, no? Not an overall video/VSE setting. I would say that if there is only a regular video with no other image on top of it, it should fallback to some opacity option. From the user point of view it makes no sense to have such a massive performance impact when they are using single movie strips.

If there is no other solution I would say it is reasonable to revert (if @Jacques Lucke (JacquesLucke) / @Brecht Van Lommel (brecht) are ok with it).

There are several things which can give a poor playback rate ex. if the footage resolution doesn't match the render resolution setting or the footage fps/colorspace doesn't match the render fps/colorspace settings.
AV-sync is also causing a severe drop in fps, but if you're not just organizing or previewing stuff in the VSE, but actually editing stuff for an emotional impact, it is crucial to have the audio in sync and reliable fps at all times while editing.
Uncached material, playback while caching can also cause a drop in fps.

On my computer using the uploaded files 2.79 gave me a playback of 10 fps and 2.80 gave me a playback at 2-3 fps. Using scene strips 2.79 gave me 24 fps and 2.80 10 fps.

@Richard Antalik (ISS) can you explain how D4737 is causing a regression again?
Personally, I think using some method to determine if a strip cannot have transparency is a perfectly valid thing to do to increase performance. If implemented well, this does not need to be a hack imo. However, I don't know the sequencer code, so...

I don't have a strong opinion on D4737, but I feel like the issue is somewhere else.

@Jacques Lucke (JacquesLucke) @Dalai Felinto (dfelinto)
Some blend effects do have early out function assigned. For example cross will skip blending if opacity is 0 or 1 falling back to input A or input B. This can be done, because cross uses constant opacity value to mix all channels of picture (including alpha)

Alpha over(+under and perhaps other blends also) however does mix RGB channels of input A and B based on alpha value of individual pixel of ether A or B (under vs over).
This means, that it has to read through all pixels of at least one input.

If we can determine if output of sequence can be transparent, we can do early out.

To do this for movies and images, we have to explicitly define alpha output capability for each codec (hacky IMO) and also consider all settings of strip and sequencer that could affect this (not maintenance free, so a bit messy if not hacky)

@Peter Fog (tintwotin) I have tested only 2.76 file
there may be some non-ideal conversions from 2.79 to 2.80 but that should apply to 2.76 file also.

I think automatically figuring out of the image has alpha or not is something we should try to do. This information is available in ImBuf (see BKE_image_has_alpha for an example), but it may not be filled in correctly for video codecs or set for sequencer evaluation.

The implementation of this is non-trivial, and I think it would be ok to change the default back to cross, and make the better default + required optimizations a To Do task.

Last night @Richard Antalik (ISS) and I did some tests(on the vse chat: https://blender.chat/channel/vse?msg=nMDvchLamuWnAorGZ ) concerning proxy formats, but a few elements which might lower the fps came up during that process:

Bug report: T70227
Blendmode: Alpha Over will give a lower fps than Cross(gif):

Bug report: T70228
A resolution mismatch between footage and render resolution will give a lower fps(gif):

Bug report: T70229
Having Draw Cache On will give a lower fps(gif):

Using files encoded with this command line(for ex. 25% proxies) by @Nathan Lovato (gdquest) :

ffmpeg -hwaccel auto -y -v quiet -i input_file -pix_fmt yuv420p -g 1 -sn -an -vf colormatrix=bt601:bt709 -vf scale=ceil(iw*0.25/2)*2:ceil(ih*0.25/2)*2 -c:v libx264 -crf 25 -preset faster output_file -threads 0

Will result in a faster encoding and image quality than the built-in jpeg proxies.

This command line:

-vcodec mpeg2video -pix_fmt yuv422p -intra -qscale 1 -qmin 1 -an -sn -threads 0

Will produce proxies with a faster playback and encoding than built-in jpeg proxies at 100%, but seeking is only working properly with cache on(gif):

If the current proxy should stay, then maybe the dead slow proxy encoding can be improved with using the -threads 0 setting or some of the new hardware accelerated ffmpeg settings: https://trac.ffmpeg.org/wiki/HWAccelIntro -hwaccel auto

When using proxies, make sure that there is no lower fps thanks to mismatch between proxy-footage resolution and render resolution.

Setting Blendmode to Cross do have an effect on scene strips, however the playback rate is still far below the accurate playback rate for scene strips on 2.79(gif):

The reason for closing this bug report is changing Blend Mode from Alpha_Over to Cross, but as mentioned above there are several additional reasons for poor playback rates in 2.80, as compared to older versions.

Some of these things are mentioned in the comments above, and they are not solved by changing the Blend Mode.

This report was closed, because changing default blend mode pretty much resolved performance difference in context of this report.

If there are other regressions such as T70229: VSE Low Playback Rate: Show Cache On will cause a lower fps they should be reported separately

As shown in the gifs above: https://developer.blender.org/T69334#782384 will Resolution Mis-match and Show Cache: On - will result in ca. minus 6 fps.

In other words, each of those equals the negative impact on the playback rate of the blend mode, so they should be taken as seriously as the Blend Mode playback degression, imo.