Page MenuHome

Video Editor: Proxy clips will play with poor performance if used as proxies, but not when added to the sequencer-timeline
Open, Confirmed, MediumPublic

Description

System Information
Operating system: Windows-10-10.0.17763 64 Bits
Graphics card: Intel(R) UHD Graphics 600 Intel 4.5.0 - Build 26.20.100.7158

Blender Version
Broken: version: 2.81 (sub 12), branch: master, commit date: 2019-09-30 22:00, hash: rB60a827a2a991

Short description of error
Blender rendered proxy files will play with a much worse performance when played as proxies, as compared to adding them directly to the timeline. On my computer the difference is around 12 fps difference on a 100 % proxy file and 10 fps on a 75% proxy file.

Exact steps for others to reproduce the error
(For faster preview performance:
Uncheck View > Show Cache
Set Sidebar > Blendmode: Cross
Sidebar uncheck Cache rendering)

Add a clip to the Sequencer.
Strip > Movie > Set Render Size(if there is a mismatch in resolution, performance will suffer)
Select the strip.
Open the Proxy Settings panel in the Sidebar.
Click the 'Set Selected Strip Proxies'
Check 100% and Overwrite.
Click 'Rebuild Proxies'
Wait for the proxies to be encoded.
In the Sequencer Preview in the 'View Settings' change Proxy Render Size to 100%
Note the frame rate of the clip playing.
On my computer the rate is 19 fps for a 29.97 fps HD clip.

Now open the BL_proxy folder(located in the same folder as your original clip), then open the clip named folder and find the proxy_100.avi file.
Add this file to the sequencer(above the first one)
In the Sequencer Preview in the 'View Settings' change Proxy Render Size to 'No proxy, full render'.
Note the frame rate of the clip playing.
On my computer the rate is the correct 30 fps.

If you in both cases get the correct fps then try with a 4k clip instead.
Free clips can be found here: https://pixabay.com/videos/search/?order=ec

Another odd thing, but maybe related, is changing the Proxy Render Size in the Preview sidebar will also lower the playrate, even though there are no proxies generated.

Comparing the playback rate of a proxyfile used as a proxy file and the same proxy file added to the timeline(gif):

Details

Type
Bug

Event Timeline

Just guessing here, but it seems to me changing the Set Proxy Size, when no proxies are generated, is still changing the resolution, properly mirroring the Resolution % in the Output options:

However values above 50% will actually lower the play-rate performance(properly because of scaling):

That could be some of the reason for a file is playing much slower when it is used as a proxy file.

If this is the case, the question is, if the swapping of generated proxies merged with Resolution % is a good idea, or if it is better to separate the proxy file swapping from the live-rescaling(Resolution %)? Or in other words, the Enum for selecting should be preserved file swapping and the Preview Resolution % should have a separate widget(like in Output options)?

This way users might avoid some of the performance loss when using the current 'Set Proxy Size' function in the 50%, 75% and 100% proxy resolutions. As mentioned in the first post about this problem a HD 29.97 proxy file played with the accurate framerate, when played from the Sequencer Timeline(so there is no need for the Resolution % on top of the file swapping).

Or alternatively check if the image resizing for the VSE could be optimized? Resizing is lowering the playrate in many areas of the VSE, and for sure causing poor playback.

But as I wrote, I just guessing here, so could this be some the reason for the performance drop?

It looks like you @Sergey Sharybin (sergey) back in the day, did some work on the (vse?) proxy system?

If you did, do you know if there are any logic explanations for playing a clip as proxy at 100%(unscaled) delivers a much worse playback rate compared to playing the very same clip directly in the VSE timeline?

I haven't been able to reproduce this bug in blender 2.81 can you please confirm that it still exists.

Peter Fog (tintwotin) closed this task as Resolved.EditedWed, Nov 27, 4:20 PM

I'm not at my computer right now, but iirc the main difference was with proxies rendered at 100℅. Did you try that?

Oh i didn't men to close this, can't reopen it now!?

Wow when posting, I can only choose between Resolved, Invalid or Archived. That must surely be a bug on Phabricator?

I tried 100% and there is no performance loss

Did you try with a clip or a frame rate(ex. 100 fps) that your hardware is not capable at playing at full speed?

Richard Antalik (ISS) reopened this task as Open.Wed, Nov 27, 7:37 PM
Richard Antalik (ISS) lowered the priority of this task from Needs Triage by Developer to Confirmed, Medium.

I could reproduce this, but I haven't opened this because I wanted to confirm cause of this.

But I can confirm that this is bug and the cause is in sequencer.c

/* dirty hack to distinguish 100% render size from PROXY_100 */
if (render_size == 99) {
  render_size = 100;
}

There is another issue in indexer.c, but that I wouldn't consider to be a bug.

/* JPEG requires this */
width = round_up(width, 8);
height = round_up(height, 8);
Peter Fog (tintwotin) closed this task as Resolved.EditedWed, Nov 27, 9:38 PM

NB. In order to test this stuff it is quite important to run the clip in a custom frame rate your hardware is not able to max out, or else you won't see the difference.

On my computer, using 2.82 these are my numbers:
100 % proxy clip is playing 12 fps slower as a proxy as compared to the same clip playing directly in the timeline.
A 75 % proxy is playing a few frames faster as a proxy as compared to the same clip playing directly in the timeline.
A 25 % proxy is playing 10-30 frames faster as a proxy as compared to the same clip playing directly in the timeline.

Since the playback rates are not the same, it seems to me some additional scaling might be going on.

Having the Proxy Render Size at Scene Render Size(btw. needs to be in title case in the Menu) and noting the playback rates changing the Scene Resolution % to 75%, 50% and 25% and comparing them with the same Scene Render Size settings at 75%, 50% and 25% of a clip with no encoded proxy files, the frame rates matches exactly. So the obvious conclusion is that a clip with no encoded proxy files is going through a identical process in the render % and in the proxy preview %.

When playing encoded proxy files as proxies the resolution is of course lowered by the smaller encoded proxy files, but it seems to me, they must be processed by additional proxy preview % down-scaling. Or in other words you only get to see 25% of the encoded 25% proxy file, which explains why the 25% file is playing much faster as a proxy file, as compared to playing the same clip directly in the timeline. And it also explains why a 100% proxy is playing slower as a proxy clip as compared to playing the same clip directly in the timeline, simply because it is a heavier process to scale to a lot of pixels as compared to scale to a few pixels.

The question is if there is additional proxy preview % scaling going on, and if there is, if it really is needed on top of the encoded clips? And if it is considered needed, then maybe the Proxy Preview Resolution % should be separated into a slider like the Render Resolution %. And Proxy Render Size should only deal with file swapping if there are any.

Richard Antalik (ISS) reopened this task as Open.Wed, Nov 27, 9:45 PM

Please don't close this task, it has to be worked on.

Peter Fog (tintwotin) closed this task as Resolved.Wed, Nov 27, 9:46 PM

Oh, man, now I closed it again, this new "thing" is super annoying, that replying auto closes the issue you're commenting... I can't open it again... :-(

Robert Guetzkow (rjg) reopened this task as Open.Wed, Nov 27, 9:51 PM
This comment was removed by Eitan (EitanSomething).

I could reproduce this, but I haven't opened this because I wanted to confirm cause of this.
But I can confirm that this is bug and the cause is in sequencer.c

/* dirty hack to distinguish 100% render size from PROXY_100 */
if (render_size == 99) {
  render_size = 100;
}

The preview_render_size should be replaced with enums and we wont need to use this hack.

Indeed. In best case we could re-use IMB_ functions and enums.

I have done this quickly in the past, haven't been too happy with solution though. I think, that in that time I wasn't aware of DNA versioning so that's why I dropped it.

Can you test this

Peter Fog (tintwotin) added a comment.EditedThu, Dec 5, 8:39 AM

@Eitan (EitanSomething) I'm not entirely sure what you did here, and what you want me to test?
Throwing some files around, it seems that the scaling of files with no encoded proxy files has been removed eg. the quality and framerate is the same no matter what Proxy Render Size is set. (Is this preview scaling to be used in a separate enum?)

When there are encoded proxy files, and the 'Proxy Render Size' is set to 'No proxy, full render' or 'Scene render size' it is actually using the 25% proxies - and not the source file. This means that I'm currently not able to compare the play rates of the proxy files played as proxies and as clips played directly in the time line(No proxy, full render).

I sent the wrong diff

The reason why proxy files at 25% are slower when added to timeline is because the VSE thinks its original size and runs

rectx = (render_size * (float)scene->r.xsch) / 100.0f + 0.5f;
recty = (render_size * (float)scene->r.ysch) / 100.0f + 0.5f;

BKE_sequencer_new_render_data(
    bmain, depsgraph, scene, rectx, recty, proxy_size, false, &context);
context.view_id = BKE_scene_multiview_view_id_get(&scene->r, viewname);

with render_size and proxy_size set to 100 when it should be 25 that is where the slowdown is.

Test files to reproduce Speed difference between using 25% proxy and proxy file added to timeline.
Steps to reproduce

  • Add Blender.mp4
  • Set Frame rate to more than what your computer will be able to play at
  • Turn on proxy for the Movie
  • Build 25% proxy
  • Set proxy render size to 25%
  • Turn on prefetch frame
  • Play Video and take down average FPS
  • Delete the video
  • Add the Proxy_25.avi file
  • Set proxy render size to no proxy, full render
  • Set Frame rate to more than what your computer will be able to play at
  • Play Video and take down average FPS

Average FPS of Proxy_25.avi in the timeline should be lower than using render proxy 25% of Blender.mp4.

@Eitan (EitanSomething)
File you provided played at 90 FPS without proxy and 280 FPS with 25% proxy
Proxy file itself plays at 300FPS

This is because proxy of 1920 x 1060 clip has size of 480 x 272
However 1920 * 0.25 = 480
and 1040 * 0.25 = 265

480 x 272 is different to 480 x 265, therefore clip has to be scaled. This is limitation of MJPEG format.

@Richard Antalik (ISS) @Eitan (EitanSomething)

That clip plays here at 18 fps when there is a resolution mismatch 1060 vs. 1080.
Using Movie Strip > Set Render Size(so render resolution fits source) the playpack rate jumps to 25 fps.
(Disabling Cache and Cache Drawing)

Playing an encoded proxy file at 25% as proxy plays at 65 fps.
Playing the same file directly in the timeline(No Proxy) it plays at 35 fps.