Regression: artefacts in video between Keyframes in VSE #86944

New Issue

Michael Soluyanov · 2021-03-25T14:59:15+01:00

Michael Soluyanov commented

2021-03-25 14:59:15 +01:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce RTX 2060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 456.38

Blender Version
Broken: version: 2.93.0 Alpha, branch: master, commit date: 2021-03-24 17:47, hash: 9ac7946dd0
Worked: version: 2.92 official

Caused by 88604b79b7

Short description of error
Image from video brakes if cut is between keyframes, between cut and next keyframe.

Exact steps for others to reproduce the error

Add video clip with codec H264 - MPEG-4 AVC (part 10) (h264)
Make a cut and set playhead at first frame after the cut
Image will be broken

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: GeForce RTX 2060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 456.38 **Blender Version** Broken: version: 2.93.0 Alpha, branch: master, commit date: 2021-03-24 17:47, hash: `9ac7946dd0` Worked: version: 2.92 official Caused by 88604b79b7 **Short description of error** Image from video brakes if cut is between keyframes, between cut and next keyframe. ![2021-03-25_16h51_12.png](https://archive.blender.org/developer/F9909285/2021-03-25_16h51_12.png) **Exact steps for others to reproduce the error** - Add video clip with codec H264 - MPEG-4 AVC (part 10) (h264) - Make a cut and set playhead at first frame after the cut - Image will be broken

Michael Soluyanov commented

2021-03-25 14:59:15 +01:00

Added subscriber: @crantisz

Philipp Oeser commented

2021-03-26 09:54:25 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2021-03-26 09:54:25 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Philipp Oeser commented

2021-03-26 09:54:25 +01:00

Cannot reproduce here.

Could you share a minimal example .blend file where this happens? (please include the video file used in the upload as well).
Then, for this example file, tell on which frame to make the cut to make this happen.

Cannot reproduce here. Could you share a minimal example .blend file where this happens? (please include the video file used in the upload as well). Then, for this example file, tell on which frame to make the cut to make this happen.

Michael Soluyanov commented

2021-03-26 10:54:44 +01:00

Here is the test file
00013.MTS

And how it goes in current version (a bit laggy but without jittering) and in version: 2.93.0 Alpha, branch: master, commit date: 2021-03-24 17:47, hash: 9ac7946dd0

2021-03-26 12-48-14.mp4

I haven't caught a broken image this time, but still have frame jittering, looks like Blender gets the wrong frames during playback

Example file:

video example.blend

Play this, playback should run without jittering, but it looks like there is some frames removed inside cuts

Here is the test file [00013.MTS](https://archive.blender.org/developer/F9910365/00013.MTS) And how it goes in current version (a bit laggy but without jittering) and in version: 2.93.0 Alpha, branch: master, commit date: 2021-03-24 17:47, hash: 9ac7946dd0 [2021-03-26 12-48-14.mp4](https://archive.blender.org/developer/F9910377/2021-03-26_12-48-14.mp4) I haven't caught a broken image this time, but still have frame jittering, looks like Blender gets the wrong frames during playback Example file: [video example.blend](https://archive.blender.org/developer/F9910391/video_example.blend) Play this, playback should run without jittering, but it looks like there is some frames removed inside cuts

Philipp Oeser commented

2021-03-26 13:28:53 +01:00

Changed status from 'Needs User Info' to: 'Confirmed'

Philipp Oeser commented

2021-03-26 13:28:53 +01:00

Added subscriber: @iss

Philipp Oeser commented

2021-03-26 13:28:53 +01:00

Can confirm.

Caused by 88604b79b7
Will set to High since this is a regression, not sure what the solution is now: insert new IFrames ourselves?

CC @iss

Can confirm. Caused by 88604b79b7 Will set to High since this is a regression, not sure what the solution is now: insert new IFrames ourselves? CC @iss

Richard Antalik commented

2021-03-26 13:59:31 +01:00

Thanks for report, I think this may be fixed by D10644(must rebase first).

At least I got test case now, I was unable to break 88604b79b7 on purpose.

Thanks for report, I think this may be fixed by [D10644](https://archive.blender.org/developer/D10644)(must rebase first). At least I got test case now, I was unable to break 88604b79b7 on purpose.

Richard Antalik commented

2021-03-26 14:50:06 +01:00

Unfortunately D10644 doesn't fix this. But I have some ideas how to fix this without overshooting seeking and therefore affecting performance.

If I won't be successful, I will have to revert the patch.

Unfortunately [D10644](https://archive.blender.org/developer/D10644) doesn't fix this. But I have some ideas how to fix this without overshooting seeking and therefore affecting performance. If I won't be successful, I will have to revert the patch.

Richard Antalik commented

2021-03-29 02:07:31 +02:00

Problem here is, that av_seek_frame(anim->pFormatCtx, -1, pos, AVSEEK_FLAG_BACKWARD) fails to seek to nearest previous I frame.
This is contrary to documentation. See https://ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#gaa23f7619d8d4ea0857065d9979c75ac8

When seeking to frame 0, ffmpeg seeks to packet 1 instead and results in wrong image being decoded.
This wasn't introduced by 88604b79b7. Possibly anim->pFormatCtx->start_time is incorrect in that file?
Same "bug" exists in VLC and older Blender versions where it is impossible to seek to first frame.

FFplay seems to have issues with seeking as well - try with ffplay.exe -seek_interval 1 00013.MTS
You will see errors like [h264 @ 0000015292f9e040] Missing reference picture, default is 2147483647

This bug can be resolved by applying small offset to pFormatCtx->start_time like this: int64_t st_time = anim->pFormatCtx->start_time - 20 * (AV_TIME_BASE / frame_rate);
This will work even for seeking to very first frame, which would fail with preseek. Effect is the same as preseek however. So seeking performance would be affected for all files.

@crantisz, I think that either seeking in ffmpeg or provided sample file is broken and this is not bug in Blender. Not sure which one is it, but I would recommend opening ticket on ffmpeg tracker with ffplay as an example (as I have mentioned above). You can mention this ticket as well.

A way how to resolve this issue is:

Have "preseek", but apply it to stream start time instead.
This "preseek" would be initialized to 0.
When decoding fails, "preseek" value will be increased and decoding for requested frame will be repeated.

This way only broken files will have performance negatively affected.

@crantisz A way to resolve this issue immediately is to build proxies, which will also build timecodes. Timecodes will help with decoding issues like this one. So even if you don't want to use proxies, you can build low quality 25% size which will build very fast.
Though it seems there are small glitches even with timecodes :(

Problem here is, that `av_seek_frame(anim->pFormatCtx, -1, pos, AVSEEK_FLAG_BACKWARD)` fails to seek to nearest previous I frame. This is contrary to documentation. See https://ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#gaa23f7619d8d4ea0857065d9979c75ac8 When seeking to frame 0, ffmpeg seeks to packet 1 instead and results in wrong image being decoded. This wasn't introduced by 88604b79b7. Possibly `anim->pFormatCtx->start_time` is incorrect in that file? Same "bug" exists in VLC and older Blender versions where it is impossible to seek to first frame. FFplay seems to have issues with seeking as well - try with `ffplay.exe -seek_interval 1 00013.MTS` You will see errors like `[h264 @ 0000015292f9e040] Missing reference picture, default is 2147483647` This bug can be resolved by applying small offset to `pFormatCtx->start_time` like this: `int64_t st_time = anim->pFormatCtx->start_time - 20 * (AV_TIME_BASE / frame_rate);` This will work even for seeking to very first frame, which would fail with preseek. Effect is the same as preseek however. So seeking performance would be affected for all files. @crantisz, I think that either seeking in ffmpeg or provided sample file is broken and this is not bug in Blender. Not sure which one is it, but I would recommend opening ticket on ffmpeg tracker with ffplay as an example (as I have mentioned above). You can mention this ticket as well. A way how to resolve this issue is: - Have "preseek", but apply it to stream start time instead. - This "preseek" would be initialized to 0. - When decoding fails, "preseek" value will be increased and decoding for requested frame will be repeated. This way only broken files will have performance negatively affected. @crantisz A way to resolve this issue immediately is to build proxies, which will also build timecodes. Timecodes will help with decoding issues like this one. So even if you don't want to use proxies, you can build low quality 25% size which will build very fast. Though it seems there are small glitches even with timecodes :(

Richard Antalik commented

2021-03-29 05:22:31 +02:00

@crantisz I have done some research, looks like this exact problem is already reported in https://trac.ffmpeg.org/ticket/1607. Unfortunately it doesn't look like it will be resolved anytime soon.
But there is no need for reporting this.

@crantisz I have done some research, looks like this exact problem is already reported in https://trac.ffmpeg.org/ticket/1607. Unfortunately it doesn't look like it will be resolved anytime soon. But there is no need for reporting this.

blender-admin commented

2021-03-30 02:52:20 +02:00

This issue was referenced by 6c33d3d01b

This issue was referenced by 6c33d3d01b6237cbe854b1d6ca54ac9680bb0563

Richard Antalik commented

2021-03-30 03:03:29 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Richard Antalik closed this issue

2021-03-30 03:03:29 +02:00

Richard Antalik self-assigned this 2021-03-30 03:03:29 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Regression: artefacts in video between Keyframes in VSE #86944