Page MenuHome

Importing any video to VSE results in audio and video having different lengths
Open, ConfirmedPublic

Description

System Information
Xubuntu 16.04, I am using a Dell and an Asus laptops. I don't think my graphics card matters for Blender VSE.

Blender Version
Broken: All later versions: 2.78 and 2.79 (obtained form the website) (although have not tested with 2.77)
Worked: Blender 2.76b f337fea (obtained form the website)

Short description of error
Importing any video to Blender VSE results in different video and audio lengths, with video usually being shorter than the audio. The fps chosen matches the video fps. Same video file is imported correctly in Blender 2.76b.

In the vast majority of cases (basically, a careful way of saying always) fiddling with custom fps settings doesn't solve the problem, the difference remains. The issue was reproduced by Blender users on the #blender IRC channel.

Exact steps for others to reproduce the error

This is the test project with a video file. The video was shot using a standard camera app of a Samsung Galaxy 4mini phone.
https://louigiverona.com/files/var/TestProject/

The video is 95 Mb (it is only 50 sec), wanted to make it somewhat longer, since for very short videos the difference might not be that large, although typically still there, even if by 1.
Since I put the file on my server, I ask to not download it if you are not planning to work on the bug, so that I don't get a huge bill by the end of the month :)

Results of the test:
Lengths in 2.76b:
audio 1404
video 1404

Lengths in 2.79:
audio 1404
video 1391

The problem is reproducible for me in the vast majority of cases, with all sorts of video files. This includes screencasting videos made with the SimpleScreenRecorder and videos downloaded from YouTube.

All videos from any of the cameras I own, from three mobile phones (Samsung Galaxy 4mini, Huawei P8 Lite, Samsung A3) and from an expensive Nikon camera - all videos without exception always end up being opened with different lengths in 2.78 and 2.79. At the same time, they are all in complete sync in Blender 2.76b, with audio and video always having the exact same length.

I have asked several people to reproduce the issue in Blender 2.79 and all the people I asked were able to reproduce the problem immediately, using their own video files.

Another way to reproduce is to download a video from YouTube. Here is a random video I watched, downloaded and tested: https://www.youtube.com/watch?v=FNLoxaSSQ2A

Here, inspite of the video being long, the difference is smaller - but still there. The video is at 29.970030 frame rate.

Blender 2.76b:
audio 200391
video 200391

Blender 2.79:
audio 200391
video 200389

It seems that the ability of later versions of Blender to automatically detect the imported video fps is at play here. Many videos will not have exactly 30 fps, for example the frame rate of the video I supply with my test project is 30.003730. This difference is small enough to be neglected, but as JA12 in the IRC channel found out, Blender seems to try to convert this fps to some exact number and fails because of the length of the field, however, he was not certain this is really the cause. He was able to immediately reproduce an issue with my project and by importing some other video and seeing the same problem. Everyone on IRC were able to reproduce the problem using my TestProject file with the video also. JA12 was also able to reproduce the problem with the latest development version of Blender.

I will do my best to provide additional information is necessary.

Details

Type
Bug

Event Timeline

Having a look at the metadata; the audio is longer than the video:
Video: Line 23: 46 s 361 ms
Audio: Line 50: 46 s 784 ms
Blender seems to be setting the correct frame rate there's just a bit of extra audio.

1General
2Complete name : C:\Users\admin\Downloads\T54655\20180415_111352.mp4
3Format : MPEG-4
4Format profile : Base Media
5Codec ID : isom (isom/3gp4)
6File size : 94.7 MiB
7Duration : 46 s 784 ms
8Overall bit rate : 17.0 Mb/s
9Encoded date : UTC 2018-04-15 09:14:40
10Tagged date : UTC 2018-04-15 09:14:40
11
12Video
13ID : 1
14Format : AVC
15Format/Info : Advanced Video Codec
16Format profile : High@L4
17Format settings : CABAC / 1 Ref Frames
18Format settings, CABAC : Yes
19Format settings, RefFrames : 1 frame
20Format settings, GOP : M=1, N=30
21Codec ID : avc1
22Codec ID/Info : Advanced Video Coding
23Duration : 46 s 361 ms
24Bit rate : 17.0 Mb/s
25Width : 1 920 pixels
26Height : 1 080 pixels
27Display aspect ratio : 16:9
28Frame rate mode : Variable
29Frame rate : 30.000 FPS
30Minimum frame rate : 29.499 FPS
31Maximum frame rate : 30.529 FPS
32Color space : YUV
33Chroma subsampling : 4:2:0
34Bit depth : 8 bits
35Scan type : Progressive
36Bits/(Pixel*Frame) : 0.273
37Stream size : 94.0 MiB (99%)
38Title : VideoHandle
39Language : English
40Encoded date : UTC 2018-04-15 09:14:40
41Tagged date : UTC 2018-04-15 09:14:40
42mdhd_Duration : 46361
43
44Audio
45ID : 2
46Format : AAC
47Format/Info : Advanced Audio Codec
48Format profile : LC
49Codec ID : mp4a-40-2
50Duration : 46 s 784 ms
51Source duration : 46 s 786 ms
52Source_Duration_FirstFrame : 2 ms
53Bit rate mode : Constant
54Bit rate : 128 kb/s
55Channel(s) : 2 channels
56Channel positions : Front: L R
57Sampling rate : 48.0 kHz
58Frame rate : 46.875 FPS (1024 SPF)
59Compression mode : Lossy
60Stream size : 704 KiB (1%)
61Source stream size : 704 KiB (1%)
62Title : SoundHandle
63Language : English
64Encoded date : UTC 2018-04-15 09:14:40
65Tagged date : UTC 2018-04-15 09:14:40
66mdhd_Duration : 46784

I am not sure how is this possible, unless this is really how all cameras and videos in general work. And somehow Blender 2.76b does everything correctly, whereas the "correct" frame rate of later versions results in audio and video being out of sync.

Yes, this in general is how all cameras and video recorders work, these days. Why? Gone are the days when you needed a bit of magnetic tape to move across a tape-head at a constant rate, or a constant 33 1/3 rotations per minute to faithfully reproduce video or sound.
All readily-available video and audio is compressed in some form, to either make it fit into a smaller distribution medium, or stream "faster" through phone lines, to save someone time, or money.
Before you load a file for editing in Blender, consider using a program (such as Handbrake) that can convert between variable-rate and constant-rate frames-per-second formats. You might also try FFMPeg, which is used by both Blender and Handbrake to do their video conversions for exporting to different formats. Handbrake is a lot easier to work with, than FFMPeg, though.
Other editing suites do similar conversions, without telling you about it, and still take just as long to process.

Stephen, this is all fine, but somehow Blender 2.76b was doing it correctly, without requiring me to use Handbrake or any other converter.

"Other editing suites do similar conversions, without telling you about it, and still take just as long to process."

Well, I tried Kdenlive. I imported the same file - there was no processing of any sort, it just got added to the timeline, everything perfectly in sync.

Since the audio length in your reports is the same between 2.76b and later versions, and it's the video that has changed length, I would argue that 2.76b was getting the video length wrong somehow. It may even be a difference in ffmpeg / libavcodec rather than in blender itself, since the library versions also change between blender releases.

FWIW, my local builds are based around 2.78ish, and I get matching audio and video lengths with the cameras I use:

  • Canon S95
  • Canon EOS M
  • TCL SVC 200

None of these cameras use variable frame rate. No editing system can perfectly match audio and video if the video frame interval is not consistent. Mobile phones and OBS are notorious for creating videos with inconsistent frame intervals due to them being busy doing other things with the CPU.

I'm not at my main machine right now, next time I am I'll try some clips in the 2.78 and 2.79 releases but I don't recall having any problems.

Okay, so I just took the test video which I supplied, used Handbrake to convert it to 30 constant fps. Now Codec info says exactly 30 fps.

Opening it in Blender 2.79:
audio: 1405
video: 1391

Project fps is at 30.

Opening it in Blender 2.76b:
audio: 1405
video: 1405

Project fps is at 30.

So, even with the constant frame rate the problem seems to persist.

@Louigi Verona (LouigiVerona) Try importing your converted video in 2.79b (or preferably a daily build) and see if the problem persists.

I just downloaded your sample... it only contains 1391 video frames (checked by transcoding it with ffmpeg and looking at the frame count at the end).

So any setup that makes you a video strip 1405 frames long is adding some frames somewhere to compensate.

Sybren A. Stüvel (sybren) triaged this task as Confirmed priority.Jun 17 2018, 10:16 AM

Recently I had a problem with audio exported from Ardour being longer than the same audio from OBS video file (both at 48kHz sampling rate both PCM streams).
It broke both in Blender and Kdenlive in similar ways. I think it could be a problem in one of the libraries these programs use to parse audio stream metadata or something.
Not sure if it's related though.

@Olly Funkster (Funkster) - Could your result be because ffmpeg is not compensating correctly for some dropped frames in the stream?

If the same file plays fine in VLC or another media player I guess it's a problem in Blender or some libraries Blender use. I know that editing video is more complex than simply playing it back, but I don't see a reason why it should make the A/V sync break. It's not what I'd expect to happen in a video editor.

@Tobiasz Karoń (unfa) VLC is pretty clever, and if the file contains information about a dropped frame or an altered presentation time, you can bet that VLC is capable of interpreting that correctly and adjusting the timing to stay in sync. Blender just takes the average frame rate, and the index of the frame you're on, and calls it a day.

Hmm, maybe that's the root of the problem? But in such case - could transcoding (or just remuxing) the file to a constant FPS file could help?

@Olly Funkster (Funkster) If you read the info I have provided above, this doesn't add up. Blender does everything correctly in previous versions, and so do Kdenlive and even OpenShot.

My forced workaround is to use an old version of Blender for editing, but then render it through the new version of Blender. Basically, I can just import all the files through the old version of Blender I mentioned, and once they are imported, this new Blender version can open the files normally and work with them.

So, the problem is very clearly in the change of file importing into VSE. I am confused as to why this demonstrable fact is still being disputed.

Any change in behaviour is in FFMPEG, not in the VSE code itself (which just takes the frames and the playback rate data from ffmpeg as it has always done).

But the real source of the error is your video file, whose length, frame count, and playback rate do not add up, and that is not blender's responsibility to fix - given the extremely limited (i.e. near-zero) amount of coding resource available for VSE development, dealing more gracefully with non-compliant video files is about as near to the bottom of the priority list as it's possible to get.

If you are able to develop a fix for your edge case that doesn't break anything else, great. Otherwise I would suggest that some kind of transcoding / editing of frame rate data / change to the way your source video is generated is your only option at this point.

Good luck! :o)

I respectfully disagree. Please, see comment https://developer.blender.org/T54655#494818 where I have transcoded a file to have audio of exact same length as video, and the import into Blender still failed. And yet it works completely fine in an old version.