Audio length inconsistency after frame rate change #94149

New Issue

Georg K · 2021-12-16T12:30:53+01:00

Georg K commented

2021-12-16 12:30:53 +01:00

System Information
Operating system: Linux-5.14.18-1-MANJARO-x86_64-with-glibc2.33 64 Bits
Graphics card: AMD VERDE (DRM 2.50.0, 5.14.18-1-MANJARO, LLVM 13.0.0) AMD 4.5 (Core Profile) Mesa 21.2.5

Blender Version
Broken: version: 3.1.0 Alpha, branch: master, commit date: 2021-12-14 21:50, hash: 644eb68524
Worked: 2.93.0

Short description of error
Importing an audio file into the video sequencer with the frame rate set to 25 results in an audio strip that ends at e.g. frame 6012. But after switching the frame rate to 24 and then back to 25, the audio strip ends at frame 6013.

Exact steps for others to reproduce the error

Start blender with video editing default
set frame rate to 25, switch sequencer to display frames instead of seconds
use add sound to add an mp3 file (at least 4 mins long)
check the location of the last frame of the audio strip
switch frame rate to 24 and back to 25
check the location of the last frame of the audio strip again - it is different than before.

Blender 2021-12-17 18-44-22.mp4

**System Information** Operating system: Linux-5.14.18-1-MANJARO-x86_64-with-glibc2.33 64 Bits Graphics card: AMD VERDE (DRM 2.50.0, 5.14.18-1-MANJARO, LLVM 13.0.0) AMD 4.5 (Core Profile) Mesa 21.2.5 **Blender Version** Broken: version: 3.1.0 Alpha, branch: master, commit date: 2021-12-14 21:50, hash: `644eb68524` Worked: 2.93.0 **Short description of error** Importing an audio file into the video sequencer with the frame rate set to 25 results in an audio strip that ends at e.g. frame 6012. But after switching the frame rate to 24 and then back to 25, the audio strip ends at frame 6013. **Exact steps for others to reproduce the error** - Start blender with video editing default - set frame rate to 25, switch sequencer to display frames instead of seconds - use add sound to add an mp3 file (at least 4 mins long) - check the location of the last frame of the audio strip - switch frame rate to 24 and back to 25 - check the location of the last frame of the audio strip again - it is different than before. [Blender 2021-12-17 18-44-22.mp4](https://archive.blender.org/developer/F12757730/Blender_2021-12-17_18-44-22.mp4)

Georg K commented

2021-12-16 12:30:53 +01:00

Added subscriber: @georgK

Pratik Borhade commented

2021-12-17 13:47:23 +01:00

Added subscriber: @PratikPB2123

Pratik Borhade commented

2021-12-17 14:18:14 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Pratik Borhade commented

2021-12-17 14:18:14 +01:00

Hi, thanks for the report. Can confirm. Note that problem does not appear with every audio file.

I can share audio file here but little unsure about policy. Added a video (see the task description) to show the exact issue

Also this is a regression. Works fine here on 2.93

(Original audio file is of 2677 sec)

Hi, thanks for the report. Can confirm. Note that problem does not appear with every audio file. I can share audio file here but little unsure about policy. Added a video (see the task description) to show the exact issue Also this is a regression. Works fine here on 2.93 (Original audio file is of 2677 sec)

Richard Antalik commented

2021-12-18 03:01:09 +01:00

Added subscriber: @iss

Richard Antalik commented

2021-12-18 03:01:09 +01:00

This is same issue as #84027. There have been ongoing discussion, but I think I can commit fix for it. In meanwhile I will merge this report.

Richard Antalik commented

2021-12-18 03:01:19 +01:00

Closed as duplicate of #84027

Richard Antalik closed this issue

2021-12-18 03:01:19 +01:00

Pratik Borhade commented

2021-12-18 03:47:03 +01:00

Hi @iss , may be you are right but this is not the case for 2.93. And the other issue (#84027) even appear on previous versions.

Underlying issue might be same but please check the report again. This appears to be a regression :)

Hi @iss , may be you are right but this is not the case for 2.93. And the other issue (#84027) even appear on previous versions. Underlying issue might be same but please check the report again. This appears to be a regression :)

Richard Antalik commented

2021-12-18 04:08:45 +01:00

Changed status from 'Duplicate' to: 'Needs User Info'

Richard Antalik reopened this issue

2021-12-18 04:08:45 +01:00

Richard Antalik commented

2021-12-18 04:08:45 +01:00

Ah sorry, missed that point, but In that case I am not able to reproduce this issue. Is length of sound key here? I would definitely appreciate sample sound file or more exact steps.

Technically fixing #84027 would resolve this but issue here may be that sound was imported with incorrect length to begin with. Can you check which length is correct and which is not?

Ah sorry, missed that point, but In that case I am not able to reproduce this issue. Is length of sound key here? I would definitely appreciate sample sound file or more exact steps. Technically fixing #84027 would resolve this but issue here may be that sound was imported with incorrect length to begin with. Can you check which length is correct and which is not?

Pratik Borhade commented

2021-12-18 04:46:37 +01:00

issue here may be that sound was imported with incorrect length to begin with

Yes, original length of audio file is 2677. Importing with 25 fps shows 2676.

I can reproduce with steps provided by Georg. Or you can refer the video as well. (Let me know if anything is not clear.)

Audio file:
Beyond_Dreams_mastered.mp3

> issue here may be that sound was imported with incorrect length to begin with Yes, original length of audio file is 2677. Importing with 25 fps shows 2676. I can reproduce with steps provided by Georg. Or you can refer the video as well. (Let me know if anything is not clear.) Audio file: [Beyond_Dreams_mastered.mp3](https://archive.blender.org/developer/F12759845/Beyond_Dreams_mastered.mp3)

Georg K commented

2021-12-18 14:20:59 +01:00

I assumed that the audio has to be longer than 4 mins since the difference was only one frame with a 4 min audio file and there was no issue with a 3:48 file (http://dig.ccmixter.org/files/airtone/64427), but that has been disproved by the file from Pratik.

The 4mins audio file is proprietary, but I found another one with CC rights that is 4749 frames long at import but 4750 frames after switching the scene from 24 fps to 25 fps and back:
http://dig.ccmixter.org/files/Milky_Blue/63912

I assumed that the audio has to be longer than 4 mins since the difference was only one frame with a 4 min audio file and there was no issue with a 3:48 file (http://dig.ccmixter.org/files/airtone/64427), but that has been disproved by the file from Pratik. The 4mins audio file is proprietary, but I found another one with CC rights that is 4749 frames long at import but 4750 frames after switching the scene from 24 fps to 25 fps and back: http://dig.ccmixter.org/files/Milky_Blue/63912

Richard Antalik commented

2021-12-20 10:02:09 +01:00

Added subscriber: @ZedDB

Richard Antalik commented

2021-12-20 10:02:09 +01:00

Thanks for samples, I can reproduce now. I haven't realized, that difference is only 1 frame. From video it looked much larger. This will likely be due to inconsistency in length caclculation when strip is added vs when FPS does change.

To make things more confusing, ffprobe reports length of 03:17.89, while audacity 03:18.06. Audaspace reports length matching ffprobe, so will go with that. 4749 is result of rounding to integer which may not be technically correct way to do this. Refresh on framechange uses next higher integer instead. I remember talking about this with @ZedDB, I think this was changed recently and not sure if this will be documented why, but will look for reasoning and sonsolidate this in any case.

Thanks for samples, I can reproduce now. I haven't realized, that difference is only 1 frame. From video it looked much larger. This will likely be due to inconsistency in length caclculation when strip is added vs when FPS does change. To make things more confusing, ffprobe reports length of 03:17.89, while audacity 03:18.06. Audaspace reports length matching ffprobe, so will go with that. 4749 is result of rounding to integer which may not be technically correct way to do this. Refresh on framechange uses next higher integer instead. I remember talking about this with @ZedDB, I think this was changed recently and not sure if this will be documented why, but will look for reasoning and sonsolidate this in any case.

Richard Antalik commented

2021-12-20 10:02:32 +01:00

Changed status from 'Needs User Info' to: 'Confirmed'

Sebastian Parborg commented

2022-01-05 13:15:53 +01:00

@iss I think that the "refresh on framechange" might be an oversight on my part.
I tried to unify how audio track length was calculated but I must have missed that one.

IIRC, I changed the rounding to always floor the frame count (cast to int).
This is because I tried a dozen of different video files and most of them seemed to line up (length wise) with the video when flooring the frame length.
From my tests it seemed like the previous logic of rounding up was because blender would previously not encode the last frame of audio properly before I did some fixes to remedy that.
So I'm guessing that the "round up" logic was there to make sure that videos/audio exported by Blender were the correct frame duration.
However this was just side stepping the root issue of blender not encoding the last frame of audio properly.

In addition to that, when we floor, we don't have any frames containing data for only half a frame or so.
Which subjectively to me, is logical because we work with whole frames in the VSE. (However the ability to shift the audio kinda pokes holes in this logic, so meh).

@iss I think that the "refresh on framechange" might be an oversight on my part. I tried to unify how audio track length was calculated but I must have missed that one. IIRC, I changed the rounding to always floor the frame count (cast to int). This is because I tried a dozen of different video files and most of them seemed to line up (length wise) with the video when flooring the frame length. From my tests it seemed like the previous logic of rounding up was because blender would previously not encode the last frame of audio properly before I did some fixes to remedy that. So I'm guessing that the "round up" logic was there to make sure that videos/audio exported by Blender were the correct frame duration. However this was just side stepping the root issue of blender not encoding the last frame of audio properly. In addition to that, when we floor, we don't have any frames containing data for only half a frame or so. Which subjectively to me, is logical because we work with whole frames in the VSE. (However the ability to shift the audio kinda pokes holes in this logic, so meh).

Richard Antalik commented

2022-01-06 02:32:27 +01:00

Thanks for info, I was thinking that it may be better to round up if for example movie files were split and this would cause gaps in audio causing pops and clicks. But I think this would happen anyway though so not sure really.

I think either way may be acceptable, but it should be consolidated.

Thanks for info, I was thinking that it may be better to round up if for example movie files were split and this would cause gaps in audio causing pops and clicks. But I think this would happen anyway though so not sure really. I think either way may be acceptable, but it should be consolidated.

Sebastian Parborg commented

2022-01-06 11:57:01 +01:00

Usually the video frame length and the audio buffer chunk size doesn't match up, so if you split a video file you will be in the middle of a audio chunk.

The same is true when creating a video file and writing the last audio portion of it.
Usually what happens is that the video data will end while in the middle of an audio chunk.

If the actual raw audio and video duration is exactly the same, then the last audio chunk will be padded with silence (the real audio and video data ends at the blue line).

If you split a video into chunks, then you will have the same issue.

The split will be in the middle of an audio chunk. So to make them line up, you will have to floor the frame duration of the audio when importing.
Otherwise you will either lose audio (if you didn't encode the overlapping audio chunk when splitting) or you end up with shift as the audio will last for too long if rounded up.

Then for the next video section you will have to encode the same audio chunk again that we cut off as the first chunk and encode an audio offset so that the chunk is properly aligned with the start frame.

The audio start offset will be the duration from the start of the audio chunk to the start video frame:

If you do this, then I think we should be able to split video files and they should still line up when importing them into blender again.

Usually the video frame length and the audio buffer chunk size doesn't match up, so if you split a video file you will be in the middle of a audio chunk. The same is true when creating a video file and writing the last audio portion of it. Usually what happens is that the video data will end while in the middle of an audio chunk. ![test1.png](https://archive.blender.org/developer/F12791345/test1.png) If the actual raw audio and video duration is exactly the same, then the last audio chunk will be padded with silence (the real audio and video data ends at the blue line). ![test2.png](https://archive.blender.org/developer/F12791350/test2.png) If you split a video into chunks, then you will have the same issue. ![test3.png](https://archive.blender.org/developer/F12791358/test3.png) The split will be in the middle of an audio chunk. So to make them line up, you will have to floor the frame duration of the audio when importing. Otherwise you will either lose audio (if you didn't encode the overlapping audio chunk when splitting) or you end up with shift as the audio will last for too long if rounded up. ![test4.png](https://archive.blender.org/developer/F12791363/test4.png) Then for the next video section you will have to encode the same audio chunk again that we cut off as the first chunk and encode an audio offset so that the chunk is properly aligned with the start frame. ![test5.png](https://archive.blender.org/developer/F12791367/test5.png) The audio start offset will be the duration from the start of the audio chunk to the start video frame: ![test6.png](https://archive.blender.org/developer/F12791375/test6.png) If you do this, then I think we should be able to split video files and they should still line up when importing them into blender again.

blender-admin commented

2022-01-25 23:53:33 +01:00

This issue was referenced by 83094d9a0d

This issue was referenced by 83094d9a0d43a95fa0ac2ec79234e54bb1c88675

Richard Antalik commented

2022-01-26 00:00:28 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Richard Antalik closed this issue

2022-01-26 00:00:28 +01:00

Richard Antalik self-assigned this 2022-01-26 00:00:28 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Audio length inconsistency after frame rate change #94149