Audio sync can backtrack in time #44195

New Issue

Antonis Ryakiotakis · 2015-03-30T18:48:12+02:00

2015-03-30 18:48:12 +02:00

System Information
Ubuntu 14.10 64bit
Memory 7,8 GiB
Intel Xeon(R) CPU E31280 @ 3.50GHz × 4
NVIDIA Quadro 2000/PCIe/SSE2

Blender Version
Broken: 590efaa

Short description of error
Audio sync can backtrack in time when playing an animation.

Exact steps for others to reproduce the error

Compile blender after uncommenting definition of PROFILE_AUDIO_SYNCH in screen_ops.c
Default scene, enable audio sync
Let animation play for some time and check at the output of blender in the command line

You should notice forward jumps and back jump detections at some point.
Forward jumps are normal (can happen when blender processing lags audio processing)
but back jumps are not - suggests an error in the code that calculates the audio offset.

**System Information** Ubuntu 14.10 64bit Memory 7,8 GiB Intel Xeon(R) CPU E31280 @ 3.50GHz × 4 NVIDIA Quadro 2000/PCIe/SSE2 **Blender Version** Broken: 590efaa **Short description of error** Audio sync can backtrack in time when playing an animation. **Exact steps for others to reproduce the error** * Compile blender after uncommenting definition of PROFILE_AUDIO_SYNCH in screen_ops.c * Default scene, enable audio sync * Let animation play for some time and check at the output of blender in the command line You should notice forward jumps and back jump detections at some point. Forward jumps are normal (can happen when blender processing lags audio processing) but back jumps are not - suggests an error in the code that calculates the audio offset.

Antonis Ryakiotakis commented

2015-03-30 18:48:12 +02:00

Changed status to: 'Open'

Joerg Mueller was assigned by Antonis Ryakiotakis

2015-03-30 18:48:12 +02:00

Antonis Ryakiotakis commented

2015-03-30 18:48:12 +02:00

Added subscriber: @Psy-Fi

Bastien Montagne commented

2015-03-31 08:06:36 +02:00

Added subscriber: @mont29

Joerg Mueller commented

2015-04-21 13:33:26 +02:00

Hi,

at first I couldn't reproduce the problem at all, but with the default suspects I quickly found that with pulseaudio running this problem appears. After looking a bit into details, I found that the reason for this problem is the buffer size used somewhere in this system, as your hint regarding the big chunks of second offsets suggested.

All your suggested solutions (frame tolerance, wait functions, semaphores) only try to work around the actual problem and have little chance of success in any case, you have to tackle the actual problem... Please revert the frame tolerance commit as it makes AV-sync less precise in any case, even when playback appears more smooth with it!

To get the buffer size down, I first tried to configure pulseaudio with the default-fragments and default-fragment-size-msec settings, but that didn't help.

Talking to Chris Robinson, the OpenAL soft developer, he told me that these values can be configured in the OpenAL configuration as well and also suggested an OpenAL soft extension for AV sync.

The configuration solves the problem for me and you should try it, just put this configuration into the ~/.alsoftrc file:

[general]
drivers = pulse
period_size = 128
periods = 2

This resolved the problem for me and I don't get any jumps anymore neither with nor without the tolerance code you added.

Regarding better AV sync, I wrote the openal_latency_patch.diff patch. The patch alone doesn't resolve the problem completely, just makes it a bit better. Also this only works with OpenAL soft, so compilation without it (on Mac and Windows) will fail and the code changes also interfere with audio in the game engine, so please do not commit this patch to master in any case! I added a define to switch the functionality off and on quickly. You might want to try it out with artists to check if they notice any difference when lip-syncing.

The ultimate solution I have for this and other problems is to write a direct pulseaudio backend for audaspace, but this requires way more time and I'd also do this with the C++11 version of audaspace, which has more improvements already.

I'll close the task for now. If the configuration file doesn't fix the problem for you, feel free to reopen the task.

Cheers

Hi, at first I couldn't reproduce the problem at all, but with the default suspects I quickly found that with pulseaudio running this problem appears. After looking a bit into details, I found that the reason for this problem is the buffer size used somewhere in this system, as your hint regarding the big chunks of second offsets suggested. All your suggested solutions (frame tolerance, wait functions, semaphores) only try to work around the actual problem and have little chance of success in any case, you have to tackle the actual problem... Please revert the frame tolerance commit as it makes AV-sync less precise in any case, even when playback appears more smooth with it! To get the buffer size down, I first tried to configure pulseaudio with the default-fragments and default-fragment-size-msec settings, but that didn't help. Talking to Chris Robinson, the OpenAL soft developer, he told me that these values can be configured in the OpenAL configuration as well and also suggested an OpenAL soft extension for AV sync. The configuration solves the problem for me and you should try it, just put this configuration into the `~/.alsoftrc` file: ``` [general] drivers = pulse period_size = 128 periods = 2 ``` This resolved the problem for me and I don't get any jumps anymore neither with nor without the tolerance code you added. Regarding better AV sync, I wrote the [openal_latency_patch.diff](https://archive.blender.org/developer/F164975/openal_latency_patch.diff) patch. The patch alone doesn't resolve the problem completely, just makes it a bit better. Also this only works with OpenAL soft, so compilation without it (on Mac and Windows) will fail and the code changes also interfere with audio in the game engine, so please do not commit this patch to master in any case! I added a define to switch the functionality off and on quickly. You might want to try it out with artists to check if they notice any difference when lip-syncing. The ultimate solution I have for this and other problems is to write a direct pulseaudio backend for audaspace, but this requires way more time and I'd also do this with the C++11 version of audaspace, which has more improvements already. I'll close the task for now. If the configuration file doesn't fix the problem for you, feel free to reopen the task. Cheers

Joerg Mueller commented

2015-04-21 13:33:57 +02:00

Changed status from 'Open' to: 'Resolved'

Joerg Mueller closed this issue

2015-04-21 13:33:57 +02:00

Antonis Ryakiotakis commented

2015-04-21 14:10:17 +02:00

Frame tolerance is not a good solution indeed, ideally this code would just sleep for a small time and re-query the sound library until frame changes. I tried this too a few weeks back without success. Maybe with the new configuration this might be more of a plausible and correct solution though. Anyway, thanks for the heads up, I will check it out.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Audio sync can backtrack in time #44195