Page MenuHome

Audio strips crackle when animating the volume
Closed, ResolvedPublic

Description

When the Volume value of an Audio Strip is animated inside the video sequencer, you can hear "crackling" stepping artifacts no matter what the input file format is. This may be related to the audio buffer as changing the size in User Preferences will affect the character and severity of the crackle. I'm attaching an example .blend file with a sound strip that has its' volume animated by a sine-wave to make the problem more audible.

Event Timeline

I noticed that when setting the buffer very high, the volume is not changed mid-buffer (it adjusts the entire buffer's volume to the keyframe value at the beginning of the buffer). My guess is the lack of interpolation between buffers is causing a sudden volume change that is causing the pops.

Am I right in thinking that this is only a problem in preview playback? I've noticed it recently due to cranking the buffer size while CPU was high, but rendered output was fine.

Have you tried doing a mixdown? I'm getting the same problem in the rendered result too. It's also not as noticeable with some sounds.

Have just tried your sample. What you are hearing is the audio level being changed with each blender frame - it does not evaluate f-curves at the audio sample rate, only at the video frame rate, so if you are changing levels that quickly you will definitely hear zipper noise.

If I get a minute (or, more likely, several hours, so don't hold your breath) I'll see if I can make it do audio strip f-curves at the sample rate, but you can bet that nobody would want the CPU time penalty.

Suspect this falls under the category of "not a bug according to the current design of Blender's audio system".

This isn't really about F-Curves, i used one in my example to illustrate the issue better. I originally noticed the zipper noise when trying to do fade-ins and fade-outs for audio strips in the VSE, which is a common use case. The only way to do them is to animate the Volume value with a couple of keyframes and a bezier curve.

What about using a high sample rate only during Mixdown and Rendering? Those processes are CPU-intensive anyway, and the only cases where smooth fades really matter.

f-curves / keyframes / modifiers / all the same evaluation system, at the video frame rate.

Having had a quick squizz there's absolutely no way I'm going to try to make that whole system run at anything other than video frame rate as it uses the frame number for all its maths.

However, I may be able to do linear interpolation between those calculated values during audio mixdown only as you suggest, which won't be perfect but would be a lot less noisy than the current system.

I'll keep you posted.

Thank you for looking into this, Olly! Interpolation between frames sounds great, and I doubt anyone would notice unless they are dealing with ridiculously low framerates for some reason.

Would it be too cpu intensive to implement this interpolation for live playback just for the audio buffer chunks, not on a sample-by-sample basis? I understand if it is, just curious if that would be more reasonable.

Proof of concept done... I had to touch more stuff than I thought, as it turns out the animation of volume isn't done where I thought it was (anyone know why there are two layers of volume animation? The m_volume.read in AUD_SequenceReader.cpp line 186 does nothing, always gives volume of 1.0!).

Here's a rendering of the test piece:

I'll make a paste of the diff shortly, but I've probably broken some other stuff so I won't submit it for code review yet.

I'm currently a bit unsure if this is a limitation or an actual bug @Aaron Carlisle (Blendify). Reading so far, the solution with the interpolation for audio buffer chunks sounds like what should be implemented right now, but I could be wrong and need to check the code better. Afair there is even spline interpolation in the code. @Olly Funkster (Funkster) can you just post the diff of what you did and then we can work together on a proper solution? Thanks!

Certainly, here you go:

https://developer.blender.org/P418

I freely admit that it is poorly documented and likely to break other stuff!

The interpolation for the value in between frames was already there, I have just added getting the value for the time that will be at the end of the current buffer length, and the interpolating between those values while going through that buffer.

Sergey Sharybin (sergey) triaged this task as Normal priority.

@Joerg Mueller (nexyon), assigning to you. You're the only one who can solve and investigate this issue anyway :)

Thanks @Sergey Sharybin (sergey), I was already on it, so here are the results of my investigation:

The crackling is caused for two different reasons in the attached test file.

  1. The amplitude of the f-curve is above 1, the sinus goes between 0 and 2. The problem here is that for a volume > 1 you get clipping and clipping can cause crackling. So if you change the amplitude to 0.5 and the mean to 0.5 as well, you get volume to change between 0 and 1 and no clipping anymore.
  2. The audaspace animation system and blender animation system need to be synchronized and this is unforunately not happening (fully) automatically. Open a properties window and hit the "Update Animation Cache" button under Scene -> Audio or call the operator directly using space.

After doing those two steps, the crackling should be gone, at least for me it is. I am talking to @Sergey Sharybin (sergey) in the chat right now to have the user use that button less, but it's difficult and a known regression. The whole existence of the button is based on the fact that the Blender animation system was never designed with audio in mind.

Please let me know if this solves it, or still get crackling.

@Olly Funkster (Funkster) thank you for your help, but unfortunately you just reimplemented stuff that was already there.

FWIW, there is no clipping happening in the attached test file. I noticed the volume > 1.0 thing too, but the wind sample is quiet enough that the multiplied output never reaches 0dBFS. In my rendered example the sine still reaches 2.0.

I clicked your button (in the released 2.78) and the crackling is reduced but not gone.

Bit more info... having clicked the same button in my patched build, it now no longer crackles in preview playback.

So there are two sources of crackles:

  1. wrong (out of date?) volume values being passed to the mixer in preview playback if the button has not been pressed (which doesn't affect rendered output... presumably the animation cache is rebuild first)
  2. lack of per-audio-sample evaluation of the volume, leading to zipper noise at the frequency of the buffer length (which my patch does a plausible job of masking).

@Olly Funkster (Funkster): you're right, it's not completely gone. I implemented a fix for this which is a bit simpler and less intrusive in the library. Thanks for your help again!

Heh, sneaky. Animated volume will be late by the audio buffer length, but I doubt anyone will notice unless they're using very long buffers. Might get a click at the very start of the render if the first animated value is supposed to be 0, as it'll fade down from 1.0 to 0 over the first buffer.

Yeah, I could set the starting value to 0, then every strip gets an automatic fadein at the beginning, but not when seeking (this includes Blender jumping back to the beginning of the animation when it reaches the end), so in any case the solution won't be perfect. :-/ A nice solution that I have in mind requires quite big changes in the library, that are planned for audaspace 2, that I haven't started coding yet.