Reverb
One of the most important and widely used techniques in computer music is the use of reverberation and the addition of various types and speeds of delays to dry sounds. Reverberation and delays can be used to simulate room and other environmental acoustics, or to create new sounds of their own, not necessarily related to existing physical spaces.
There are a number of commonly used techniques for simulating and modelling different reverberations and physical environments. One interesting technique is to actually record the ambience of a room and superimpose that onto a sound recorded elsewhere. This technique, called convolution, will be discussed further below under frequency domain techniques.
A Little Mathematical Excursion: Convolution in the Time Domain
Convolution can be accomplished in two equivalent ways, either in the time domain, or in the frequency domain. We'll talk about the time domain version first. Convolution is like "running average" — that is, we take the original function, representing the music — we'll call that one m(n), where m(n) is the amplitude of the music function at time n, and then we make a new, smoothed function in the following way: at time n, we'll make the amplitude (a)of our new function equal to:
a = (1/3)[m(n-2) + m(n-1) + m(n)]
Lets call this new function equal r(n) — for running average — and let's look at this a little more closely:
Just to make things work out, we'll let r(n) start off by making
r(0) = 0
and
r(1) = 0
Then
r(2) = 1/3[m(0) + m(1) + m(2)]
and
r(3) = 1/3[m(1) + m(2) + m(3)]
and so on....see, we're running along, and taking the average of every 3 values of m! Hence the name, "running average"! Now, another way to look at this is as taking the music m(n) and "convolving it against the filter a(n)" where a(n) is another function, defined by
a(0) = 1/3, a(1) = 1/3, a(2) = 1/3
and all the rest of the a(n) are 0.
Now, what is this mysterious "convolving against" thing? Well, first off we write it like
m*a (n)
and the
m*a(n) = m(n)a(0) + m(n-1)a(1) + m(n-2)a(2) +....+ m(0)a(n)
and
m*a(n) = m(n)(1/3) + m(n-1)(1/3) + m(n-2)(1/3)
Hey, look at that ma, I'm convolving!!!
Now, there is nothing special about the filter a(n), in fact it could be any kind of function. It could be a function that reflects the acoustics of your room, and then what we are doing — look back at the formula— is shaping the input function m(n) according to the filter function a(n). A little more terminology (sorry, but everything in life needs a name! People, dogs, boats, mathematical procedures, teddy bears...): the number of nonzero values that a(n) takes on is called the number of "taps" for a(n). So, our running average is a three-tap filter.
Reverb in the Time Domain
Well, convolution is sure fun, and we'll see more about it below, but a far easier way to create reverb, in the time domain, is to simply delay the signal some number of times (by some very small time value), feeding it back onto itself, simulating the way a sound bounces around a room. There are a large variety of commercially available and inexpensive devices that allow for many different reverb effects. One interesting possibility is to change reverb over the course of a sound, in effect making a room grow and shrink over time.
Before there were digital systems, engineers created various ways to get reverb-type effects. Any sort of situation that could cause delay was used. Engineers used reverberant chambers — sending sound via a loudspeaker into a tiled room and re-recording the reverb-ed sound. When one of us was a kid, he or she ran a microphone into their parents' shower to record a trumpet solo!
Reverb units made with springs and metal plates make interesting effects and many digital signal processing units have algorithms that can be dialed up to create similar effects. Of course we like to think that with a combination of delayed copies of signals we can simulate any type of enclosure, whether it be the most desired recording studio room or a parking garage.
As you'll see in the diagram below, there are some essential elements for creating a reverberant sound. Much of the time a good reverb "patch," or set of digital algorithms will make use of a number of copies of the original signal — just like there are many versions of a sound bouncing around a room. There might be one copy of the signal dedicated to making the first reflection: the very first reflected sound we hear when a sound is introduced into a reverberant space. Others might be algorithms dedicated to making the early reflections — the first sounds we hear after the initial reflection. The rest of the software for a good digital reverb is likely designed to blend the reverberant sound. Filters are usually used to make the reverb tails sound as if they are far away. Any filter that attenuates the higher frequencies — like 5 KHz or up — makes a signal sound farther away from us, since high frequencies have very little energy and don't travel very far.
When we are working to simulate rooms with digital systems we have to take a number of things into consideration:
- How large is the room? How long will it take for the first reflection to get back to our ears?
- What are the surfaces like in the room? Not just the walls, ceiling and floor, but are there objects in the room that can reflect sound waves? For example, are there any huge ice sculptures in the middle of the room?
- Are there surfaces in the room that can suck up sound? These are called absorptive surfaces. For example: movie theaters usually have curtains hanging on the walls to suck up sound waves. Maybe there are people in the room, most bodies are soft (not ours) and soft surfaces remove energy from reflected sounds. Perhaps there's a large flock of geese sleeping on the floor?
In the diagram above you see the graphic representation of a direct sound being introduced into a reverberant space. The RT 60 refers to Reverb Time or how long it takes the sound in the space to reduce by -60 dB. Photo courtesy of Garry Clennell <gcat@clara.co.uk> A typical reverb algorithm can have lots of control variables such as room size (gymnasium, small club, closet, bathroom), brightness (hard walls, soft walls or curtains, jello) and feedback or absorption coefficient (are there people, sand, rugs, llamas in the room — how quickly does the sound die out)?. Photo courtesy of Garry Clennell <gcat@clara.co.uk>
Figure .x Time domain reverb is conceptually similar to the idea of physical modeling—we take a signal (say a hand clap) and feed it into a model of a real-world environment (a large room). The signal is modified according to our model room and then output as a new signal.
In the diagram above you can see how sound from a sound source (like a loudspeaker) can bounce around a reverberant space to create what we call reverb.
What does a delay do? It takes a function (a signal), shifts it "backwards" in time, then combines it with another "copy" of the same signal. This is, once again, a kind of averaging.
Here's a basic diagram showing how a signal is delayed (in the delay box) then fed back and added to the original signal with some attenuated (that is, not as loud) version the original signal added. This would create a type of effect known as comb filtering (a short delay with feedback which emphasizes specific harmonics) as well as a delay.
This type of unit is used to create blended reverb. A set of delay algorithms would probably have some combination of a number of comb type delays seen in the figure above as well as some allpass filters to create a blend.
When we both feed-back and feed-forward the same signal at the same time, phase inverted, we get what is called an allpass filter. With this type of filter/delay we don't get the comb filtering effect (because of the phase inversion of the feed-forward signal— this fills in the missing frequencies created by the comb filter of the feedback-ed signal).
Making Reverb in the Frequency Domain: A Return to Convolution
As we mentioned above convolution is a great way to get reverberant effects. But we mainly talked about the mathematics of it, and why convolving a signal is basically a type of filter.
But now we're going to talk about how to actually do it. The process is fairly straight forward:
- Locate a space whose reverberant characteristics we like.
- Set up a microphone and some digital media for recording.
- Make a short, loud sound in this room, like hitting two drum sticks together (popping a balloon or firing a starter's pistol are other, standard ways of doing this — sometimes it's called "shooting the room")
- Record this sound, called the impulse.
- Analyze the sound that we collected, the impulse in the room; this gives us the impulse response of the room, the characteristics of the room's response to sound .
- Use the technique of convolution to combine this impulse response with other sound sources, more or less bringing the room to the sound.
Convolution can be powerful, but it's a fairly complicated software process. It takes each sample in the impulse response file (the one we recorded in the room, which should be short) and multiplies that sample by each sample in the the soundfile that we want to "put in" that room. So each sample input soundfile, like a vocal sound file to which we want to add reverb to, is multiplied by each sample in the impulse response file.
That's a lot of multiplies! Let's pretend, for the moment, we have a 3 second impulse file, and a 1 minute sound file to which we want to add the reverberant characteristics of some space. At 44.1 kHz, that's:
3 * 44,100 * 60 * 44,100
= 180 * 680,683,500,000
= 122,523,403,030,000
You might raise an interesting question here: isn't this the time domain? We're just multiplying signals together (well, more than that, we're multiplying each point in each function by every other point in the other function — called a cross multiply). But this multiply in the time domain actually produces what we refer to as the convolution in the frequency domain.
Now, we're not math geniuses (well, one of us is), but even we know that's this is a whole mess of multiplies (sorry to get technical).! That's why convolution, which is very computationally expensive (that's just a one minute sound!) had not been a popular technique until your average computer got fast enough to do it. It was also completely unknown as sound manipulation idea until digital recording technology made it feasible.
But nowadays, we can do it, and the reason we can do it is our old friend, the FFT! You see, it turns out that for filters with lots o' taps (remember that this means one with lots of nonzero values) that it is easier to compute the convolution in the spectral domain. And it goes something (ok, well exactly) like this.
Suppose we want to convolve our music function m(n) against our filter function a(n). We can tell you immediately that the convolution of these (which remember, is another sound, a function which we call c(n)) has a spectrum which is equal to the pointwise product of the spectrum of the music function and the filter function.
Another way if saying this is that the Fourier coefficients of the convolution can be computed by simply multiplying together each of the Fourier coefficients of m(n) and a(n). The zero coefficient (DC term) is the product of the DC terms of a(n) and m(n), the first coefficient is the product of the first Fourier coefficient of a(n) and m(n), and so on.
So, here's a sneaky algorithm (procedure, recipe, dance by the former Vice President) for making the convolution.
Step 1: compute the Fourier coefficients of both m(n) and a(n)
Step 2: compute the pointwise products of the Fourier coefficients of a(n) and m(n)
Step 3: compute the inverse Fourier transform (IFFT) of the result of Step 1
And we're done! Ta-Dah!! This was one of the great properties of the FFT. It made convolution just about as easy as multiplication! Go FFT- GO!
So, once again, if we have a big library of great impulse responses— the best sounding cathedrals, the best recording studios, concert halls, the Grand Canyon, Grand Central Station, your shower — we can simulate any space for any sound. And indeed this is how many digital effects processors and reverb plugins for computer programs work (more and more as the computers get faster). This is all very cool when you're working hard to get that "just right" sound.
Thanks to Nigel Redmon of earlevel.com for this graphic.
The impulse on the left and the room's response on the right.
Multitap delay applet.
applet Flanges, delays, reverb: .effect box fun applet
Changing the reverberant characteristics of a sound over time. Here is a nail being hammered. the room size goes from very small to very large over the course of about 15 seconds. This can't really happen in the real, physical world, and it's a good example of the way the computer musician can create "imaginary soundscapes."
Soundfile .x