How 3D Audio Works: Explained
Have you ever wanted to transport yourself into the heart of a concert hall or immerse yourself in the sounds of nature without leaving the comfort of home? With the emergence of spatial audio, this easily becomes a reality. In this beginner's guide, we’re exploring the world of spatial audio, how it works, and how you can listen yourself.
3D audio effects are a group of sound effects that manipulate the sound produced by stereo speakers, surround-sound speakers, speaker-arrays, or headphones. It is the phenomenon of transforming sound waves (using head-related transfer function or HRTF filters and cross talk cancellation techniques) to mimic natural sounds waves, which emanate from a point in a 3-D space. Using head-related transfer functions and reverberation, the changes of sound on its way from the source (including reflections from walls and floors) to the listener's ear can be simulated.
3D Positional Audio effects emerged in the 1990s in PC and video game consoles. Some amusement parks have created attractions based around the principles of 3-D audio. One example is Sounds Dangerous! at Disney's Hollywood Studios at the Walt Disney World Resort in Florida. Guests wear special earphones as they watch a short film starring comedian Drew Carey. At a point in the film, the screen goes dark while a 3-D audio sound-track immerses the guests in the ongoing story. To ensure that the effect is heard properly, the earphone covers are color-coded to indicate how they should be worn.
There have been developments in using 3D audio for DJ performances including the world's first Dolby Atmos event on 23 January 2016 held at Ministry of Sound, London.
Spatial audio? Heard of it? Chances are good that you’ve stumbled across it before. Some know it from gaming, some from movies or VR. The most fantastic marketing terms quickly come up - immersive audio, Dolby Atmos, 3D, 360°, 8D sound , 3d audio, 3d sound or “the music revolution” etc.. After all, you want to sell new software or hardware.
What is Spatial Audio?
Spatial audio is an innovative technology that creates a three-dimensional listening experience, making it seem as if sound is coming from various directions and distances. Unlike traditional stereo sound, where audio is delivered through two channels (left and right), spatial audio adds an extra dimension by incorporating height. This technology allows you to perceive sound as though it's coming from specific locations around you, mimicking how sound travels in real life.
In this context, people also like to talk about “immersive audio”. An enveloping sound that is so natural that we humans feel really comfortable in this digital reality. We virtually immerse ourselves in the artificial world and forget everything around us (and bit like the immersion in VR glasses, only related to the sound). Ideally, however, all senses are included in the immersion. But if you will, even a stereo mix or even mono can be “immersive” if the content is well done. At the disco, people still get sucked into the music - even if it doesn’t have more than two audio channels.
Apple also does it this way, in English they call it “Spatial Audio”. This is often translated back here as “spatial sound”. But you can see that there is not yet complete agreement, so let’s see what they say in a few years. And Apple‘s spatial audio feature doesn’t work with any device. You’ll need compatible hardware and software such as a current iOS device, supported headphones and a supported app. You can experience spatial audio on iOS devices if you turn on the spatial audio icon in your volume control center. This also works with volume control on a Macbook Pro, or newer iPad models.
To enable spatial audio on iOS devices, open the Control Center and look for the spatial audio controls.
How Spatial Audio Works
Unlike stereo or surround sound, spatial audio uses sophisticated algorithms, advanced processing techniques, and specialized hardware to recreate lifelike soundscapes. Using object-based sound technology, such as Dolby Atmos, sound objects (including vocals, instruments, or effects) are strategically assigned to specific locations in a 3D space rather than a fixed channel.
Say you’re listening to spatial audio on a single speaker, like Sonos Era 300. To produce sound that feels like it’s coming from above, upward-firing drivers in the speaker bounce sound off the walls and ceiling, which get reflected to a specific location in the room. Because audio isn’t being projected in one direction, as is the case with mono and stereo sound, your content feels like it’s hitting you from every direction.
A standout quality of Dolby Atmos is its ability to adapt to various hardware and playback setups. Whether listening on headphones, a smart speaker, or a complete home theater system, Dolby Atmos optimizes the sound to your environment for a more realistic listening experience.
Honestly, not even 5.1 surround has really made its way into the living room. As the name suggests, you would need a total of five speakers, which you place in front of the left, center and right, as well as in the back on the left and right. Sounds all kind of impractical - it is. From the speakers you not only need the space, you also need to pull cables across and should have a room that is at least a bit acoustically adjusted. For 3D audio, you need even more speakers. Here, 7.1.4 is just becoming the quasi-standard for recording studios. For consumers, all this is not quite reasonable, but there is already a remedy in the form of soundbars.
These elongated speakers are usually placed under or in front of the TV, where there is usually enough space. But this space is used as efficiently as possible, because there are actually several built in speakers, into these speakers. A few face forward - where the couch is in the average living room. Huh, so past us? Right, but the target is actually your walls or ceiling, where the sound is reflected and we the people watching a movie on the couch, it sounds like you actually have speakers hanging on the ceiling, or to the side of you. The device measures itself and your room with all audio settings at startup.
This way, it knows approximately where your walls are and eliminates the so-called room modes, i.e. frequencies where your room could sound dull. What doesn’t work so well yet is the “from behind”, because you have to play the sound over the band twice here. Therefore, the package is sold with two additional speakers and a subwoofer for bass lovers. Now you just have to pay attention to how many speakers are installed in the soundbar, like 5.1.2 or even 9.1.6. In fact, you can say here: A lot helps a lot, the more the better. If it also says Dolby Atmos, chances are good that you can connect the device directly to your smart TV via HDMI and get that cinema feeling at home via streaming services.
These homepods often use a similar principle. They understand where they are in the room and try to use this situation as well as possible. Nevertheless, the usage is usually quite different. The interaction between compatible devices is usually different. Mostly, a voice assistant is integrated, which you blithely tell your wishes. In the hope that he or she understands what you want. In reality, this doesn’t work perfectly, but it can be easier than manually navigating to the desired audio content on your smartphone.
But back to the topic of surround sound: I was allowed to supervise a bachelor thesis, where my student wanted to find out why there are actually several speakers in a smart speaker. Simply put, the device wants to sound bigger than it is. And indeed, with the right 3D audio content, the box no longer sounds like it looks.
Spatial hearing is fascinating and lets us experience the world around us in 3D. But how can we do that with just two ears? It’s all a matter of timing and intensity. Our ears hear sound waves from the environment in different ways, depending on where the different sounds come from. The brain processes these differences to create a personalized spatial audio map that tells us where a sound is coming from. It’s an incredibly complex process, and yet we manage it effortlessly and without even realizing it.
It all sounds a bit abstract, so let’s take a quick example. Let’s imagine we are standing on a street and hear a car honking from the right. Then the sound reaches the right ear first, before it reaches the left ear (ITD). After all, the right ear is closer to the car, even if it is only 17-20 centimeters. In addition, the sound is also louder on the right ear than on the left, because the head shades the sound event like a small wall. And last but not least, the horn has a different frequency response on both, which is due to the shape of our pinna. While on the right the sound can enter the ear canal quite easily, on the left the sound is refracted around our head and captured by the pinna.
So what our brains do in real time all our lives, software algorithms are now trying to recreate - that‘s how spatial audio works. So the tools always ask themselves: I have so-and-so many 3D objects in my virtual space - how would that sound to two human ears now. This process is also called binauralization.
This is the simple principle, but as you can imagine, these algorithms are not only CPU-hungry, but also only approximately correct. For example, every person has a different ear shape and head size, which is why the calculation usually only renders with average numbers. But if the renderer knows what your own ears look like, it can be adjusted. This is called personalized HRTF, and in fact Apple was one of the first to make the setup process of taking pictures of your own ears socially acceptable.
Let’s move a bit away from technology and towards perception. What spatial audio wants to create via headphones is precisely the impression that the sound is happening around us and that we are more or less at the center of the action. Not quite, because I’m going to tell you a problem that you didn’t know was a problem. When you listen to a podcast on headphones, for example, that is a mono signal, what happens on the headphones is this. The signal is output on the left and right channels at the same time. As a result, the sound arrives at the eardrum at the same time, if we turn spatial audio around here in mono, it doesn’t change anything. This means that the difference in level, time and frequency is 0, which leads our brain to the logical conclusion: the sound must be in our head.
So mono is always perceived in the middle of our head via headphones. Even if you work with stereo, you can only turn the sound source to the left or right, but our brain still knows that the sound comes from the headphones. There are microphone methods like the ORTF, which, like an artificial head, makes use of a distance between two microphones.
But only when the three parameters of spatial hearing are fulfilled does the feeling really arise that the sound is coming “from outside”. I also like to say that it feels like you’re not wearing headphones at all. It has happened to me many times that I have listened to 3D content through headphones, but thought the sound was coming from my speakers and wanted to take the headphones off again. Which left me sitting in a silent recording studio and realizing: the speakers weren’t even on and the headphones were tricking me.
There are two ways to do this: Record the sound already three-dimensional with special microphones, i.e. the right hardware. Recording 3D sound is nothing new, by the way. Some of you may be familiar with artificial head recordings. These are created with a microphone that was modeled on the human hearing apparatus and actually has ears. Like a mannequin with a microphone in each ear.
The prime example of how such a microphone can be used creatively is the Virtual Barbershop. Unfortunately, such a dummy head has a big disadvantage: What works great on headphones, does not work at all when played back on speakers. Feel free to make a comparison yourself. To record sounds three-dimensionally in such a way that they also work well on loudspeakers, other methods are suitable. For example Ambisonics or ORTF-3D from SChopes or the Sennheiser Ambeo VR Micro. Here 4, 8 or more microphones point in all directions, a bit like a 360° camera where several lenses form a sphere.
Once this sound field has been captured, it can usually be reproduced quite flexibly afterwards on 1, 2, 4 or 8 loudspeakers placed around you. But you can already guess that this is a bit unwieldy. And what if I want to create scenes to which I can’t just drag a microphone array. Here comes the second variant, which makes it possible to create spatial audio content: Namely with the appropriate software, often called “Spatializer”. There are various plugins that take a mono signal as input and place it in an artificial space, e.g. right behind us, coming from above.
So with these directional audio filters the sound suddenly gets three-dimensional information. Now the software only has to convert the signal in real time in such a way that it creates directional spatial audio with filters and the illusion via headphones that it is located exactly at this position. If you want to use multiple speakers, the software needs to know what kind of configuration you are using. For example 5.1, 7.1, or 7.1.4 (seven speakers on the horizontal, one subwoofer, 4 speakers hanging from the ceiling). Again, this calculation happens in real time and you can have a mono object flying around your head in three dimensions. Probably the best known software is Dolby Atmos.
That gives us everything we need, doesn’t it? Not quite. So far, we’ve simply moved a...
Why Should I Listen with Spatial Audio?
Spatial audio brings you closer to the creator's original intent, allowing you to hear your content the way it was meant to be heard. When listening to music, you'll feel as if you're in the recording studio, surrounded by every instrument and nuanced detail. When watching a movie, the explosions will reverberate around you, the dialogue will appear to come from specific corners of the room, and ambient sounds will transport you into the center of the scene - it's like having a private cinema right in your living room.
Spatial audio enriches your emotional connection to your content by making it more engaging and lifelike.
How Can I Listen to Spatial Audio at Home?
To experience your music and movies in spatial audio from the comfort of home, you’ll need two things: a Dolby-Atmos supported streaming service and a compatible device.
Many think that you need special headphones to support spatial audio, but that’s not true. Different headphones can play spatial audio, including models like the Apple AirPods Pro, Sony WH-1000XM4, and Bose QuietComfort 35 II. This means that a huge infrastructure is already in place to play back 3D spatial audio content well. The only prerequisite is that the content in question must be available binaurally (we’ll get to how that works in a moment).
Or that the content is converted to two channels in real time by the playback device (as with Dolby Atmos). The problem is that often where it says spatial audio, it doesn’t really is spatial audio. 3D spatial audio with dolby atmos tracks is not a seal of quality and is often used as a marketing label. Even Dolby Atmos playback does not mean “that sounds great”. There are simply some contents that work better or worse in 3D. It’s a bit like 3D movies: it’s fun for action, but it doesn’t always make sense for quieter genres. Just like 3D audio with dolby atmos doesn‘t make sense for any type of music.
I recommend over-ear headphones for listening pleasure. Simply because the sound is then generated as far away from the eardrum as possible. Experience has shown that this works somewhat better than when the sound is generated in the eardrum, as with earbuds, and thus has to travel little distance through our hearing apparatus. As a sound engineer, I am a fan of headphones with a linear frequency response, these distort the audio the least. Well, unless you want that certain something. And that is dynamic head tracking in this case.
This means that acceleration sensors are installed in the hardware, which recognize where you are looking and turn spatial audio sound effects around in real time. If you move your head, the apple’s spatial audio sound field rotates accordingly. The dynamic head tracking technology is already built into all Apple devices, and the competition has already followed suit. But what do you hear? This question will be answered in detail in the next article.
Netflix recently announced a partnership with Sennheiser’s Ambeo to make the streaming and personalized spatial audio experience accessible to all customers who want to experience personalized spatial audio without special hardware. A custom signal processing algorithm maintains dialogue integrity and adds a sense of spaciousness to the surround sound, without artificial reverberation or room sounds. Netflix tested this option for over two years and had it approved by sound engineers for the streaming catalog.
If you want to watch the currently available movie titles with 3D spatial audio compatible headphones on, just search for “Spatial Audio” on Netflix and dive into a new era of streaming!
Compatible Devices
Below are a few types of devices you can use to play Dolby Atmos content:
- Headphones: One of the most popular ways to experience spatial audio is with a pair of headphones. Over-ear headphones like Sonos Ace have the ability to create an exceptional acoustic seal around your ears, making you feel completely surrounded by what’s playing.
- Smart speaker: If you want to experience spatial audio out loud instead of using headphones, some smart speakers - like Sonos Era 300 - can fill a large space with immersive Dolby Atmos content. No matter where you are in the room, it will feel like the music is playing all around you.
- Soundbar: Similar to a smart speaker, some high-end soundbars, like Sonos Arc Ultra, can support spatial audio for movies and TV shows for a theater-like experience. These systems often include upward-firing drivers to bounce sound off the ceiling to create an all-encompassing effect.
- Gaming console: If you want to put yourself inside your games, certain consoles and PCs with compatible sound cards can deliver a truly lifelike experience when playing content mixed in Dolby Atmos.
- Virtual reality (VR) headset: Some VR headsets provide spatial audio as part of their virtual experiences. When pairing spatial audio with VR content, these headsets can offer an incredibly immersive audio-visual experience.
Experience Spatial Audio
Spatial audio represents a groundbreaking leap in the way we enjoy our favorite movies, music, and games. Its ability to create immersive, lifelike audio environments opens up new dimensions of entertainment where sound isn’t just heard but felt. Sonos Era 300 or Sonos Arc Ultra speakers feature cutting-edge technology that allow you to effortlessly experience spatial audio at home.
If you move your head, the apple’s spatial audio sound field rotates accordingly. The dynamic head tracking technology is already built into all Apple devices, and the competition has already followed suit.