Tempest 3D Audio Explained: Immersive Sound on the PlayStation 5

With the release of the PlayStation 5, there’s been plenty of discussion about the console’s upgraded audio capabilities. The PS5 boasts a new system architecture designed to bring a greater level of immersion to the audio experience. Developed for the PS5, Sony’s 3D audio uses an object-based system to place virtual objects in 3D space around the user.

Rather than adopt or build on an existing standard such as Dolby Atmos or DTS:X, Sony created its own competing system from the ground up. Since all the processing is built-in, no special external hardware is necessary. Sony 3D audio will work over your existing TV speakers, a stereo, 5.1 or 7.1 system, or preferably, headphones, which convey the effect of 3D audio most dramatically. The developers position game sounds in 3D space around the player to reflect where those sounds should be, relative to the player and the screen in front. This provides increased “presence and locality,” in Sony’s words.

How Does Sony’s 3D Audio Work?

The term HRTF refers to the effect our shoulders, head, and outer ears have on sound waves coming from different sources before they encounter our eardrums. Those physical interactions impact the timing, levels, and phase relationships of the different frequency content of the sounds we hear. As part of our hearing system, our brains are able to decode all those changes to decipher where the sound likely originated in the space around us.

If you were to place tiny microphones inside your ears and make a recording, it would include the effect of your personal HRTF. You would have a binaural recording that would work perfectly for you if you played it back over headphones, but it wouldn’t work quite as well for anyone else.

Sony clearly spent a lot of time developing its Tempest audio engine and the results of the PlayStation 5’s 3D audio are indeed impressive. At this time it is most effective over headphones, which is clearly where the company focused the majority of its research. How well the effect works over speakers will depend on your speaker setup and on your room. Performance over speakers, in their various possible configurations, is expected to improve over time with future updates.

Additionally, Sony has hinted at plans to roll out further refinements to the HRTF options, with the potential for customized or even fully personalized HRTFs in the future.

Setting Up 3D Audio on Your PS5

Find out how to enhance your PlayStation®5 console audio by making the sound three-dimensional. 3D audio gives you immersive surround sound that makes you feel like you're inside the game. With realistic audio, you can actually hear footsteps from the exact direction that an enemy is approaching.

How to setup 3D audio on PS5

3D Audio for TV Speakers

Use your TV speakers and the built-in mic on your wireless controller to measure your room acoustics. Sit in the same position you would when you normally use your PS5 console.

Go to Settings > Sound.
Select 3D Audio (TV) and follow the on-screen instructions to adjust the optimal 3D audio setting based on the measurement results.
After measurement is complete, you can listen to and compare both stereo audio and 3D audio to see which you prefer.

The next time you select Settings > Sound > 3D Audio (TV) and turn on Enable 3D Audio for TV Speakers, stereo audio switches to 3D audio. When you turn off this setting, 3D audio switches to stereo audio.

Whenever you change the position of your TV, your seating, or the room environment, you should measure your room acoustics again.

If you're having difficulty hearing the effects of 3D audio or you’d like to improve it, try the following:

Check that nothing is blocking the TV speakers.
If Apply 3D Audio Measurement Results is turned off, sit in front and center of your TV.
Watch your TV at a distance suitable for its size. Check with your TV's manufacturer for the recommended distance.
Disable any sound quality adjustment functions for your TV, such as surround sound.
The volume may change depending on the 3D audio effect. In this case, adjust the volume of your TV.

3D Audio for Headphones on PS5 Consoles

Listen to sounds played at different positions and angles to create an audio profile tailored to your hearing. There are two ways to create a profile:

Create a personalized 3D audio profile
- Measure how you hear 3D audio to create an optimized experience.
Select from 3D audio presets
- Select from 5 preset settings for an easy way to enjoy 3D audio.

Go to Settings > Sound.
Select 3D Audio (Headphones) > Create Personalized 3D Audio Profile and follow the on-screen instructions.
Listen to sounds and answer questions about the position and direction of the sound to set up your personalized 3D audio profile.

Once your personalized 3D audio profile is created you can enable Apply Personalized 3D Audio Profile. When you turn off this setting, the preset 3D audio profile will be used for 3D audio.

The next time you select Settings > Sound > 3D Audio (Headphones) and turn on Enable 3D Audio for Headphones, stereo audio switches to 3D audio.

Pulse 3D Wireless Headset

Sony released the Pulse 3D headset to accompany the PlayStation 5. This is definitely a headset aimed at couch-bound gamers. While this headset was specifically developed for the console, it isn’t necessary to get the full 3D audio experience. With a refined design, dual noise-cancelling microphones, and an array of easy-access controls, the Pulse 3D wireless headset offers a seamless experience for both the PS5 and PS4. Headphone audio is the current gold standard for 3D Audio on PS5, as Mark Cerny mentioned in his “Road to PS5” talk in March.

The Impact of 3D Audio in Game Development: A Returnal Case Study

Some devs from PlayStation's Creative Arts Sound team gave a very detailed (and very long) interview on the sound design of Returnal, and also explained a lot more about the PS5's 3D audio.

"The advantage of being a PS5 exclusive title is that it gives us a license to go "all in" with the platform features. 3D Audio, from the beginning of our involvement, was our target and we were aiming at doing the maximum we could to provide the best possible audio for Tempest (and haptics). At CSG, we have years of expertise designing, implementing, and mixing 3D Audio, as we have been making PSVR titles since the launch.

I think a good point to make is that it's really about "designing for 3D" across the whole game. But beyond the combat advantage, 3D audio allows us to build totally new experiences for the player. It also has an impact on the implementation, as we need to use more game objects, and play more voices (e.g. In Returnal, we have a mix of 5th Order Ambisonics, 3D Objects, 7.1 Passthrough, as well as stereo haptics and controller speaker.

As for challenges, certainly designing for 3D requires more content. There is much more space for audio to surround the player; we need to consider the location and size of objects more carefully at the design stage. The gain structure takes time and experimentation to get under control. The mixer in Returnal is 540 busses wide, with another 260 aux busses. While the technology can play hundreds of sounds in 3D, less than 5 are enough to saturate our ability to understand what is going on.

We do have more available "space" to cover with positional sounds, and more CPU to generate them, but we don't have "more brain" to understand the soundscape! The 3D mix ends up delivered in two channels of binaural stereo. That is a lot more information to print on the "same old" two waveforms.

I started in the wrong direction of treating the game like a blockbuster shooter and realized that the powerful player gun approach, as satisfying as it felt, was not right. You need to hear everything around you when you play, otherwise you're dead, haha! It led to creating a priority-based system, where all enemy sounds were assigned a priority level (this includes everything, footsteps, etc.). The higher priority sounds are given the right to affect the lower, usually by ducking them as they play.

Much of mixing the game was to define what sound needed what priority in what context, and how it should affect the rest. Obviously, while those enemies are attacking, you also have Selene firing, moving, potentially talking, the music blasting, and the UI punctuating all of this.

Haptics and 3D Audio

The advent of haptics on PS5 was another novel concept for the audio team to get to grips with and the approach for using it in Returnal took some time to figure out. Initially, attempts were made to utilize the haptics signal path as a secondary set of speakers (or a third set if you count the controller speaker) in order to invoke the same kind of response you would expect from hearing sound effects.

The initial assumption was that perhaps the haptics were capable of delivering the same kind of response - delivering information to the brain (via the hands) that would interpret it in the same way as sound. Through experimentation, we learned that the perception of haptics content is almost always swayed by the other senses. Similarly, the exact same haptic content can be perceived differently depending on the visual and audio context. For example, the same haptic pulses used for landing on solid ground after jumping will be perceived as feeling different if landing in mud (despite the content being the same).

So, with that knowledge behind us we established a general rule for what worked best: haptic content works best when all senses are in agreement. If the eyes see rain, the ears hear rain (better yet, they hear rain in 3D Audio) and the hands feel a sensation like rain. So the brain can be totally convinced by all the data it is receiving that it is indeed raining.

Challenges in Game Development

A lot of those audio concepts and iterations were also reviewed by Housemarque directors, to ensure it was fitting their creative expectations. Worth noting, it was only at a short peak of the production that we had 30 simultaneous sound designers (this includes outsourcing), and yes it was crazy haha! For the most part of our 2-year audio development on Returnal, we had around 10 people between Housemarque and Sony CSG. The reason for expanding so much was the sheer amount of varied content to produce under tight deadlines.

From the technical side, the "ever-changing" nature of the game presented some exciting challenges for us. Firstly, the world is different each cycle. The level layout is generated on-the-fly from smaller modules we called "rooms." Because there is no fixed level, we had to build systems to automatically handle the ambience beds and reverb zones as well as stitching them all together with portals to allow for propagation between rooms.

We decided that the Electromech arsenal would draw heavily from synth and electricity sources (we were keen to steer away from laser 'pew pew' sounds). For the alien fauna vocals, we wanted to use organic elements that reflected their mass and the kind of materials that make up their physical structure. For most non-humanoid creatures, resonance played an important part in making their vocal tones not only flavourful but believable. The massive bipedal monstrosity that claimed many players on their first encounter consists of numerous deep breaths, hisses, and gurgly inhales (mostly blowing air into yoghurt through plastic tubing).