How to Create 3D Audio: A Guide to Binaural Sound Technology
Imagine closing your eyes and letting sound guide you. You’ll reach your destination as simply and precisely as if you were following the directions on your usual navigation application. This is the promise of 3D audio, achieved through binaural sound technology.
The adventure began 7 years ago, when François Alouges, a professor at Polytechnique’s Centre de Mathématiques Appliquées (CMAP), and his team investigated the applications of “binaural sound”. Binaural sound is to hearing what 3D vision glasses are to sight.
Sylvain Ferrand, a research engineer, had heard of blind athletes using sound guidance for rollerblading or running. The athlete is guided by the sound of his guide’s footsteps or roller skates. Following in the footsteps of these sportsmen and women, the engineer and his team imagined enabling visually impaired people to find their way by following a sound.
Understanding Binaural Sound
Meaning “relating to both ears”, binaural sound is based on a method of capturing sound adapted to the morphology of the human head. When we look at an object, each eye has its own field of vision (our right eye sees the same image as our left eye but offset by a few centimetres). Similarly, if a noise is emitted to the right of our head, our left ear will hear it slightly out of phase.
In both cases, it is our brain that is responsible for reproducing a single image or a single sound. The same applies to intensity. With a loud sound, the brain realises that the source is close. With a weaker, more muffled sound, it will deduce that the source is far away. Our auditory system is designed to hear particularly well facing us, in the direction we are looking.
Recreating the Three-Dimensional Effect
But how do we recreate this three-dimensional effect? By the shape and placement of the microphones. Installed so that their capsules are 18 centimetres apart (the average spacing between the two ears in humans), they are mounted on a support that simulates the shape of a head. The team develops signal processing algorithms to reproduce these characteristics and simulate virtual sound sources from monophonic sounds: this is binaural synthesis.
For the result to be perfect, the binaural listening system needs to be connected to a head-tracker, a sensor that takes into account the user’s head movements in real time. So if the user turns their head to the right, the violin they were hearing in front of them, for example, will be on their left. The same applies to all sounds, wherever they come from.
There are already several technologies that exist for the three-dimensional reproduction of sound, but they require several speakers spread around a room, such as Dolby Atmos. A quick reminder of the benefits of this technique: the stereo sound we are used to hearing, patented in the 1930s, was already intended to reconstitute the spatial distribution of sound sources. To do this, recordings from two microphones are played back through two loudspeakers or two headphones.
In stereo, when listening to a symphony orchestra from your sofa, for example, you can distinguish the sound of the piano on the left from that of the double bass on the right. But binaural sound offers far more realistic immersion.
Applications and Development
Binaural sound can be a powerful guidance tool if it is coupled with a head-tracking system. The sound source indicating direction remains independent of head orientation. As part of his thesis, Sylvain Ferrand tested and developed an initial adaptive guidance prototype with visually impaired people. A sound source continuously precedes the person to indicate the path to follow.
To be used in a sporting context, the device had to be extremely responsive and lightweight. “The first prototype weighed 2.7 kilos. Now we’re using standard headphones weighing just a few grams in conjunction with a smartphone”, explains Sylvain Ferrand.
What can guide the visually impaired could also help anyone trying to take their eyes off the smartphone while following a guidance application. In 2019, the CMAP researchers joined forces to develop a start-up as part of the Polytechnique incubator, the Drahi‑X novation centre. They were joined by Philippe Le Borgne, an entrepreneur with over twenty years’ experience in IT and social and environmental impact companies.
The trio are now looking for funding to finalise the project and offer a software kit that can be integrated into various navigation applications and implemented on headsets and headphones already marketed to the general public (Apple, Sony, Google…). The algorithm still requires development and optimisation. But tomorrow, an ordinary user could set off to discover a city, on foot with his nose to the wind, simply guided by the sound of his music.
“Walking around following a sound is totally intuitive, you don’t have to learn anything, and the precision is extreme, to the order of a few degrees”, explain the co-founders.