Comprehensive Guide to Audio Testing Methods
In the realm of audio devices, discerning the characteristics that separate exceptional sound quality from mediocrity is a complex task. This article delves into the methods used to test audio devices, ensuring they meet high standards for both experts and everyday users. Furthermore, it will guide you on how to confidently evaluate audio equipment independently.
The primary goal of audio testing is to employ methods that assist users in discovering audio gear that aligns with their preferences.
Guiding Principles for Audio Testing
Two fundamental rules govern the audio testing procedures:
- Multiple Opinions: Seeking input from various individuals to avoid bias from a single reviewer's perspective.
- Comprehensive Review: Evaluating as many models as possible within a specific category and price range to provide a broad overview.
These principles sometimes conflict, especially when the number of models increases, making it challenging to gather opinions from a diverse listening panel.
Initially, a solo testing session is conducted to eliminate poorly performing models. This involves identifying characteristics that are generally undesirable, such as:
- Excessive distortion
- Extreme tonal imbalances
- Extraneous noise (hiss and hum)
For panel tests, the number of contenders is narrowed down to six or fewer to facilitate accurate comparisons. The identities of the models are concealed, particularly during speaker tests, to prevent bias.
Testing Conventional Speakers
Conventional speakers typically include box-shaped models with at least two drivers: a woofer for bass and a tweeter for treble. This category encompasses bookshelf speakers, outdoor speakers, computer speakers, and surround-sound systems. Budget and high-performance subwoofers are also tested using similar methods.
Listening panels consist of Wirecutter staff, musicians, audio professionals, and audiophiles. The speakers are hidden behind thin, black fabric to ensure anonymity. A switching device allows panelists to listen to each speaker for an extended period and switch between them, identified only by number. The numbers are randomized for each panelist to avoid any advantage due to the order of presentation.
Unbiased evaluation is crucial for assessing sound quality. Studies have shown that knowing the product's identity can influence opinions more than the actual sound. Playback levels are matched using a shaped noise tone recorded from a Dolby Digital AV receiver to eliminate the effect of bass and treble differences on perceived volume. Panelists are asked to report any volume discrepancies, which are then corrected.
Panelists are encouraged to use familiar music and movies. Additional test material is added if the panelists' selections do not adequately reveal the differences among the models. After the evaluation, the identities and prices are revealed, and panelists discuss the features and indicate which models they would buy or recommend for specific situations.
The data is then analyzed, and picks are proposed to the editors. The team collectively decides on the best options for various Wirecutter readers.
Frequency response measurements are conducted to identify any technical flaws that the listening panel might have missed. CTA-2010 bass-output measurements are performed on subwoofers and some conventional speakers to assess their ability to reproduce deep bass frequencies.
Testing Headphones and Earbuds
Testing headphones and earbuds involves a slightly different approach due to the difficulty of concealing their identities. Even with blindfolding, the feel of the headphones can provide clues about their design and brand.
Ergonomics play a significant role in the evaluation. Testers assess the comfort of the earcups and the seal provided by the ear tips. Poor fit can lead to air leakage, affecting sound quality. Any issues preventing a good fit are noted to better target recommendations.
The initial step involves eliminating underperforming models and those with operational issues, such as unreliable Bluetooth pairing. Remaining models are evaluated by the listening panel in any order, using their preferred music.
Modern headphones and earbuds often include features like noise cancellation, preset sound modes, and equalization (EQ) controls. Testers are asked to use these features and adjust the sound to their preferences. The success or failure of these adjustments is considered in the overall performance evaluation.
Volume levels are adjusted to match as closely as possible, although precise matching is challenging due to the inherent imprecision of headphone and earbud measurements. Slight changes in positioning on the test equipment can result in measured level differences.
Once the panelists share their opinions, the writer consults with the editors to finalize the picks.
Frequency response measurements are generally not performed on headphones or earbuds due to the complexity of interpreting the data. However, measurements of noise-cancelling capability are conducted.
Testing Soundbars and Bluetooth Speakers
Soundbars and Bluetooth speakers are similar to conventional speakers, but they include built-in amplification and digital signal processing (DSP). This DSP tunes the sound and provides additional functions like bass and treble controls, listening modes, and surround sound.
The testing process begins with eliminating models unlikely to receive approval from the listening panel or those with functionality problems, such as HDMI connectivity issues or Bluetooth pairing failures.
Listening tests for soundbars and Bluetooth speakers are similar to those for conventional speakers. Soundbars are tested with music and movies, while Bluetooth speakers are tested with music and podcasts. Volume levels are matched as closely as possible, although precise matching is difficult due to coarse volume adjustments.
The devices are typically used at their factory-default settings or in the mode appropriate for the content being played. After the tests, the panelists discuss the contenders' identities, prices, and features. Noteworthy sound modes are demonstrated, and Bluetooth speakers are tested at full volume to assess clarity and distortion.
Frequency response measurements are not conducted on soundbars and Bluetooth speakers due to surround-sound simulation technology and volume limiters. However, maximum-volume measurements are performed on Bluetooth speakers using pink noise and dynamic range compression recordings. CTA-2010 bass-response measurements are also conducted on subwoofers included with soundbars.
Testing Audio Devices Yourself
While professional reviews are valuable, developing your own methods for evaluating audio gear is beneficial. Although you may not have access to listening-panel tests or frequency response measurements, you can create a testing regimen to assess an audio device's performance quickly.
Select tracks that reveal common audio flaws. These tracks should include clear vocal tracks (male and female), tracks with high treble energy, and tracks with strong, deep bass. Use your own downloaded files or files ripped from CDs to avoid variations from streaming services.
Effective test tracks include:
- Holly Cole’s “Train Song”
- José González’s “Heartbeats”
- James Taylor’s “Shower the People” (live version)
- Kanye West’s “Love Lockdown”
- Tracy Chapman’s “Fast Car”
Familiarity with your test tracks is crucial. After selecting your tracks, you should be able to assess the quality of audio equipment after listening to three or four tunes.
For devices with apparent volume limitations, test them at full volume to evaluate their performance at high levels.
When testing soundbars or surround-sound systems, use action-movie DVDs or Blu-ray discs with scenes that utilize the entire sound field, including rear channels and the subwoofer.
Objective vs. Subjective Testing
Audio equipment testing involves both objective and subjective methods. Objective testing relies on scientifically verifiable performance using standardized tests in controlled conditions. Subjective testing involves personal listening experiences and preferences.
Objectivists emphasize engineering training and technical knowledge, while subjectivists rely on demonstrations and comparisons. Both approaches have their strengths and limitations.
Here’s a comparison table outlining the key differences between objective and subjective testing methods:
| Feature | Objective Testing | Subjective Testing |
|---|---|---|
| Method | Standardized tests, measurements | Listening tests, personal experience |
| Focus | Verifiable performance metrics | Perceived sound quality, user preference |
| Environment | Controlled conditions | Real-world conditions |
| Bias | Minimizes bias through controlled variables | Potential for bias based on personal taste |
| Tools | Audio analyzers, measurement equipment | Human ear, personal audio equipment |