Understanding Emotional Prosody: The Melody of Speech
Speech involves more than just the words we use; it encompasses how we say them. This “how” is known as prosody, referring to the pitch, loudness, and timing of speech. The term "prosody" originates from the Greek word prosōidia, meaning "song" or "melody." Therefore, prosody is often regarded as the melody of speech.
Without prosody, our speech would sound robotic and could be challenging to understand.
When we speak, we emphasize specific words by elevating our pitch, pronouncing them louder, and elongating them. By emphasizing words in this manner, we convey particular meanings, such as making a correction or introducing a new subject. Thanks to prosody, individuals can discern whether we are posing a question (“You know Nina?”) or making a statement (“You know Nina!”).
Prosody also enables us to communicate emotion in our speech and convey our unique speaking style. Each person possesses a unique way of speaking. Thanks to prosody, we can determine if someone is happy, angry, sad, or bored. Before the advent of cell phones and caller IDs, individuals could call a family's landline and identify the speaker solely based on their voice and speaking style. That distinctive speaking style is prosody.
The Two Main Communication Purposes of Prosody
Prosody serves two primary communication purposes: affective and augmentative.
- Affective prosody: This occurs when a speaker employs prosody to express emotion, mental state, speaker attitude, speech act, or other information conveyed sentence by sentence.
- Augmentative prosody: This is used to disambiguate and reinforce the verbal component.
We can describe prosody as the aspect of human communication that expresses emotion, emphasizes words, indicates speaker attitude, divides a sentence into phrases, governs sentence rhythm, and controls the intonation, pitch, or tune of the utterance.
Prosody is a speech property responsible for various paralinguistic functions, while emotion refers to the speaker's personal status. Emotional prosody is the most fundamental type of prosody, enabling people to convey or understand emotion.
Acoustic Elements of Emotional Prosody
In linguistics and the study of human behavior, emotional prosody refers to the modulation of acoustic elements in speech-such as pitch, rhythm, intensity, and duration-to express emotions. This subtle yet powerful phenomenon works alongside the semantic content of speech, adding an emotionally rich layer to verbal expression.
- Pitch: Arguably the most prominent feature of emotional prosody, pitch plays a key role in conveying emotional states. A wider pitch range and higher average pitch are often associated with positive emotions like joy or excitement.
- Rhythm: The rhythmic aspects of speech, including tempo and timing, are key components of emotional prosody, the vocal patterns that convey emotions and intent. Speaking at an accelerated tempo often conveys excitement, urgency, or enthusiasm. Pauses, both silent and vocalized, are powerful tools for emotional expression.
- Intensity: Intensity, which relates to how loud or soft speech is, helps express emotions. In emotional communication, loudness plays a more prominent role than pitch in influencing perceptions of truthfulness.
- Duration: Duration refers to how long sounds, pauses, and speech patterns last. Long vowels, extended pauses, or fast speech all add meaning to emotions.
Research advocates for analyzing speech holistically by combining prosodic elements like pauses and loudness with additional lexical and speaker-specific features.
The Interplay of Verbal and Vocal Channels
Language can be divided into two components: the verbal and vocal channels. The verbal channel is the semantic content made by the speaker's chosen words. In the verbal channel, the semantic content of the speakers words determines the meaning of the sentence. The way a sentence is spoken, however, can change its meaning which is the vocal channel. This channel of language conveys emotions felt by the speaker and gives us as listeners a better idea of the intended meaning. Nuances in this channel are expressed through intonation, intensity, a rhythm which combined for prosody. Usually these channels convey the same emotion, but sometimes they differ.
Neurological processes integrating verbal and vocal (prosodic) components are relatively unclear. However, it is assumed that verbal content and vocal are processed in different hemispheres of the brain. Verbal content composed of syntactic and semantic information is processed in the left hemisphere. Syntactic information is processed primarily in the frontal regions and a small part of the temporal lobe of the brain while semantic information is processed primarily in the temporal regions with a smaller part of the frontal lobes incorporated. In contrast, prosody is processed primarily in the same pathway as verbal content, but in the right hemisphere.
Neuroimaging studies using functional magnetic resonance imaging (fMRI) machines provide further support for this hemisphere lateralization and temporo-frontal activation. Some studies however show evidence that prosody perception is not exclusively lateralized to the right hemisphere and may be more bilateral.
Cultural and Individual Influences
Culture and personal traits both shape how emotions are expressed through speech. Different cultures have unique ways of using tone and rhythm to show feelings. At the same time, personal traits add another layer of variety. Personality, thinking styles, and life experiences can change how someone expresses or understands emotions. For example, an outgoing person might use more dramatic tones, while a quieter person might express emotions gently. These differences show how both culture and individuality influence emotional communication.
Applications in Various Fields
Understanding emotional prosody goes far beyond theoretical interest, playing a significant role in numerous practical fields such as human-computer interaction, artificial intelligence, and clinical psychology.
- For instance, the ability to recognize and replicate emotional prosody in speech synthesis systems has been instrumental in creating virtual assistants that feel more natural and emotionally engaging.
- In clinical settings, the analysis of emotional prosody offers valuable insights into psychological health. It has proven useful in diagnosing and treating a range of psychological disorders, such as autism spectrum disorders, mood disorders, and even depression.
- For individuals with autism, prosody analysis can highlight challenges in emotional expression, providing therapists with actionable data to guide interventions.
- Emotional prosody research has implications for education and social training. Programs designed to enhance emotional literacy and communication skills can use prosody insights to teach individuals how to better express and interpret emotions through speech.
Emotional prosody serves as a crucial link between people as well as a captivating bridge between linguistics and psychology, unraveling the intricate ways our voices encode and decode emotions.
Prosodic Impairments and Neurological Conditions
Neurological conditions such as ataxia, Parkinson’s Disease, and Amyotrophic Lateral Sclerosis frequently result in what is called dysarthria: a neurological impairment in the execution of speech. There are unique dysarthria characteristics for each neurological condition. Dysarthria in ataxia, termed “ataxic dysarthria” by speech-language pathologists, is uniquely characterized by variable prosody.
Variable prosody means that prosody is not consistent during speech, but rather varies, sometimes unpredictably. For example, sometimes people talk louder at the start of the phrase, called “explosive loudness.” Other times, people may raise and lower their pitch more than what is typical while talking. Lastly, it is common that the timing of words and pauses between words is inconsistent.
Prosodic impairments in ataxia range from mild to severe and can impact how the speaker’s message is interpreted. Some people with ataxia have reported that people think they are angry when they are not because they are speaking too loudly. Other people say they are often misinterpreted, potentially because of stressing the wrong word while talking.
Research is still being completed to find effective therapies for improving prosody in ataxia. There is currently no gold standard for treatment. The main recommendation is to seek a referral for an evaluation by a speech-language pathologist to determine your own unique prosodic difficulties, and what may be causing them.
For example, if you have explosive loudness bursts while talking, it may be beneficial to work on improved breath control while speaking. This could include learning the effective amount of air to inhale before speaking and how to better control exhalation.
Aprosodia
Prosody refers to the melodic and rhythmic aspects of speech. Two forms of prosody are typically distinguished: ‘affective prosody’ refers to the expression of emotion in speech, whereas ‘linguistic prosody’ relates to the intonation of sentences, including the specification of focus within sentences and stress within polysyllabic words. While these two processes are united by their use of vocal pitch modulation, they are functionally distinct.
RHD often causes difficulty with producing or understanding prosody, especially emotional prosody. This disorder is called aprosodia. Aprosodia often results in “flat” sounding or monotone speech. This is often coupled with minimal change in facial expressions and body language, making it hard to read that person’s emotions or intentions (was he joking or being serious?). The person may also have difficulty understanding others’ use of prosody and body language. This can cause misunderstandings and make it appear like the person with RHD is being insensitive to their partners’ emotions and subtle meaning.
Decoding Emotions in Speech
Decoding emotions in speech includes three stages: determining acoustic features, creating meaningful connections with these features, and processing the acoustic patterns in relation to the connections established. In the processing stage, connections with basic emotional knowledge is stored separately in memory network specific to associations. These associations can be used to form a baseline for emotional expressions encountered in the future.
On average, listeners are able to perceive intended emotions exhibited to them at a rate significantly better than chance (chance=approximately 10%). However, error rates are also high.
Age-Related Changes in Prosody Perception
It has been found that it gets increasingly difficult to recognize vocal expressions of emotion with increasing age. Older adults have slightly more difficulty labeling vocal expressions of emotion, particularly sadness and anger than young adults but have a much greater difficulty integrating vocal emotions and corresponding facial expressions.
A possible explanation for this difficulty is that combining two sources of emotion requires greater activation of emotion areas of the brain, in which adults show decreased volume and activity. Another possible explanation is that hearing loss could have led to a mishearing of vocal expressions.
Gender Differences in Emotional Prosody
Men and women differ in both how they use language and also how they understand it. It is known that there is a difference in the rate of speech, the range of pitch, and the duration of speech, and pitch slope.
For example, "In a study of relationship of spectral and prosodic signs, it was established that the dependence of pitch and duration differed in men and women uttering the sentences in affirmative and inquisitive intonation. Tempo of speech, pitch range, and pitch steepness differ between the genders".
Women and men are also different in how they neurologically process emotional prosody. In an fMRI study, men showed a stronger activation in more cortical areas than female subjects when processing the meaning or manner of an emotional phrase. In the manner task, men had more activation in the bilateral middle temporal gyri. For women, the only area of significance was the right posterior cerebellar lobe.
Male subjects in this study showed stronger activation in the prefrontal cortex, and on average needed a longer response time than female subjects. This result was interpreted to mean that men need to make conscious inferences about the acts and intentions of the speaker, while women may do this sub-consciously.