by Tim Killough
Music can make us feel strong emotion, but what is it about different sounds that cause us to feel one way or the other? Casady Bowman and Takashi Yamauchi have taken some steps to find out in their recent publication, Perceiving Categorical Emotion in Sound: The Role of Timbre. Musical notes can be distinguished from each other in several ways. Notes can be different pitches or be played at different volumes, and these have already been studied for their emotional impact. What remains is the lesser-studied timbre. Timbre is what makes a violin and a trumpet sound different, and is determined by multiple dimensions of acoustic features.
In this study, Bowman and Yamauchi addressed two questions: what sound qualities determine timbre, and what sound qualities determine specific emotions. To answer these questions, sounds (C notes) from a flute, clarinet, tuba, guitar, French horn, alto sax, violin, piano, trumpet, and bells were combined into 45 pairs so as to disguise individual instruments. 5 undergrad students then created four subjective emotional sounds from each pair: happy, sad, angry, and fearful. They did this by mixing and modifying parts of each pair’s frequency spectrum (to make a sound fearful, they might have removed the mid frequencies and boosted the higher and lower frequencies) In total, they created 180 distinct sounds.
Participants were divided into a timbre group and an emotion group. They listened to all 180 sounds in random order, rating them one by one. The timbre group rated how much each sample sounded like each of the ten instruments, and the emotion group rated how much each sample sounded happy, sad, angry, fearful, or disgusted. Participant ratings for timbre and emotion were matched up to 29 acoustic features that varied across sound samples. Four of those features could predict both timbre differentiation and emotional differentiation:
Envelope centroid – attack, decay, sustain, release.
Violins have a slow attack and a long release, trumpets have a fast attack and a short release. Caused a note to sound more sad or more angry.
Regularity – how jagged the frequency envelope is.
Irregular sounds are rougher and less full. High regularity sounds were happier, irregular sounds were more sad, angry, and fearful.
Sub band 3 – contains low frequencies, sounds full, powerful.
Sounds active in sub band 3 frequencies sounded more sad and fearful, and sounds without sub band 3 were happier.
Sub Band 9 – contains high frequencies, sounds bright, shiny, piercing.
Sounds active in sub band 9 frequencies sounded happier, and sounds without were angrier, sadder, and more disgusting.
These results show that timbre does affect subjective emotion of a sound. Anger can be invoked by modifying envelope centroid and removing the high frequencies of sub band 9. Fear can be increased with more irregularity and more sub band 3 (low frequencies). Disgust can only be increased by removing high frequencies. Happiness can only be directly invoked by removing the low frequencies of a sound, and sadness can be modified with all four features.
This study broke down instruments to determine the core of what makes one instrument more happy or sad than another. Future work could use these findings to go the other direction, by designing the ideal electronic instrument for each emotion and testing whether or not it has the desired effect on listeners. The current and potential future findings are useful to psychologists who use music to manipulate emotion, as well as to musicians who want to evoke certain emotions through their music.
Bowman, C., & Yamauchi, T. (2016). Perceiving categorical emotion in sound: The role of timbre. Psychomusicology: Music, Mind, and Brain, 26(1), 15-25. doi:10.1037/pmu0000105