How Timbre Affects Emotional Sound

by Tim Killough

Music can make us feel strong emotion, but what is it about different sounds that cause us to feel one way or the other? Casady Bowman and Takashi Yamauchi have taken some steps to find out in their recent publication, Perceiving Categorical Emotion in Sound: The Role of Timbre. Musical notes can be distinguished from each other in several ways. Notes can be different pitches or be played at different volumes, and these have already been studied for their emotional impact. What remains is the lesser-studied timbre. Timbre is what makes a violin and a trumpet sound different, and is determined by multiple dimensions of acoustic features.

In this study, Bowman and Yamauchi addressed two questions: what sound qualities determine timbre, and what sound qualities determine specific emotions. To answer these questions, sounds (C notes) from a flute, clarinet, tuba, guitar, French horn, alto sax, violin, piano, trumpet, and bells were combined into 45 pairs so as to disguise individual instruments. 5 undergrad students then created four subjective emotional sounds from each pair: happy, sad, angry, and fearful. They did this by mixing and modifying parts of each pair’s frequency spectrum (to make a sound fearful, they might have removed the mid frequencies and boosted the higher and lower frequencies) In total, they created 180 distinct sounds.

Participants were divided into a timbre group and an emotion group. They listened to all 180 sounds in random order, rating them one by one. The timbre group rated how much each sample sounded like each of the ten instruments, and the emotion group rated how much each sample sounded happy, sad, angry, fearful, or disgusted. Participant ratings for timbre and emotion were matched up to 29 acoustic features that varied across sound samples. Four of those features could predict both timbre differentiation and emotional differentiation:

Envelope centroid – attack, decay, sustain, release.

Violins have a slow attack and a long release, trumpets have a fast attack and a short release. Caused a note to sound more sad or more angry.

Regularity – how jagged the frequency envelope is.

Irregular sounds are rougher and less full. High regularity sounds were happier, irregular sounds were more sad, angry, and fearful.

Sub band 3 – contains low frequencies, sounds full, powerful.

Sounds active in sub band 3 frequencies sounded more sad and fearful, and sounds without sub band 3 were happier.

Sub Band 9 – contains high frequencies, sounds bright, shiny, piercing.

Sounds active in sub band 9 frequencies sounded happier, and sounds without were angrier, sadder, and more disgusting.

These results show that timbre does affect subjective emotion of a sound. Anger can be invoked by modifying envelope centroid and removing the high frequencies of sub band 9. Fear can be increased with more irregularity and more sub band 3 (low frequencies). Disgust can only be increased by removing high frequencies. Happiness can only be directly invoked by removing the low frequencies of a sound, and sadness can be modified with all four features.

This study broke down instruments to determine the core of what makes one instrument more happy or sad than another. Future work could use these findings to go the other direction, by designing the ideal electronic instrument for each emotion and testing whether or not it has the desired effect on listeners. The current and potential future findings are useful to psychologists who use music to manipulate emotion, as well as to musicians who want to evoke certain emotions through their music.

References

Bowman, C., & Yamauchi, T. (2016). Perceiving categorical emotion in sound: The role of timbre. Psychomusicology: Music, Mind, and Brain, 26(1), 15-25.           doi:10.1037/pmu0000105

Should an Eyewitness be Trusted?

by Katie Nickel

It is commonly assumed that emotions improve memory, but would you be willing to risk someone’s freedom on it? Imagine yourself shopping in a convenience store when you witness a man at the counter assault the young attendant and steal cash from the register. You are still experiencing unpleasant emotions from this event as the police take your statement, but you are able to give a detailed description of the perpetrator. A few days later the police ask you to identify the offender from a photographic lineup. You point to the man you think committed the crime and continue on with your day, but were you able to identify the right man? Or did the negative emotions you experienced during the crime cause you to accuse an innocent person? Houston and her colleagues conducted two experiments to explore these questions.

Houston’s experiments used student participants divided into two groups to test how negative emotions affect an individual’s memory of an event, as well as their ability to identify the offender from the event. Each group watched a different, pretested film designed to produce specific emotions. One group acquired negative emotions after viewing a man mugging an elderly woman, while the other group remained neutral after watching a film with all of the same variables as the first, but no mugging. When asked to recall details of the event, the emotionally stimulated group remembered more details about the perpetrator than the neutral group, but their descriptions were no more accurate than the other group. In the second experiment, the participants were shown the same films and then asked to identify the offender in a photographic lineup. The group with heightened emotions picked the wrong man more often than the other group, showing that negative emotions impaired their recognition abilities.

Although these experiments used only student participants and relied on films to produce emotions instead of more natural situations, this data is useful in identifying the ill effects negative emotions have on memory and recognition skills. During an event, like the robbery of a convenience store, the stressful situation makes it challenging for the witness to recognize the face of the perpetrator. While further studies are needed to verify the validity of these experiments, the findings challenge the common belief that emotional experiences aid memory retrieval.

References

Houston, K. A., Clifford, B. R., Phillips, L. H., & Memon, A. (2013). The emotional eyewitness: The effects of emotion on specific aspects of eyewitness recall and recognition performance. Emotion, 13(1), 118-128.

Electroconvulsive Therapy Improves the Ability to Recognize Disgust in People With Schizophrenia

by Madelaine Morton

For most of us, reading other people’s emotions is an automatic and essential part of daily interaction. Knowing your partner had a bad day the moment they walk in the door or avoiding your dad when he looks grumpy is a skill that many don’t need to put conscious effort into. Emotion recognition is not a hard-wired ability for everyone though. For example, those with schizophrenia tend to have difficulties recognizing emotional facial expressions, especially those conveying disgust.

Mixed results have been reported in previous studies examining the effect of antipsychotic medication on people with schizophrenia’s ability to interpret facial expressions. In addition to medication, electroconvulsive therapy (ECT) is sometimes used to treat symptoms of schizophrenia. This somewhat controversial treatment involves inducing seizures in the patient by passing small electric currents through the brain while under careful medical supervision.

Dr. Mihriban Dalkıran and her team wondered what effect ECT might have on the accuracy of recognizing facial expressions in people with schizophrenia and designed a study to find out. They predicted that patients would be better at recognizing emotions, specifically disgust, after being treated and would be faster at identifying facial emotions.

Thirty individuals who were already diagnosed with schizophrenia and recommended ECT in addition to their previously prescribed medication participated in Dalkıran’s study. The day before the scheduled ECT, participants were asked to perform an exercise in reading facial expressions. The exercise, officially called the Facial Emotion Recognition Test, consisted of 56 mixed photos of four male and four female models displaying sad, happy, fearful, angry, surprised, disgusted, and neutral facial expressions. Participants were asked to identify the emotions being expressed and the time taken to respond was noted. The next day, subjects received ECT and, within a week, were asked to complete the exercise again. Results between the two instances were then compared.

The researchers discovered that the people with schizophrenia’s ability to recognize disgust rom the facial expressions improved after ECT, but this effect was not seen with other emotions. Additionally, they were faster at identifying happy and fearful expressions. The researchers believe that ECT may have an effect on the right insula, a certain part of the brain which has been implicated in facial recognition of disgust. These findings are important because they provide psychologists with clues as to why people with schizophrenia have difficulties reading emotions and what avenues may be explored to help them overcome such a socially crippling symptom of the illness. While this study was the first to examine these particular effects, future studies in this area would benefit from having half of the participants not undergo ECT to act as a comparison group. This would allow the researchers to be confident that it was the ECT itself that caused the change in the participants, not anxiety of going through a procedure or some other explanation. Future research could also look at how results might differ in a more realistic setting, such as in a conversation with another person; the laboratory setting may have influenced the participants to respond differently than they would in a day-to-day  situation.

References

Dalkıran, M., Tasdemir, A., Salihoglu, T., et al. (2016). The change in facial emotion recognition ability in inpatients with treatment resistant schizophrenia after electroconvulsive therapy. Psychiatric Quarterly. doi:10.1007/s11126-016-9466-7