How Timbre Affects Emotional Sound

by Tim Killough

Music can make us feel strong emotion, but what is it about different sounds that cause us to feel one way or the other? Casady Bowman and Takashi Yamauchi have taken some steps to find out in their recent publication, Perceiving Categorical Emotion in Sound: The Role of Timbre. Musical notes can be distinguished from each other in several ways. Notes can be different pitches or be played at different volumes, and these have already been studied for their emotional impact. What remains is the lesser-studied timbre. Timbre is what makes a violin and a trumpet sound different, and is determined by multiple dimensions of acoustic features.

In this study, Bowman and Yamauchi addressed two questions: what sound qualities determine timbre, and what sound qualities determine specific emotions. To answer these questions, sounds (C notes) from a flute, clarinet, tuba, guitar, French horn, alto sax, violin, piano, trumpet, and bells were combined into 45 pairs so as to disguise individual instruments. 5 undergrad students then created four subjective emotional sounds from each pair: happy, sad, angry, and fearful. They did this by mixing and modifying parts of each pair’s frequency spectrum (to make a sound fearful, they might have removed the mid frequencies and boosted the higher and lower frequencies) In total, they created 180 distinct sounds.

Participants were divided into a timbre group and an emotion group. They listened to all 180 sounds in random order, rating them one by one. The timbre group rated how much each sample sounded like each of the ten instruments, and the emotion group rated how much each sample sounded happy, sad, angry, fearful, or disgusted. Participant ratings for timbre and emotion were matched up to 29 acoustic features that varied across sound samples. Four of those features could predict both timbre differentiation and emotional differentiation:

Envelope centroid – attack, decay, sustain, release.

Violins have a slow attack and a long release, trumpets have a fast attack and a short release. Caused a note to sound more sad or more angry.

Regularity – how jagged the frequency envelope is.

Irregular sounds are rougher and less full. High regularity sounds were happier, irregular sounds were more sad, angry, and fearful.

Sub band 3 – contains low frequencies, sounds full, powerful.

Sounds active in sub band 3 frequencies sounded more sad and fearful, and sounds without sub band 3 were happier.

Sub Band 9 – contains high frequencies, sounds bright, shiny, piercing.

Sounds active in sub band 9 frequencies sounded happier, and sounds without were angrier, sadder, and more disgusting.

These results show that timbre does affect subjective emotion of a sound. Anger can be invoked by modifying envelope centroid and removing the high frequencies of sub band 9. Fear can be increased with more irregularity and more sub band 3 (low frequencies). Disgust can only be increased by removing high frequencies. Happiness can only be directly invoked by removing the low frequencies of a sound, and sadness can be modified with all four features.

This study broke down instruments to determine the core of what makes one instrument more happy or sad than another. Future work could use these findings to go the other direction, by designing the ideal electronic instrument for each emotion and testing whether or not it has the desired effect on listeners. The current and potential future findings are useful to psychologists who use music to manipulate emotion, as well as to musicians who want to evoke certain emotions through their music.

References

Bowman, C., & Yamauchi, T. (2016). Perceiving categorical emotion in sound: The role of timbre. Psychomusicology: Music, Mind, and Brain, 26(1), 15-25.           doi:10.1037/pmu0000105

Should an Eyewitness be Trusted?

by Katie Nickel

It is commonly assumed that emotions improve memory, but would you be willing to risk someone’s freedom on it? Imagine yourself shopping in a convenience store when you witness a man at the counter assault the young attendant and steal cash from the register. You are still experiencing unpleasant emotions from this event as the police take your statement, but you are able to give a detailed description of the perpetrator. A few days later the police ask you to identify the offender from a photographic lineup. You point to the man you think committed the crime and continue on with your day, but were you able to identify the right man? Or did the negative emotions you experienced during the crime cause you to accuse an innocent person? Houston and her colleagues conducted two experiments to explore these questions.

Houston’s experiments used student participants divided into two groups to test how negative emotions affect an individual’s memory of an event, as well as their ability to identify the offender from the event. Each group watched a different, pretested film designed to produce specific emotions. One group acquired negative emotions after viewing a man mugging an elderly woman, while the other group remained neutral after watching a film with all of the same variables as the first, but no mugging. When asked to recall details of the event, the emotionally stimulated group remembered more details about the perpetrator than the neutral group, but their descriptions were no more accurate than the other group. In the second experiment, the participants were shown the same films and then asked to identify the offender in a photographic lineup. The group with heightened emotions picked the wrong man more often than the other group, showing that negative emotions impaired their recognition abilities.

Although these experiments used only student participants and relied on films to produce emotions instead of more natural situations, this data is useful in identifying the ill effects negative emotions have on memory and recognition skills. During an event, like the robbery of a convenience store, the stressful situation makes it challenging for the witness to recognize the face of the perpetrator. While further studies are needed to verify the validity of these experiments, the findings challenge the common belief that emotional experiences aid memory retrieval.

References

Houston, K. A., Clifford, B. R., Phillips, L. H., & Memon, A. (2013). The emotional eyewitness: The effects of emotion on specific aspects of eyewitness recall and recognition performance. Emotion, 13(1), 118-128.

Electroconvulsive Therapy Improves the Ability to Recognize Disgust in People With Schizophrenia

by Madelaine Morton

For most of us, reading other people’s emotions is an automatic and essential part of daily interaction. Knowing your partner had a bad day the moment they walk in the door or avoiding your dad when he looks grumpy is a skill that many don’t need to put conscious effort into. Emotion recognition is not a hard-wired ability for everyone though. For example, those with schizophrenia tend to have difficulties recognizing emotional facial expressions, especially those conveying disgust.

Mixed results have been reported in previous studies examining the effect of antipsychotic medication on people with schizophrenia’s ability to interpret facial expressions. In addition to medication, electroconvulsive therapy (ECT) is sometimes used to treat symptoms of schizophrenia. This somewhat controversial treatment involves inducing seizures in the patient by passing small electric currents through the brain while under careful medical supervision.

Dr. Mihriban Dalkıran and her team wondered what effect ECT might have on the accuracy of recognizing facial expressions in people with schizophrenia and designed a study to find out. They predicted that patients would be better at recognizing emotions, specifically disgust, after being treated and would be faster at identifying facial emotions.

Thirty individuals who were already diagnosed with schizophrenia and recommended ECT in addition to their previously prescribed medication participated in Dalkıran’s study. The day before the scheduled ECT, participants were asked to perform an exercise in reading facial expressions. The exercise, officially called the Facial Emotion Recognition Test, consisted of 56 mixed photos of four male and four female models displaying sad, happy, fearful, angry, surprised, disgusted, and neutral facial expressions. Participants were asked to identify the emotions being expressed and the time taken to respond was noted. The next day, subjects received ECT and, within a week, were asked to complete the exercise again. Results between the two instances were then compared.

The researchers discovered that the people with schizophrenia’s ability to recognize disgust rom the facial expressions improved after ECT, but this effect was not seen with other emotions. Additionally, they were faster at identifying happy and fearful expressions. The researchers believe that ECT may have an effect on the right insula, a certain part of the brain which has been implicated in facial recognition of disgust. These findings are important because they provide psychologists with clues as to why people with schizophrenia have difficulties reading emotions and what avenues may be explored to help them overcome such a socially crippling symptom of the illness. While this study was the first to examine these particular effects, future studies in this area would benefit from having half of the participants not undergo ECT to act as a comparison group. This would allow the researchers to be confident that it was the ECT itself that caused the change in the participants, not anxiety of going through a procedure or some other explanation. Future research could also look at how results might differ in a more realistic setting, such as in a conversation with another person; the laboratory setting may have influenced the participants to respond differently than they would in a day-to-day  situation.

References

Dalkıran, M., Tasdemir, A., Salihoglu, T., et al. (2016). The change in facial emotion recognition ability in inpatients with treatment resistant schizophrenia after electroconvulsive therapy. Psychiatric Quarterly. doi:10.1007/s11126-016-9466-7

Remember When: Recalling Positive Memories Through Music

by Julianna Facchinelli

Neurologist Oliver Sacks was onto something when he said “Music evokes emotion and emotion can bring it’s memory”. We all have those songs that make us respond with a groan and perhaps a few curse words every time we hear them on the radio, but it is equally true that there are songs that delightfully transport us to a vivid moment from our past. As humans, it is common, even expected, to remember the emotional aspects of a memory, and musical cues can aid in bringing those memories to mind.

Researchers Jaclyn Ford, David Rubin, and Kelly Giovanello were curious about these notions regarding the musical connection to positive memories; they figured that listening to familiar songs would allow people to recall specific personal memories of a positive nature associated with that song. Makes sense, right? I think we’d be hard-pressed to think of an emotional, personal moment in time that wasn’t accompanied by music (sobbing to breakup songs, anyone ?). In their study, fifteen young adults and sixteen older adults were presented with snippets of popular songs from either the 2000s or the 1950s, which tailored the songs to the ages of the participants. They were asked to retrieve personal memories that immediately came to mind when listening to the songs and rated song familiarity and memory pleasantness. The study found that songs rated as highly familiar were more likely to allow the individuals to remember memories that were specific and very positive . Therefore, familiar songs increased the chance of retrieving joyful emotional memories, and this effect was found in both the young adults and the older adults. Interestingly, brain imagery taken during memory recall revealed activity in areas of the brain responsible for emotional memory processing and retrieval for both age groups. These results support the idea that the emotional memories linked to familiar music can be longstanding and brought to mind just by hearing a familiar song.

However, since the study only used common songs from specific decades, it would not be generalizable to non-familiar songs or songs from outside those time frames. It would be interesting to see this study tested out with other age groups, such as children, or even individuals with memory impairments to explore the extent of positive memory retrieval using musical influences- would it work for people suffering from memory loss? Another consideration could be the nature of the songs played: music selections were generally very positive. Future research might include sad songs to potentially retrieve negative memories, an element which this current study is lacking. Still, the findings are promising, so pay attention the next time you’re listening to your tunes—you may just be making some new memories.

Ford, J.H., Rubin, D.C., & Giovanello, K. (2016). The effects of song familiarity and age on phenomenological characteristics and neural recruitment during autobiographical memory retrieval. Psychomusicology: Music, Mind, and Brain, 26(3), 199-210. doi:10.1037/pmu0000152

 

 

Social anxiety may cause you to see others as a threat, even when they’re not

by Kristyn Trickey, Thompson Rivers University

Think about the last time you were walking in a public place, or interacting one-on-one with another human being. In our daily social interactions, we are constantly putting out ever-changing clues as to the emotions we are feeling inside; these may include our body language, tone of voice, and, notably, our facial expressions. In addition to helping us understand what those around us may be feeling, facial expressions also let us know when we can expect someone to be friendly and welcoming (eg. smiling face), or when they might be better left alone (eg. an angry face). Being aware of when a threat may arise in a social situation is important, but for some, particularly those who experience social anxiety, this system of detecting threat may be overly sensitive, leading us to avoid or be fearful of others when it is often unnecessary.

Researchers Gutierrez-Garcia and Calvo wanted to uncover more about this. Do people experiencing social anxiety truly see others as more threatening than those without social anxiety? Therefore, would they be more likely to rate others’ facial expressions as untrustworthy? Does it make a difference whether the emotion expressed has a positive (happiness), negative (anger or disgust), or neutral (sadness, surprise or fear) connotation?

To test this out, they recruited two groups of university students. One group met the criteria for having a high level of social anxiety, while the other group had a low level. The students were shown short video clips of real people with six different facial expressions (happy, sad, angry, disgusted, surprised, fearful). Each participant viewed all the expressions at multiple intensities, including the full-blown emotional expressions, more subtle versions of the expressions (eg. 50% intensity), and purely neutral faces. In a split-second participants had to decide how “trustworthy” or “untrustworthy” each face was.

So, what did they find? Those with social anxiety do tend to rate people as appearing more untrustworthy, BUT only when those people looked angry or disgusted. Students with social anxiety were better at picking out these emotions, even when the expression was subtle; on the other hand, their ratings of positive and neutral emotions did not differ from the students with low social anxiety. However, a problem with this study is that the facial expressions shown, though morphing, were posed and potentially unrealistic (eg. more intense than what is seen in everyday life). A question for future research could be whether this sensitivity to facial changes can ever have an adaptive, protective function, or if it simply unhelpful. Keep this in mind next time you spot an angry face on your morning walk – perhaps what you see could be deceiving!

References

Gutierrez-Garcia, A. & Calvo, M. G. (2016). Social anxiety and trustworthiness judgements of dynamic facial expressions of emotion. Journal of Behavior Therapy and Experimental Psychiatry, 52, 119-127. doi:10.1016/j.jbtep.2016.04.003

Psychology of Emotion (Psyc3380) class blog posts

Students from the fall 2016 class of Psyc3380 will be posting recent research findings of general interest on the topic of emotion.

Jealousy and Cell Phone Usage in the Modern World

by: Shayla W. Piccini, Thompson Rivers University

In modern times, technology continues to become increasingly popular, especially with the use of cell phones. Around 82% of adults around the world own a cell phone, allowing us to easily communicate with others and form relationships; however, they also have the ability to aid in the destruction of relationships. Studies on infidelity and cell phone usage have found that phones can expedite the process of cheating. This darker side of cell phone usage reveals the jealousy in human nature.

Some psychologists use a Darwinian approach to examine this issue, arguing that jealousy could have been adaptive in ancient times by helping to ward off competition between mates. It can also help to ensure that one’s resources aren’t wasted trying to reproduce with an unfaithful mate. Studies have found that men seem to be more distressed by evidence of sexual infidelity, while females are more distressed by evidence of emotional infidelity. With all of this in mind, Michael Dunn and Holly Mclean set out to examine cell phone usage through this perspective. They looked at the difference between males’ and females’ focus on emotional versus sexual texts that revealed an imagined partner was cheating.

To test their theory, Dunn and Mclean recruited 42 single, heterosexual university students (20 males and 22 females). Participants were asked to imagine that they were currently in a relationship and were then shown two emotionally explicit texts and two sexually explicit texts. For example, one of the emotional text messages read “I think I’m falling in love with you!” and one of the sexually explicit texts read “I enjoyed last night, still up for another shag next week?”. While participants viewed the texts, they wore a device which tracked their eye movement, letting the researchers know how long they spent looking at each text. The results showed that females focused on both the sexual and emotional texts for equal amounts of time, while males focused on the sexual texts more than females did.

Overall, females spent more time than males looking at the emotional texts and males spent more time than females looking at the sexual texts. People tend to focus more on things that are important to them, so these results indicate that emotional infidelity matters more to females and sexual infidelity matters more to males. These results are important because they provide insight into how an evolutionary perspective of jealousy can be linked to the modern world through cell phone usage. Future studies could look at possible factors such as sexual habits, sexual orientation, and age. Sexual habits could influence future results, as some people who are in open relationships or tend to be more promiscuous may find these texts to be of less importance. While people in same sex relationships may focus on both types of texts equally or to a lesser extent depending on their relationship. Finally, older participants may focus more or less on either the sexual or emotional texts based on their own life experiences.

References

Dunn, M., & Mclean, H. (2015). Jealousy-induced sex differences in eye gaze directed at either emotional- or sexual infidelity-related mobile phone messages. Cyberpsychology, Behaviour and Social Networking, 18 (1), 37-40. doi:10.1089/cyber.2014.0351