Featured Research

from universities, journals, and other organizations

Smartphones might soon develop emotional intelligence: Algorithm for speech-based emotion classification developed

Date:
December 4, 2012
Source:
University of Rochester
Summary:
If you think having your phone identify the nearest bus stop is cool, wait until it identifies your mood. Engineers may soon make that possible. Researchers have developed a new computer program that gauges human feelings through speech, with substantially greater accuracy than existing approaches.

If you think having your phone identify the nearest bus stop is cool, wait until it identifies your mood. New research by a team of engineers at the University of Rochester may soon make that possible. At the IEEE Workshop on Spoken Language Technology on Dec. 5, the researchers will describe a new computer program that gauges human feelings through speech, with substantially greater accuracy than existing approaches.

Surprisingly, the program doesn't look at the meaning of the words. "We actually used recordings of actors reading out the date of the month -- it really doesn't matter what they say, it's how they're saying it that we're interested in," said Wendi Heinzelman, professor of electrical and computer engineering.

Heinzelman explained that the program analyzes 12 features of speech, such as pitch and volume, to identify one of six emotions from a sound recording. And it achieves 81 percent accuracy -- a significant improvement on earlier studies that achieved only about 55 percent accuracy.

The research has already been used to develop a prototype of an app. The app displays either a happy or sad face after it records and analyzes the user's voice. It was built by one of Heinzelman's graduate students, Na Yang, during a summer internship at Microsoft Research. "The research is still in its early days," Heinzelman added, "but it is easy to envision a more complex app that could use this technology for everything from adjusting the colors displayed on your mobile to playing music fitting to how you're feeling after recording your voice."

Heinzelman and her team are collaborating with Rochester psychologists Melissa Sturge-Apple and Patrick Davies, who are currently studying the interactions between teenagers and their parents. "A reliable way of categorizing emotions could be very useful in our research,." Sturge-Apple said. "It would mean that a researcher doesn't have to listen to the conversations and manually input the emotion of different people at different stages."

Teaching a computer to understand emotions begins with recognizing how humans do so.

"You might hear someone speak and think 'oh, he sounds angry!' But what is it that makes you think that?" asks Sturge-Apple. She explained that emotion affects the way people speak by altering the volume, pitch and even the harmonics of their speech. "We don't pay attention to these features individually, we have just come to learn what angry sounds like -- particularly for people we know," she adds.

But for a computer to categorize emotion it needs to work with measurable quantities. So the researchers established 12 specific features in speech that were measured in each recording at short intervals. The researchers then categorized each of the recordings and used them to teach the computer program what "sad," "happy," "fearful," "disgusted," or "neutral" sound like.

The system then analyzed new recordings and tried to determine whether the voice in the recording portrayed any of the known emotions. If the computer program was unable to decide between two or more emotions, it just left that recording unclassified.

"We want to be confident that when the computer thinks the recorded speech reflects a particular emotion, it is very likely it is indeed portraying this emotion," Heinzelman explained.

Previous research has shown that emotion classification systems are highly speaker dependent; they work much better if the system is trained by the same voice it will analyze. "This is not ideal for a situation where you want to be able to just run an experiment on a group of people talking and interacting, like the parents and teenagers we work with," Sturge-Apple explained.

Their new results also confirm this finding. If the speech-based emotion classification is used on a voice different from the one that trained the system, the accuracy dropped from 81 percent to about 30 percent. The researchers are now looking at ways of minimizing this effect, for example, by training the system with a voice in the same age group and of the same gender. As Heinzelman said, "there are still challenges to be resolved if we want to use this system in an environment resembling a real-life situation, but we do know that the algorithm we developed is more effective than previous attempts."


Story Source:

The above story is based on materials provided by University of Rochester. Note: Materials may be edited for content and length.


Cite This Page:

University of Rochester. "Smartphones might soon develop emotional intelligence: Algorithm for speech-based emotion classification developed." ScienceDaily. ScienceDaily, 4 December 2012. <www.sciencedaily.com/releases/2012/12/121204081324.htm>.
University of Rochester. (2012, December 4). Smartphones might soon develop emotional intelligence: Algorithm for speech-based emotion classification developed. ScienceDaily. Retrieved July 22, 2014 from www.sciencedaily.com/releases/2012/12/121204081324.htm
University of Rochester. "Smartphones might soon develop emotional intelligence: Algorithm for speech-based emotion classification developed." ScienceDaily. www.sciencedaily.com/releases/2012/12/121204081324.htm (accessed July 22, 2014).

Share This




More Computers & Math News

Tuesday, July 22, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Google Plans To Speed Up Web Pages With New Image Format

Google Plans To Speed Up Web Pages With New Image Format

Newsy (July 21, 2014) Google is using compressed images in WebP format to help boost page loading times. The files are 25-to-34 percent smaller than PNGs and JPEGs. Video provided by Newsy
Powered by NewsLook.com
Uruguayan Creates Chess Game for Multiple Opponents

Uruguayan Creates Chess Game for Multiple Opponents

AFP (July 19, 2014) It no longer takes two to play chess – or at least according to a new version of the game invented by Uruguayan Gabriel Baldi, where up to four opponents can play. Duration: 00:31 Video provided by AFP
Powered by NewsLook.com
Clock Ticks Down on Internet Speed Debate

Clock Ticks Down on Internet Speed Debate

Reuters - US Online Video (July 18, 2014) The FCC received more than 800,000 comments on whether and how internet speeds should be regulated, even crashing its system. Lily Jamali reports. Video provided by Reuters
Powered by NewsLook.com
Google Won't Call Games With In-App Add-Ons Free, Apple Will

Google Won't Call Games With In-App Add-Ons Free, Apple Will

Newsy (July 18, 2014) The European Commission asked Google and Apple not to label apps "free" if they include in-app purchases. Google has complied; Apple has resisted. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins