Featured Research

from universities, journals, and other organizations

Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony

Date:
October 23, 2013
Source:
Northwestern University
Summary:
Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer. A new study did more than that. To understand how the brain responds to complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while synchronized with the light show made by the iTunes Jelly visualizer.

Scalp topography of the ASSR amplitude difference between the AV-aligned and single-task control conditions (left), and between the AV-aligned and dual-task control conditions (right). Bar to the right indicates scale.
Credit: Mossbridge JA, Grabowecky M, Suzuki S (2013) Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE 8(10): e77201. doi:10.1371/journal.pone.0077201

Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer.

A new Northwestern University study did much more than that.

To understand how the brain responds to highly complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while it was synchronized with the light show made by the iTunes Jelly visualizer.

The study shows how and when the auditory system encodes auditory-visual synchrony between complex and changing sounds and images.

Much of related research looks at how the brain processes simple sounds and images. Locating a woodpecker in a tree, for example, is made easier when your brain combines the auditory (pecking) and visual (movement of the bird) streams and judges that they are synchronous. If they are, the brain decides that the two sensory inputs probably came from a single source.

While that research is important, Julia Mossbridge, lead author of the study and research associate in psychology at Northwestern, said it also is critical to expand investigations to highly complex stimuli like music and movies.

"These kinds of things are closer to what the brain actually has to manage to process in every moment of the day," she said. "Further, it's important to determine how and when sensory systems choose to combine stimuli across their boundaries.

"If someone's brain is mis-wired, sensory information could combine when it's not appropriate," she said. "For example, when that person is listening to a teacher talk while looking out a window at kids playing, and the auditory and visual streams are integrated instead of separated, this could result in confusion and misunderstanding about which sensory inputs go with what experience."

It was already known that the left auditory cortex is specialized to process sounds with precise, complex and rapid timing; this gift for auditory timing may be one reason that in most people, the left auditory cortex is used to process speech, for which timing is critical. The results of this study show that this specialization for timing applies not just to sounds, but to the timing of complex and dynamic sounds and images.

Previous research indicates that there are multi-sensory areas in the brain that link sounds and images when they change in similar ways, but much of this research is focused particularly on speech signals (e.g., lips moving as vowels and consonants are heard). Consequently, it hasn't been clear what areas of the brain process more general auditory-visual synchrony or how this processing differs when sounds and images should not be combined.

"It appears that the brain is exploiting the left auditory cortex's gift at processing auditory timing, and is using similar mechanisms to encode auditory-visual synchrony, but only in certain situations; seemingly only when combining the sounds and images is appropriate," Mossbridge said.


Story Source:

The above story is based on materials provided by Northwestern University. The original article was written by Hilary Hurd Anyaso. Note: Materials may be edited for content and length.


Journal Reference:

  1. Julia A. Mossbridge, Marcia Grabowecky, Satoru Suzuki. Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE, 2013; 8 (10): e77201 DOI: 10.1371/journal.pone.0077201

Cite This Page:

Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. ScienceDaily, 23 October 2013. <www.sciencedaily.com/releases/2013/10/131023183248.htm>.
Northwestern University. (2013, October 23). Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony. ScienceDaily. Retrieved September 3, 2014 from www.sciencedaily.com/releases/2013/10/131023183248.htm
Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. www.sciencedaily.com/releases/2013/10/131023183248.htm (accessed September 3, 2014).

Share This



More Mind & Brain News

Wednesday, September 3, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Can You Train Your Brain To Eat Healthy?

Can You Train Your Brain To Eat Healthy?

Newsy (Sep. 1, 2014) New research says if you condition yourself to eat healthy foods, eventually you'll crave them instead of junk food. Video provided by Newsy
Powered by NewsLook.com
Coffee Then Napping: The (New) Key To Alertness

Coffee Then Napping: The (New) Key To Alertness

Newsy (Aug. 30, 2014) Researchers say having a cup of coffee then taking a nap is more effective than a nap or coffee alone. Video provided by Newsy
Powered by NewsLook.com
Young Entrepreneurs Get $100,000, If They Quit School

Young Entrepreneurs Get $100,000, If They Quit School

AFP (Aug. 29, 2014) Twenty college-age students are getting 100,000 dollars from a Silicon Valley leader and a chance to live in San Francisco in order to work on the start-up project of their dreams, but they have to quit school first. Duration: 02:20 Video provided by AFP
Powered by NewsLook.com
Baby Babbling Might Lead To Faster Language Development

Baby Babbling Might Lead To Faster Language Development

Newsy (Aug. 29, 2014) A new study suggests babies develop language skills more quickly if their parents imitate the babies' sounds and expressions and talk to them often. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins