Featured Research

from universities, journals, and other organizations

Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony

Date:
October 23, 2013
Source:
Northwestern University
Summary:
Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer. A new study did more than that. To understand how the brain responds to complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while synchronized with the light show made by the iTunes Jelly visualizer.

Scalp topography of the ASSR amplitude difference between the AV-aligned and single-task control conditions (left), and between the AV-aligned and dual-task control conditions (right). Bar to the right indicates scale.
Credit: Mossbridge JA, Grabowecky M, Suzuki S (2013) Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE 8(10): e77201. doi:10.1371/journal.pone.0077201

Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer.

A new Northwestern University study did much more than that.

To understand how the brain responds to highly complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while it was synchronized with the light show made by the iTunes Jelly visualizer.

The study shows how and when the auditory system encodes auditory-visual synchrony between complex and changing sounds and images.

Much of related research looks at how the brain processes simple sounds and images. Locating a woodpecker in a tree, for example, is made easier when your brain combines the auditory (pecking) and visual (movement of the bird) streams and judges that they are synchronous. If they are, the brain decides that the two sensory inputs probably came from a single source.

While that research is important, Julia Mossbridge, lead author of the study and research associate in psychology at Northwestern, said it also is critical to expand investigations to highly complex stimuli like music and movies.

"These kinds of things are closer to what the brain actually has to manage to process in every moment of the day," she said. "Further, it's important to determine how and when sensory systems choose to combine stimuli across their boundaries.

"If someone's brain is mis-wired, sensory information could combine when it's not appropriate," she said. "For example, when that person is listening to a teacher talk while looking out a window at kids playing, and the auditory and visual streams are integrated instead of separated, this could result in confusion and misunderstanding about which sensory inputs go with what experience."

It was already known that the left auditory cortex is specialized to process sounds with precise, complex and rapid timing; this gift for auditory timing may be one reason that in most people, the left auditory cortex is used to process speech, for which timing is critical. The results of this study show that this specialization for timing applies not just to sounds, but to the timing of complex and dynamic sounds and images.

Previous research indicates that there are multi-sensory areas in the brain that link sounds and images when they change in similar ways, but much of this research is focused particularly on speech signals (e.g., lips moving as vowels and consonants are heard). Consequently, it hasn't been clear what areas of the brain process more general auditory-visual synchrony or how this processing differs when sounds and images should not be combined.

"It appears that the brain is exploiting the left auditory cortex's gift at processing auditory timing, and is using similar mechanisms to encode auditory-visual synchrony, but only in certain situations; seemingly only when combining the sounds and images is appropriate," Mossbridge said.


Story Source:

The above story is based on materials provided by Northwestern University. The original article was written by Hilary Hurd Anyaso. Note: Materials may be edited for content and length.


Journal Reference:

  1. Julia A. Mossbridge, Marcia Grabowecky, Satoru Suzuki. Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE, 2013; 8 (10): e77201 DOI: 10.1371/journal.pone.0077201

Cite This Page:

Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. ScienceDaily, 23 October 2013. <www.sciencedaily.com/releases/2013/10/131023183248.htm>.
Northwestern University. (2013, October 23). Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony. ScienceDaily. Retrieved October 21, 2014 from www.sciencedaily.com/releases/2013/10/131023183248.htm
Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. www.sciencedaily.com/releases/2013/10/131023183248.htm (accessed October 21, 2014).

Share This



More Mind & Brain News

Tuesday, October 21, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Your Birth Season Might Determine Your Temperament

Your Birth Season Might Determine Your Temperament

Newsy (Oct. 20, 2014) A new study says the season you're born in can determine your temperament — and one season has a surprising outcome. Video provided by Newsy
Powered by NewsLook.com
Court Ruling Means Kids' Online Activity Could Be On Parents

Court Ruling Means Kids' Online Activity Could Be On Parents

Newsy (Oct. 17, 2014) In a ruling attorneys for both sides agreed was a first of its kind, a Georgia appeals court said parents can be held liable for what kids put online. Video provided by Newsy
Powered by NewsLook.com
The Best Foods To Boost Your Mood

The Best Foods To Boost Your Mood

Buzz60 (Oct. 17, 2014) Feeling down? Reach for the refrigerator, not the medicine cabinet! TC Newman (@PurpleTCNewman) shares some of the best foods to boost your mood. Video provided by Buzz60
Powered by NewsLook.com
You Can Get Addicted To Google Glass, Apparently

You Can Get Addicted To Google Glass, Apparently

Newsy (Oct. 15, 2014) Researchers claim they’ve diagnosed the first example of the disorder in a 31-year-old U.S. Navy serviceman. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Health & Medicine

Mind & Brain

Living & Well

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins