Featured Research

from universities, journals, and other organizations

Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony

Date:
October 23, 2013
Source:
Northwestern University
Summary:
Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer. A new study did more than that. To understand how the brain responds to complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while synchronized with the light show made by the iTunes Jelly visualizer.

Scalp topography of the ASSR amplitude difference between the AV-aligned and single-task control conditions (left), and between the AV-aligned and dual-task control conditions (right). Bar to the right indicates scale.
Credit: Mossbridge JA, Grabowecky M, Suzuki S (2013) Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE 8(10): e77201. doi:10.1371/journal.pone.0077201

Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer.

A new Northwestern University study did much more than that.

To understand how the brain responds to highly complex auditory-visual stimuli like music and moving images, the study tracked parts of the auditory system involved in the perceptual processing of "Moonlight Sonata" while it was synchronized with the light show made by the iTunes Jelly visualizer.

The study shows how and when the auditory system encodes auditory-visual synchrony between complex and changing sounds and images.

Much of related research looks at how the brain processes simple sounds and images. Locating a woodpecker in a tree, for example, is made easier when your brain combines the auditory (pecking) and visual (movement of the bird) streams and judges that they are synchronous. If they are, the brain decides that the two sensory inputs probably came from a single source.

While that research is important, Julia Mossbridge, lead author of the study and research associate in psychology at Northwestern, said it also is critical to expand investigations to highly complex stimuli like music and movies.

"These kinds of things are closer to what the brain actually has to manage to process in every moment of the day," she said. "Further, it's important to determine how and when sensory systems choose to combine stimuli across their boundaries.

"If someone's brain is mis-wired, sensory information could combine when it's not appropriate," she said. "For example, when that person is listening to a teacher talk while looking out a window at kids playing, and the auditory and visual streams are integrated instead of separated, this could result in confusion and misunderstanding about which sensory inputs go with what experience."

It was already known that the left auditory cortex is specialized to process sounds with precise, complex and rapid timing; this gift for auditory timing may be one reason that in most people, the left auditory cortex is used to process speech, for which timing is critical. The results of this study show that this specialization for timing applies not just to sounds, but to the timing of complex and dynamic sounds and images.

Previous research indicates that there are multi-sensory areas in the brain that link sounds and images when they change in similar ways, but much of this research is focused particularly on speech signals (e.g., lips moving as vowels and consonants are heard). Consequently, it hasn't been clear what areas of the brain process more general auditory-visual synchrony or how this processing differs when sounds and images should not be combined.

"It appears that the brain is exploiting the left auditory cortex's gift at processing auditory timing, and is using similar mechanisms to encode auditory-visual synchrony, but only in certain situations; seemingly only when combining the sounds and images is appropriate," Mossbridge said.


Story Source:

The above story is based on materials provided by Northwestern University. The original article was written by Hilary Hurd Anyaso. Note: Materials may be edited for content and length.


Journal Reference:

  1. Julia A. Mossbridge, Marcia Grabowecky, Satoru Suzuki. Seeing the Song: Left Auditory Structures May Track Auditory-Visual Dynamic Alignment. PLoS ONE, 2013; 8 (10): e77201 DOI: 10.1371/journal.pone.0077201

Cite This Page:

Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. ScienceDaily, 23 October 2013. <www.sciencedaily.com/releases/2013/10/131023183248.htm>.
Northwestern University. (2013, October 23). Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony. ScienceDaily. Retrieved August 20, 2014 from www.sciencedaily.com/releases/2013/10/131023183248.htm
Northwestern University. "Seeing the song: Study aims to understand how, when the auditory system registers complex auditory-visual synchrony." ScienceDaily. www.sciencedaily.com/releases/2013/10/131023183248.htm (accessed August 20, 2014).

Share This




More Mind & Brain News

Wednesday, August 20, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Researcher Testing on-Field Concussion Scanners

Researcher Testing on-Field Concussion Scanners

AP (Aug. 19, 2014) Four Texas high school football programs are trying out an experimental system designed to diagnose concussions on the field. The technology is in response to growing concern over head trauma in America's most watched sport. (Aug. 19) Video provided by AP
Powered by NewsLook.com
Kids' Drawings At Age 4 Linked To Intelligence At Age 14

Kids' Drawings At Age 4 Linked To Intelligence At Age 14

Newsy (Aug. 19, 2014) A study by King's College London says there's a link between how well kids draw at age 4 and how intelligent they are later in life. Video provided by Newsy
Powered by NewsLook.com
Mental, Neurological Disabilities Up 21% Among Kids

Mental, Neurological Disabilities Up 21% Among Kids

Newsy (Aug. 18, 2014) New numbers show a decade's worth of changes in the number of kids with disabilities. They suggest mental disabilities are up; physical ones are down. Video provided by Newsy
Powered by NewsLook.com
Fake Weed Wreaks Havoc In New Hampshire

Fake Weed Wreaks Havoc In New Hampshire

Newsy (Aug. 17, 2014) New Hampshire's governor declared a state of emergency after more than 40 overdoses of synthetic marijuana in one week throughout the state. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins