New! Sign up for our free email newsletter.
Science News
from research organizations

Hearing through your fingers: Device that converts speech

Date:
June 3, 2019
Source:
IOS Press
Summary:
A new proof-of-concept study provides the first evidence that a speech-to-touch sensory substitution device can improve hearing in the hearing-impaired without any training, scientists report.
Share:
FULL STORY

A novel study published in Restorative Neurology and Neuroscience provides the first evidence that a simple and inexpensive non-invasive speech-to-touch sensory substitution device has the potential to improve hearing in hearing-impaired cochlear implant patients, as well as individuals with normal hearing, to better discern speech in various situations like learning a second language or trying to deal with the "cocktail party effect." The device can provide immediate multisensory enhancement without any training.

"Despite recent advancements in hearing aid and cochlear implants, the most widespread surgical implant to restore audition, hearing-impaired users still encounter significant practical and social challenges with or without aids," explained lead investigator Amir Amedi, PhD, Department of Medical Neurobiology, Institute for Medical Research Israel-Canada, Faculty of Medicine, Hebrew University of Jerusalem, Hadassah Ein-Kerem, and The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel. "In particular, they all struggle with understanding speech in challenging, noisy acoustic environments, especially in presence of a competing speaker."

The number of sensory deprived patients and auditory deprived patients is expected to rise so it is crucial to develop efficient techniques for auditory recovery designed to convey the missing information to patients. "We live in a world that is becoming steadily more multisensory and we really need to understand the mechanisms underlying multisensory perception and integration. Providing relevant information using the sense of touch can significantly help hearing," commented Dr. Amedi.

In this current proof-of-concept study investigators hypothesized that they would be able to improve speech understanding under challenging conditions by exploiting the ability of the brain to integrate information coming simultaneously from different senses. They designed a minimalistic auditory-to-tactile sensory substitution device (SSD) that transforms low-frequency speech signals into tactile vibrations delivered on two fingertips. They asked a group of non-native English speakers to repeat a series of sentences, which was degraded by embedding them in speech-like noise.

As expected, when participants could rely only on audition, their understanding of such sentences was poor. Crucially, however, their sentence understanding significantly improved when they paired the degraded speech signal with complementary vibration delivered on the participants fingertips. The vibration conveyed a specific set of frequencies known as fundamental frequencies that characterize speech signals.

The reported improvement at the group level was 6 dB -- a major difference considering that an increase of 10 dB represents a doubling of the perceived loudness. These results are especially relevant when compared to earlier SSD studies showing effects in behavior only after demanding cognitive training.

"Our results carry important implications for further research, as well as possible clinical and practical solutions," noted co-author Tomasz Wolak, PhD Eng, Head of the Bioimaging Research Center, Institute of Physiology and Pathology of Hearing, World Hearing Center, Warsaw, Poland. "The ability to 'hear through one's fingers can significantly help hearing. Our approach suggests that multisensory stimulations providing the same type of information (in this case spoken language conveyed through touch in addition to hearing) should be processed in the same brain region (in this case spoken language centers), ultimately then predicting that multisensory stimulations (both sounds and touch) should enhance perception.

According to lead author Katarzyna Cie?la, PhD, World Hearing Center, Warsaw, and Hebrew University of Jerusalem, "The most compelling aspect of our study is the fact that learning to use such speech-to-touch sensory substitution device did not require any training. We believe it can also serve as an aid for the elderly population, which finds it challenging to follow extensive training regimes. This might be the first study showing such immediate relevant enhancement of a sensory substitution device and suggests the brain is much more multisensory than the common wisdom."

"This study is a major step forward to introduce multisensory plasticity of the brain as an innovative paradigm to maximize the potential of patients to compensate for their sensory loss," commented Bernhard Sabel, PhD, Editor-in-Chief of Restorative Neurology and Neuroscience.

Next the team plans to further improve the device and training regimes in order to reach the goal of 10 dB enhancement and to test for human brain mechanisms using an MRI-compatible version of the device in various populations (both hearing and hearing-impaired people).


Story Source:

Materials provided by IOS Press. Note: Content may be edited for style and length.


Journal Reference:

  1. Katarzyna Cieśla, Tomasz Wolak, Artur Lorens, Benedetta Heimler, Henryk Skarżyński, Amir Amedi. Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution. Restorative Neurology and Neuroscience, 2019; 37 (2): 155 DOI: 10.3233/RNN-190898

Cite This Page:

IOS Press. "Hearing through your fingers: Device that converts speech." ScienceDaily. ScienceDaily, 3 June 2019. <www.sciencedaily.com/releases/2019/06/190603124549.htm>.
IOS Press. (2019, June 3). Hearing through your fingers: Device that converts speech. ScienceDaily. Retrieved March 28, 2024 from www.sciencedaily.com/releases/2019/06/190603124549.htm
IOS Press. "Hearing through your fingers: Device that converts speech." ScienceDaily. www.sciencedaily.com/releases/2019/06/190603124549.htm (accessed March 28, 2024).

Explore More

from ScienceDaily

RELATED STORIES