Featured Research

from universities, journals, and other organizations

Generating 'Oohs' And 'Aahs': Vocal Joystick Uses Voice To Surf The Internet

Date:
October 10, 2007
Source:
University of Washington
Summary:
A new tool lets people with disabilities control a computer cursor without lifting a finger. Early tests suggest that an experienced user of Vocal Joystick would have as much control as someone using a handheld device.

Doctoral student Brandi House uses Vocal Joystick to control the movement of a robotic arm. The screen on the lower right shows how the software analyzes her vocalizations to create instructions for the arm's movement.
Credit: Image courtesy of University of Washington

The Internet offers wide appeal to people with disabilities. But many of those same people find it frustrating or impossible to use a handheld mouse. Software developed at the University of Washington provides an alternative using the oldest and most versatile mode of communication: the human voice.

"There are many people who have perfect use of their voice who don't have use of their hands and arms," said Jeffrey Bilmes, a UW associate professor of electrical engineering. "I think there are several reasons why Vocal Joystick might be a better approach, or at least a viable alternative, to brain-computer interfaces." The tool's latest developments will be presented this month in Tempe, Ariz. at the Assets Conference on Computers and Accessibility.

Vocal Joystick detects sounds 100 times a second and instantaneously turns that sound into movement on the screen. Different vowel sounds dictate the direction: "ah," "ee," "aw" and "oo" and other sounds move the cursor one of eight directions. Users can transition smoothly from one vowel to another, and louder sounds make the cursor move faster. The sounds "k" and "ch" simulate clicking and releasing the mouse buttons.

Versions of Vocal Joystick exist for browsing the Web, drawing on a screen, controlling a cursor and playing a video game. A version also exists for operating a robotic arm, and Bilmes believes the technology could be used to control an electronic wheelchair.

Existing substitutes for the handheld mouse include eye trackers, sip-and-puff devices andhead-tracking systems. Each technology has drawbacks. Eye-tracking devices are expensive and require that the eye simultaneously take in information and control the cursor, which can cause confusion. Sip-and-puff joysticks held in the mouth must be spit out if the user wants to speak, and can be tiring. Head-tracking devices require neck movement and expensive hardware.

Vocal Joystick requires only a microphone, a computer with a standard sound card and a user who can produce vocal sounds.

"A lot of people ask: 'Why don't you just use speech recognition?'" Bilmes said. "It would be very slow to move a cursor using discrete commands like 'move right' or 'go faster.' The voice, however, is able to do continuous commands quickly and easily." Early tests suggest that an experienced user of Vocal Joystick would have as much control as someone using a handheld device.

In the laboratory, doctoral student Jonathan Malkin, who helped develop the tool, uses Vocal Joystick to play a game called Fish Tale. It takes two minutes to train the program for Malkin's voice. He then moves the fish character easily around the screen, raising his voice slightly to speed up and avoid being eaten by a predator fish.

The newest development, which will be presented at the October meeting in Tempe, uses Vocal Joystick to control a robotic arm. The pitch of the tone moves the arm up and down; other commands are unchanged. This is the first time that vocal commands have been used to control a three-dimensional object, Bilmes said.

One initial concern, he said, was whether people would feel self-conscious using the tool.

"But once you try it you immediately forget what you're saying," Bilmes said. "I usually go to the New York Times' Web site to test the system and then I get distracted and start reading the news. I forget that I'm using it."

To test the device, the group has been working with about eight spinal-cord injury patients at the UW Medical Center since March.

"It's a really exciting idea. I think it has tremendous potential," said Kurt Johnson, a professor of rehabilitation medicine who is helping with the tests.

Bilmes said he hopes people will become more adept at using the system over time. Future research will incorporate more advanced controls that use more aspects of the human voice, such as repeated vocalizations, vibrato, degree of nasality and trills.

"While people use their voices to communicate with just words and phrases," Bilmes said, "the human voice is an incredibly flexible instrument, and can do so much more."


Story Source:

The above story is based on materials provided by University of Washington. Note: Materials may be edited for content and length.


Cite This Page:

University of Washington. "Generating 'Oohs' And 'Aahs': Vocal Joystick Uses Voice To Surf The Internet." ScienceDaily. ScienceDaily, 10 October 2007. <www.sciencedaily.com/releases/2007/10/071009131947.htm>.
University of Washington. (2007, October 10). Generating 'Oohs' And 'Aahs': Vocal Joystick Uses Voice To Surf The Internet. ScienceDaily. Retrieved July 30, 2014 from www.sciencedaily.com/releases/2007/10/071009131947.htm
University of Washington. "Generating 'Oohs' And 'Aahs': Vocal Joystick Uses Voice To Surf The Internet." ScienceDaily. www.sciencedaily.com/releases/2007/10/071009131947.htm (accessed July 30, 2014).

Share This




More Health & Medicine News

Wednesday, July 30, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Health Insurers' Profits Slide

Health Insurers' Profits Slide

Reuters - Business Video Online (July 30, 2014) Obamacare-related costs were said to be behind the profit plunge at Wellpoint and Humana, but Wellpoint sees the new exchanges boosting its earnings for the full year. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com
Concern Grows Over Worsening Ebola Crisis

Concern Grows Over Worsening Ebola Crisis

AFP (July 30, 2014) Pan-African airline ASKY has suspended all flights to and from the capitals of Liberia and Sierra Leone amid the worsening Ebola health crisis, which has so far caused 672 deaths in Guinea, Liberia and Sierra Leone. Duration: 00:43 Video provided by AFP
Powered by NewsLook.com
At Least 20 Chikungunya Cases in New Jersey

At Least 20 Chikungunya Cases in New Jersey

AP (July 30, 2014) At least 20 New Jersey residents have tested positive for chikungunya, a mosquito-borne virus that has spread through the Caribbean. (July 30) Video provided by AP
Powered by NewsLook.com
Generics Eat Into Pfizer's Sales

Generics Eat Into Pfizer's Sales

Reuters - Business Video Online (July 29, 2014) Pfizer, the world's largest drug maker, cut full-year revenue forecasts because generics could cut into sales of its anti-arthritis drug, Celebrex. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

    Environment News

    Technology News



      Save/Print:
      Share:

      Free Subscriptions


      Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

      Get Social & Mobile


      Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

      Have Feedback?


      Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
      Mobile: iPhone Android Web
      Follow: Facebook Twitter Google+
      Subscribe: RSS Feeds Email Newsletters
      Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins