Featured Research

from universities, journals, and other organizations

Student uses skin as input for mobile devices

Date:
April 7, 2010
Source:
Carnegie Mellon University
Summary:
A combination of simple bio-acoustic sensors and some sophisticated machine learning makes it possible for people to use their fingers or forearms -- potentially, any part of their bodies -- as touchpads to control smart phones or other mobile devices.

Chris Harrison demonstrates Skinput technology.
Credit: Image courtesy of Carnegie Mellon University

A combination of simple bio-acoustic sensors and some sophisticated machine learning makes it possible for people to use their fingers or forearms -- potentially, any part of their bodies -- as touchpads to control smart phones or other mobile devices.

The technology, called Skinput, was developed by Chris Harrison, a third-year Ph.D. student in Carnegie Mellon University's Human-Computer Interaction Institute (HCII), along with Desney Tan and Dan Morris of Microsoft Research. Harrison will describe the technology in a paper to be presented on April 12, at CHI 2010, the Association for Computing Machinery's annual Conference on Human Factors in Computing Systems in Atlanta, Ga.

Skinput, www.chrisharrison.net/projects/skinput/, could help people take better advantage of the tremendous computing power now available in compact devices that can be easily worn or carried. The diminutive size that makes smart phones, MP3 players and other devices so portable, however, also severely limits the size and utility of the keypads, touchscreens and jog wheels typically used to control them.

"With Skinput, we can use our own skin -- the body's largest organ -- as an input device," Harrison said "It's kind of crazy to think we could summon interfaces onto our bodies, but it turns out to make a lot of sense. Our skin is always with us, and makes the ultimate interactive touch surface."

In a prototype developed while Harrison was an intern at Microsoft Research last summer, acoustic sensors are attached to the upper arm. These sensors capture sound generated by such actions as flicking or tapping fingers together, or tapping the forearm. This sound is not transmitted through the air, but by transverse waves through the skin and by longitudinal, or compressive, waves through the bones.

Harrison and his colleagues found that the tap of each fingertip, a tap to one of five locations on the arm, or a tap to one of 10 locations on the forearm produces a unique acoustic signature that machine learning programs could learn to identify. These computer programs, which improve with experience, were able to determine the signature of each type of tap by analyzing 186 different features of the acoustic signals, including frequencies and amplitude.

In a trial involving 20 subjects, the system was able to classify the inputs with 88 percent accuracy overall. Accuracy depended in part on proximity of the sensors to the input; forearm taps could be identified with 96 percent accuracy when sensors were attached below the elbow, 88 percent accuracy when the sensors were above the elbow. Finger flicks could be identified with 97 percent accuracy.

"There's nothing super sophisticated about the sensor itself," Harrison said, "but it does require some unusual processing. It's sort of like the computer mouse -- the device mechanics themselves aren't revolutionary, but are used in a revolutionary way." The sensor is an array of highly tuned vibration sensors -- cantilevered piezo films.

The prototype armband includes both the sensor array and a small projector that can superimpose colored buttons onto the wearer's forearm, which can be used to navigate through menus of commands. Additionally, a keypad can be projected on the palm of the hand. Simple devices, such as MP3 players, might be controlled simply by tapping fingertips, without need of superimposed buttons; in fact, Skinput can take advantage of proprioception -- a person's sense of body configuration -- for eyes-free interaction.

Though the prototype is of substantial size and designed to fit the upper arm, the sensor array could easily be miniaturized so that it could be worn much like a wristwatch, Harrison said.

Testing indicates the accuracy of Skinput is reduced in heavier, fleshier people and that age and sex might also affect accuracy. Running or jogging also can generate noise and degrade the signals, the researchers report, but the amount of testing was limited and accuracy likely would improve as the machine learning programs receive more training under such conditions.

Harrison, who delights in "blurring the lines between technology and magic," is a prodigious inventor. Last year, he launched a company, Invynt LLC, to market a technology he calls "Lean and Zoom," which automatically magnifies the image on a computer monitor as the user leans toward the screen. He also has developed a technique to create a pseudo-3D experience for video conferencing using a single webcam at each conference site. Another project explored how touchscreens can be enhanced with tactile buttons that can change shape as virtual interfaces on the touchscreen change.

Skinput is an extension of an earlier invention by Harrison called Scratch Input, which used acoustic microphones to enable users to control cell phones and other devices by tapping or scratching on tables, walls or other surfaces.

"Chris is a rising star," said Scott Hudson, HCII professor and Harrison's faculty adviser. "Even though he's a comparatively new Ph.D. student, the very innovative nature of his work has garnered a lot of attention both in the HCI research community and beyond."


Story Source:

The above story is based on materials provided by Carnegie Mellon University. Note: Materials may be edited for content and length.


Cite This Page:

Carnegie Mellon University. "Student uses skin as input for mobile devices." ScienceDaily. ScienceDaily, 7 April 2010. <www.sciencedaily.com/releases/2010/04/100406125547.htm>.
Carnegie Mellon University. (2010, April 7). Student uses skin as input for mobile devices. ScienceDaily. Retrieved July 24, 2014 from www.sciencedaily.com/releases/2010/04/100406125547.htm
Carnegie Mellon University. "Student uses skin as input for mobile devices." ScienceDaily. www.sciencedaily.com/releases/2010/04/100406125547.htm (accessed July 24, 2014).

Share This




More Computers & Math News

Thursday, July 24, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Creative Makeovers for Ugly Cellphone Towers

Creative Makeovers for Ugly Cellphone Towers

AP (July 24, 2014) Mobile phone companies and communities across the country are going to new lengths to disguise those unsightly cellphone towers. From a church bell tower to a flagpole, even a pencil, some towers are trying to make a point. (July 24) Video provided by AP
Powered by NewsLook.com
Robot Parking Valet Creates Stress-Free Travel

Robot Parking Valet Creates Stress-Free Travel

AP (July 23, 2014) 'Ray' the robotic parking valet at Dusseldorf Airport in Germany lets travelers to avoid the hassle of finding a parking spot before heading to the check-in desk. (July 23) Video provided by AP
Powered by NewsLook.com
Facebook Earnings Put Smile on Investors Faces

Facebook Earnings Put Smile on Investors Faces

Reuters - Business Video Online (July 23, 2014) Facebook earnings beat forecasts- with revenue climbing 61 percent. Bobbi Rebell reports. Video provided by Reuters
Powered by NewsLook.com
StubHub Caught in Global Cyber Crime Ring

StubHub Caught in Global Cyber Crime Ring

Reuters - Business Video Online (July 23, 2014) eBay's StubHub is caught up in an international cyber crime ring stretching from North America to Europe. Conway G. Gittens reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

    Health News

      Environment News

        Technology News



          Save/Print:
          Share:

          Free Subscriptions


          Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

          Get Social & Mobile


          Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

          Have Feedback?


          Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
          Mobile: iPhone Android Web
          Follow: Facebook Twitter Google+
          Subscribe: RSS Feeds Email Newsletters
          Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins