Featured Research

from universities, journals, and other organizations

Woman with quadriplegia feeds herself chocolate using mind-controlled robot arm

Date:
December 17, 2012
Source:
University of Pittsburgh Medical Center
Summary:
Reaching out to "high five" someone, grasping and moving objects of different shapes and sizes, feeding herself dark chocolate. For Jan Scheuermann and a team of researchers, accomplishing these seemingly ordinary tasks demonstrated for the first time that a person with longstanding quadriplegia can maneuver a mind-controlled, human-like robot arm in seven dimensions (7D) to consistently perform many of the natural and complex motions of everyday life.

Jan Scheuermann, who has quadriplegia, brings a chocolate bar to her mouth using a robot arm she is guiding with her thoughts. Researcher Elke Brown, M.D., watches in the background. Click the photo to download it in high resolution.
Credit: UPMC

Reaching out to "high five" someone, grasping and moving objects of different shapes and sizes, feeding herself dark chocolate. For Jan Scheuermann and a team of researchers from the University of Pittsburgh School of Medicine and UPMC, accomplishing these seemingly ordinary tasks demonstrated for the first time that a person with longstanding quadriplegia can maneuver a mind-controlled, human-like robot arm in seven dimensions (7D) to consistently perform many of the natural and complex motions of everyday life.

Related Articles


In a study published in the online version of The Lancet, the researchers described the brain-computer interface (BCI) technology and training programs that allowed Ms. Scheuermann, 53, of Whitehall Borough in Pittsburgh, Pa. to intentionally move an arm, turn and bend a wrist, and close a hand for the first time in nine years.

Less than a year after she told the research team, "I'm going to feed myself chocolate before this is over," Ms. Scheuermann savored its taste and announced as they applauded her feat, "One small nibble for a woman, one giant bite for BCI."

"This is a spectacular leap toward greater function and independence for people who are unable to move their own arms," agreed senior investigator Andrew B. Schwartz, Ph.D., professor, Department of Neurobiology, Pitt School of Medicine. "This technology, which interprets brain signals to guide a robot arm, has enormous potential that we are continuing to explore. Our study has shown us that it is technically feasible to restore ability; the participants have told us that BCI gives them hope for the future."

In 1996, Ms. Scheuermann was a 36-year-old mother of two young children, running a successful business planning parties with murder-mystery themes and living in California when one day she noticed her legs seemed to drag behind her. Within two years, her legs and arms progressively weakened to the point that she required a wheelchair, as well as an attendant to assist her with dressing, eating, bathing and other day-to-day activities. After returning home to Pittsburgh in 1998 for support from her extended family, she was diagnosed with spinocerebellar degeneration, in which the connections between the brain and muscles slowly, and inexplicably, deteriorate.

"Now I can't move my arms and legs at all. I can't even shrug my shoulders," she said. "But I have come to the conclusion that worrying about something is experiencing it twice. I try to dwell on the good things that I have."

A friend pointed out an October 2011 video about another Pitt/UPMC BCI research study in which Tim Hemmes, a Butler, Pa., man who sustained a spinal cord injury that left him with quadriplegia, moved objects on a computer screen and ultimately reached out with a robot arm to touch his girlfriend.

"Wow, it's so neat that he can do that," Ms. Scheuermann thought as she watched him. "I wish I could do something like that." She had her attendant call the trial coordinator immediately, and said, "I'm a quadriplegic. Hook me up, sign me up! I want to do that!"

On Feb. 10, 2012, after screening tests to confirm that she was eligible for the study, co-investigator and UPMC neurosurgeon Elizabeth Tyler-Kabara, M.D., Ph.D., assistant professor, Department of Neurological Surgery, Pitt School of Medicine, placed two quarter-inch square electrode grids with 96 tiny contact points each in the regions of Ms. Scheuermann's brain that would normally control right arm and hand movement.

"Prior to surgery, we conducted functional imaging tests of the brain to determine exactly where to put the two grids," she said. "Then we used imaging technology in the operating room to guide placement of the grids, which have points that penetrate the brain's surface by about one-sixteenth of an inch."

The electrode points pick up signals from individual neurons and computer algorithms are used to identify the firing patterns associated with particular observed or imagined movements, such as raising or lowering the arm, or turning the wrist, explained lead investigator Jennifer Collinger, Ph.D., assistant professor, Department of Physical Medicine and Rehabilitation (PM&R), and research scientist for the VA Pittsburgh Healthcare System. That intent to move is then translated into actual movement of the robot arm, which was developed by Johns Hopkins University's Applied Physics Lab.

Two days after the operation, the team hooked up the two terminals that protrude from Ms. Scheuermann's skull to the computer. "We could actually see the neurons fire on the computer screen when she thought about closing her hand," Dr. Collinger said. "When she stopped, they stopped firing. So we thought, 'This is really going to work.'"

Within a week, Ms. Scheuermann could reach in and out, left and right, and up and down with the arm, which she named Hector, giving her 3-dimensional control that had her high-fiving with the researchers. "What we did in the first week they thought we'd be stuck on for a month," she noted.

Before three months had passed, she also could flex the wrist back and forth, move it from side to side and rotate it clockwise and counter-clockwise, as well as grip objects, adding up to what scientists call 7D control. In a study task called the Action Research Arm Test, Ms. Scheuermann guided the arm from a position four inches above a table to pick up blocks and tubes of different sizes, a ball and a stone and put them down on a nearby tray. She also picked up cones from one base to restack them on another a foot away, another task requiring grasping, transporting and positioning of objects with precision.

"Our findings indicate that by a variety of measures, she was able to improve her performance consistently over many days," Dr. Schwartz explained. "The training methods and algorithms that we used in monkey models of this technology also worked for Jan, suggesting that it's possible for people with long-term paralysis to recover natural, intuitive command signals to orient a prosthetic hand and arm to allow meaningful interaction with the environment."

In a separate study, researchers also continue to study BCI technology that uses an electrocortigraphy (ECoG) grid, which sits on the surface of the brain rather than slightly penetrates the tissue as in the case of the grids used for Ms. Scheuermann.

In both studies, "we're recording electrical activity in the brain, and the goal is to try to decode what that activity means and then use that code to control an arm," said senior investigator Michael Boninger, M.D., professor and chair, PM&R, and director of UPMC Rehabilitation Institute. "We are learning so much about how the brain controls motor activity, thanks to the hard work and dedication of our trial participants. Perhaps in five to 10 years, we will have a device that can be used in the day-to-day lives of people who are not able to use their own arms."

The next step for BCI technology will likely use a two-way electrode system that can not only capture the intention to move, but in addition, will stimulate the brain to generate sensation, potentially allowing a user to adjust grip strength to firmly grasp a doorknob or gently cradle an egg.

After that, "we're hoping this can become a fully implanted, wireless system that people can actually use in their homes without our supervision," Dr. Collinger said. "It might even be possible to combine brain control with a device that directly stimulates muscles to restore movement of the individual's own limb."

For now, Ms. Scheuermann is expected to continue to put the BCI technology through its paces for two more months, and then the implants will be removed in another operation.

"This is the ride of my life," she said. "This is the rollercoaster. This is skydiving. It's just fabulous, and I'm enjoying every second of it."

In addition to Drs. Collinger, Tyler-Kabara, Boninger and Schwartz, study co-authors include Brian Wodlinger, Ph.D., John E. Downey, Wei Wang, Ph.D., and Doug Weber, Ph.D., all of PM&R; and Angus J. McMorland, Ph.D., and Meel Velliste, Ph.D., of the Department of Neurobiology, Pitt School of Medicine.

The BCI projects are funded by the Defense Advanced Research Projects Agency, National Institutes of Health grant 8KL2TR000146-07, the U.S. Department of Veteran's Affairs, the UPMC Rehabilitation Institute and the University of Pittsburgh Clinical and Translational Science Institute.


Story Source:

The above story is based on materials provided by University of Pittsburgh Medical Center. Note: Materials may be edited for content and length.


Journal Reference:

  1. Jennifer L Collinger, Brian Wodlinger, John E Downey, Wei Wang, Elizabeth C Tyler-Kabara, Douglas J Weber, Angus JC McMorland, Meel Velliste, Michael L Boninger, Andrew B Schwartz. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet, 2012; DOI: 10.1016/S0140-6736(12)61816-9

Cite This Page:

University of Pittsburgh Medical Center. "Woman with quadriplegia feeds herself chocolate using mind-controlled robot arm." ScienceDaily. ScienceDaily, 17 December 2012. <www.sciencedaily.com/releases/2012/12/121217030957.htm>.
University of Pittsburgh Medical Center. (2012, December 17). Woman with quadriplegia feeds herself chocolate using mind-controlled robot arm. ScienceDaily. Retrieved December 21, 2014 from www.sciencedaily.com/releases/2012/12/121217030957.htm
University of Pittsburgh Medical Center. "Woman with quadriplegia feeds herself chocolate using mind-controlled robot arm." ScienceDaily. www.sciencedaily.com/releases/2012/12/121217030957.htm (accessed December 21, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Sunday, December 21, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Touch-Free Smart Phone Empowers Mobility-Impaired

Touch-Free Smart Phone Empowers Mobility-Impaired

Reuters - Innovations Video Online (Dec. 21, 2014) A touch-free phone developed in Israel enables the mobility-impaired to operate smart phones with just a movement of the head. Suzannah Butcher reports. Video provided by Reuters
Powered by NewsLook.com
Building Google Into Cars

Building Google Into Cars

Reuters - Business Video Online (Dec. 19, 2014) Google's next Android version could become the standard that'll power your vehicle's entertainment and navigation features, Reuters has learned. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com
After Sony Hack, What's Next?

After Sony Hack, What's Next?

Reuters - US Online Video (Dec. 19, 2014) The hacking attack on Sony Pictures has U.S. government officials weighing their response to the cyber-attack. Linda So reports. Video provided by Reuters
Powered by NewsLook.com
Navy Unveils Robot Fish

Navy Unveils Robot Fish

Reuters - Light News Video Online (Dec. 18, 2014) The U.S. Navy unveils an underwater device that mimics the movement of a fish. Tara Cleary reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins