Featured Research

from universities, journals, and other organizations

Robot can be programmed by casually talking to it

Date:
June 23, 2014
Source:
Cornell University
Summary:
A professor of computer science is teaching robots to understand instructions in natural language from various speakers, account for missing information, and adapt to the environment at hand.

A computer science professor is teaching robots to understand instructions in natural language from various speakers, account for missing information, and adapt to the environment at hand.
Credit: Image courtesy of Cornell University

Robots are getting smarter, but they still need step-by-step instructions for tasks they haven't performed before. Before you can tell your household robot "Make me a bowl of ramen noodles," you'll have to teach it how to do that. Since we're not all computer programmers, we'd prefer to give those instructions in English, just as we'd lay out a task for a child.

Related Articles


But human language can be ambiguous, and some instructors forget to mention important details. Suppose you told your household robot how to prepare ramen noodles, but forgot to mention heating the water or tell it where the stove is.

In his Robot Learning Lab, Ashutosh Saxena, assistant professor of computer science at Cornell University, is teaching robots to understand instructions in natural language from various speakers, account for missing information, and adapt to the environment at hand.

Saxena and graduate students Dipendra K. Misra and Jaeyong Sung will describe their methods at the Robotics: Science and Systems conference at the University of California, Berkeley, July 12-16.

The robot may have a built-in programming language with commands like find (pan); grasp (pan); carry (pan, water tap); fill (pan, water); carry (pan, stove) and so on. Saxena's software translates human sentences, such as "Fill a pan with water, put it on the stove, heat the water. When it's boiling, add the noodles." into robot language. Notice that you didn't say, "Turn on the stove." The robot has to be smart enough to fill in that missing step.

Saxena's robot, equipped with a 3-D camera, scans its environment and identifies the objects in it, using computer vision software previously developed in Saxena's lab. The robot has been trained to associate objects with their capabilities: A pan can be poured into or poured from; stoves can have other objects set on them, and can heat things. So the robot can identify the pan, locate the water faucet and stove and incorporate that information into its procedure. If you tell it to "heat water" it can use the stove or the microwave, depending on which is available. And it can carry out the same actions tomorrow if you've moved the pan, or even moved the robot to a different kitchen.

Other workers have attacked these problems by giving a robot a set of templates for common actions and chewing up sentences one word at a time. Saxena's research group uses techniques computer scientists call "machine learning" to train the robot's computer brain to associate entire commands with flexibly defined actions. The computer is fed animated video simulations of the action -- created by humans in a process similar to playing a video game -- accompanied by recorded voice commands from several different speakers.

The computer stores the combination of many similar commands as a flexible pattern that can match many variations, so when it hears "Take the pot to the stove," "Carry the pot to the stove," "Put the pot on the stove," "Go to the stove and heat the pot" and so on, it calculates the probability of a match with what it has heard before, and if the probability is high enough, it declares a match. A similarly fuzzy version of the video simulation supplies a plan for the action: Wherever the sink and the stove are, the path can be matched to the recorded action of carrying the pot of water from one to the other.

Of course the robot still doesn't get it right all the time. To test, the researchers gave instructions for preparing ramen noodles and for making affogato -- an Italian dessert combining coffee and ice cream: "Take some coffee in a cup. Add ice cream of your choice. Finally, add raspberry syrup to the mixture."

The robot performed correctly up to 64 percent of the time even when the commands were varied or the environment was changed, and it was able to fill in missing steps. That was three to four times better than previous methods, the researchers reported, but "There is still room for improvement."

You can teach a simulated robot to perform a kitchen task at the "Tell me Dave" website, and your input there will become part of a crowdsourced library of instructions for the Cornell robots. Aditya Jami, visiting researcher at Cornell, is helping Tell Me Dave to scale the library to millions of examples. "With crowdsourcing at such a scale, robots will learn at a much faster rate," Saxena said.

Video: http://www.youtube.com/watch?v=73ofcHex92E

Further information: http://tellmedave.cs.cornell.edu/


Story Source:

The above story is based on materials provided by Cornell University. Note: Materials may be edited for content and length.


Cite This Page:

Cornell University. "Robot can be programmed by casually talking to it." ScienceDaily. ScienceDaily, 23 June 2014. <www.sciencedaily.com/releases/2014/06/140623091836.htm>.
Cornell University. (2014, June 23). Robot can be programmed by casually talking to it. ScienceDaily. Retrieved November 26, 2014 from www.sciencedaily.com/releases/2014/06/140623091836.htm
Cornell University. "Robot can be programmed by casually talking to it." ScienceDaily. www.sciencedaily.com/releases/2014/06/140623091836.htm (accessed November 26, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Wednesday, November 26, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

FCC Forces T-Mobile To Alert Customers Of Data Throttling

FCC Forces T-Mobile To Alert Customers Of Data Throttling

Newsy (Nov. 25, 2014) T-Mobile and the FCC have reached an agreement requiring the company to alert customers when it throttles their data speeds. Video provided by Newsy
Powered by NewsLook.com
Symantec Uncovers Sophisticated Spying Malware Regin

Symantec Uncovers Sophisticated Spying Malware Regin

Newsy (Nov. 24, 2014) A Symantec white paper reveals details about Regin, a spying malware of unusual complexity which is believed to be state-sponsored. Video provided by Newsy
Powered by NewsLook.com
How to Keep Your Android Device Safe This Holiday Season

How to Keep Your Android Device Safe This Holiday Season

Howdini (Nov. 24, 2014) Protect yourself against malware and hackers, especially during the hectic online shopping season. Mobile device security makes a great holiday gift and protects your loved ones from cyber attacks and identity theft. Video provided by Howdini
Powered by NewsLook.com
How to Keep You and Your Family's Identitiy Safe Online This Holiday Season

How to Keep You and Your Family's Identitiy Safe Online This Holiday Season

Howdini (Nov. 24, 2014) The hectic holiday season is a prime time for online identity theft, so make sure you’re protected.Be cautious when shopping online Internet security software makes a great holiday gift and protects your loved ones from cyber attacks and identity theft. Video provided by Howdini
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins