Featured Research

from universities, journals, and other organizations

Computers With 'Common Sense'

Date:
October 18, 2007
Source:
University of California - San Diego
Summary:
Using a little-known Google Labs widget, computer scientists have brought common sense to an automated image labeling system. This common sense is the ability to use context to help identify objects in photographs. For example, if a conventional automated object identifier labels a person, a tennis racket, a tennis court and a lemon in a photo, the new post-processing context check will re-label the lemon a tennis ball.

Looking at the photo above, you see a person on a tennis court, wielding a tennis racket and chasing a...lemon. Right? Wrong. You don’t think it’s a lemon. You know it's a tennis ball. Computers with the latest image labeling algorithms don't have the contextual wits to know a lemon is very unlikely in this scene. UCSD computer scientists are looking to change that.
Credit: UC San Diego

Using a little-known Google Labs widget, computer scientists from UC San Diego and UCLA have brought common sense to an automated image labeling system. This common sense is the ability to use context to help identify objects in photographs.

For example, if a conventional automated object identifier has labeled a person, a tennis racket, a tennis court and a lemon in a photo, the new post-processing context check will re-label the lemon as a tennis ball.

“We think our paper is the first to bring external semantic context to the problem of object recognition,” said computer science professor Serge Belongie from UC San Diego.

The researchers show that the Google Labs tool called Google Sets can be used to provide external contextual information to automated object identifiers.

Google Sets generates lists of related items or objects from just a few examples. If you type in John, Paul and George, it will return the words Ringo, Beatles and John Lennon. If you type “neon” and “argon” it will give you the rest of the noble gasses.

“In some ways, Google Sets is a proxy for common sense. In our paper, we showed that you can use this common sense to provide contextual information that improves the accuracy of automated image labeling systems,” said Belongie.

The image labeling system is a three step process. First, an automated system splits the image up into different regions through the process of image segmentation. In the photo above, image segmentation separates the person, the court, the racket and the yellow sphere.

Next, an automated system provides a ranked list of probable labels for each of these image regions.

Finally, the system adds a dose of context by processing all the different possible combinations of labels within the image and maximizing the contextual agreement among the labeled objects within each picture.

It is during this step that Google Sets can be used as a source of context that helps the system turn a lemon into a tennis ball. In this case, these “semantic context constraints” helped the system disambiguate between visually similar objects.

In another example, the researchers show that an object originally labeled as a cow is (correctly) re-labeled as a boat when the other objects in the image – sky, tree, building and water – are considered during the post-processing context step. In this case, the semantic context constraints helped to correct an entirely wrong image label. The context information came from the co-occurrence of object labels in the training sets rather than from Google Sets.

The computer scientists also highlight other advances they bring to automated object identification. First, instead of doing just one image segmentation, the researchers generated a collection of image segmentations and put together a shortlist of stable image segmentations. This increases the accuracy of the segmentation process and provides an implicit shape description for each of the image regions.

Second, the researchers ran their object categorization model on each of the segmentations, rather than on individual pixels. This dramatically reduced the computational demands on the object categorization model.

In the two sets of images that the researchers tested, the categorization results improved considerably with inclusion of context. For one image dataset, the average categorization accuracy increased more than 10 percent using the semantic context provided by Google Sets. In a second dataset, the average categorization accuracy improved by about 2 percent using the semantic context provided by Google Sets. The improvements were higher when the researchers gleaned context information from data on co-occurrence of object labels in the training data set for the object identifier.

Right now, the researchers are exploring ways to extend context beyond the presence of objects in the same image. For example, they want to make explicit use of absolute and relative geometric relationships between objects in an image – such as “above” or “inside” relationships. This would mean that if a person were sitting on top of an animal, the system would consider the animal to be more likely a horse than a dog.

Reference: “Objects in Context,” by Andrew Rabinovich, Carolina Galleguillos, Eric Wiewiora and Serge Belongie from the Department of Computer Science and Engineering at the UCSD Jacobs School of Engineering. Andrea Vedaldi from the Department of Computer Science, UCLA.

The paper will be presented on Thursday 18 October 2007 at ICCV 2007 – the 11th IEEE International Conference on Computer Vision in Rio de Janeiro, Brazil.

Funders: National Science Foundation, Afred P. Sloan Research Fellowship, Air Force Office of Scientific Research, Office of Naval Research.


Story Source:

The above story is based on materials provided by University of California - San Diego. Note: Materials may be edited for content and length.


Cite This Page:

University of California - San Diego. "Computers With 'Common Sense'." ScienceDaily. ScienceDaily, 18 October 2007. <www.sciencedaily.com/releases/2007/10/071017174328.htm>.
University of California - San Diego. (2007, October 18). Computers With 'Common Sense'. ScienceDaily. Retrieved October 2, 2014 from www.sciencedaily.com/releases/2007/10/071017174328.htm
University of California - San Diego. "Computers With 'Common Sense'." ScienceDaily. www.sciencedaily.com/releases/2007/10/071017174328.htm (accessed October 2, 2014).

Share This



More Computers & Math News

Thursday, October 2, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Mozilla Bets On Software To Sell Its Chromecast Competitor

Mozilla Bets On Software To Sell Its Chromecast Competitor

Newsy (Oct. 1, 2014) Mozilla's Matchstick streaming device is entering a crowded market. The company is banking on open-source software to rise above the competition. Video provided by Newsy
Powered by NewsLook.com
App Teaches Kindergarteners to Code

App Teaches Kindergarteners to Code

AP (Oct. 1, 2014) They can't all read yet, but soon kindergarteners may be able to create basic computer code. Researchers in Massachusetts developed an app that teaches young kids a simple computer programming language. (Oct. 1) Video provided by AP
Powered by NewsLook.com
Microsoft Goes For Familiarity Over Novelty In Windows 10

Microsoft Goes For Familiarity Over Novelty In Windows 10

Newsy (Sep. 30, 2014) At a special event in San Francisco, Microsoft introduced its latest operating system, Windows 10, which combines key features from earlier versions. Video provided by Newsy
Powered by NewsLook.com
French Apple Fans Discover the Apple Watch

French Apple Fans Discover the Apple Watch

AFP (Sep. 30, 2014) Apple fans in France discover the latest toy, the Apple Watch. The watch comes in two sizes and an array of interchangeable, fashionable wrist straps. Duration: 00:42 Video provided by AFP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins