Featured Research

from universities, journals, and other organizations

Researchers Teach Computers How To Name Images By 'Thinking'

Date:
November 1, 2006
Source:
Penn State
Summary:
Penn State researchers have "taught" computers how to interpret images using a vocabulary of up to 330 English words, so that a computer can describe a photograph of two polo players, for instance, as "sport," "people," "horse," "polo."

Penn State researchers have "taught" computers how to interpret images using a vocabulary of up to 330 English words, so that a computer can describe a photograph of two polo players, for instance, as "sport," "people," "horse," "polo."

Related Articles


The new system, which can automatically annotate entire online collections of photographs as they are uploaded, means significant time-savings for the millions of Internet users who now manually tag or identify their images. It also facilitates retrieval of images through the use of search terms, said James Wang, associate professor in the Penn State College of Information Sciences and Technology, and one of the technology's two inventors.

The system is described in a paper, "Real-Time Computerized Annotation of Pictures," given at the recent ACM Multimedia 2006 conference in Santa Barbara, Calif., and authored by Jia Li, associate professor, Department of Statistics, and Wang. Penn State has filed a provisional patent application on the invention. Major search engines currently rely upon uploaded tags of text to describe images. While many collections are annotated, many are not. The result: Images without text tags are not accessible to Web searchers. Because it provides text tags, the ALIPR system-Automatic Linguistic Indexing of Pictures-Real Time-makes those images visible to Web users.

ALIPR does this by analyzing the pixel content of images and comparing that against a stored knowledge base of the pixel content of tens of thousands of image examples. The computer then suggests a list of 15 possible annotations or words for the image.

"By inputting tens of thousands of images, we have trained computers to recognize certain objects and concepts and automatically annotate those new or unseen images," Wang said. "More than half the time, the computer's first tag out of the top 15 tags is correct."

In addition, for 98 percent of images tested, the system has provided at least one correct annotation in the top 15 selected words. The system, which completes the annotation in about 1.4 seconds, also can be applied to other domains such as art collections, satellite imaging and pathology slides, Wang said. The new system builds on the authors' previous invention, ALIP, which also analyzes image content. But unlike ALIP which characterized images by incorporating computational-intensive spatial modeling, ALIPR characterizes images by modeling distributions of color and texture.

The researchers acknowledge computers trained with their algorithms have difficulties when photos are fuzzy or have low contrast or resolution; when objects are shown only partially; and when the angle used by the photographer presents an image in a way that is different than how the computer was trained on the object. Adding more training images as well as improving the training process may reduce these limitations-future areas of research.

A demonstration of the ALIPR system can be found at http://www.alipr.com.

In a companion paper also presented at the ACM conference, the researchers describe another of their systems-one that can use annotations in a retrieval process. This new system leverages annotations from different sources, human and computer. The researchers, who have built a prototype of the system, are working on testing it in real-world situations. That paper, "Toward Bridging the Annotation-Retrieval Gap in Image Search by a Generative Modeling Approach," was authored by Ritendra Datta and Weina Ge, Ph.D. students in computer science and engineering; Li; and Wang.

"Our approach aims at making all pictures on the Internet visible to the users of search engines," Wang said. Research on both systems was supported by the National Science Foundation.


Story Source:

The above story is based on materials provided by Penn State. Note: Materials may be edited for content and length.


Cite This Page:

Penn State. "Researchers Teach Computers How To Name Images By 'Thinking'." ScienceDaily. ScienceDaily, 1 November 2006. <www.sciencedaily.com/releases/2006/11/061101122927.htm>.
Penn State. (2006, November 1). Researchers Teach Computers How To Name Images By 'Thinking'. ScienceDaily. Retrieved November 21, 2014 from www.sciencedaily.com/releases/2006/11/061101122927.htm
Penn State. "Researchers Teach Computers How To Name Images By 'Thinking'." ScienceDaily. www.sciencedaily.com/releases/2006/11/061101122927.htm (accessed November 21, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Friday, November 21, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Google Announces Improvements To Balloon-Borne Wi-Fi Project

Google Announces Improvements To Balloon-Borne Wi-Fi Project

Newsy (Nov. 21, 2014) In a blog post, Google said its balloons have traveled 3 million kilometers since the start of Project Loon. Video provided by Newsy
Powered by NewsLook.com
NSA Director: China Can Damage US Power Grid

NSA Director: China Can Damage US Power Grid

AP (Nov. 20, 2014) China and "one or two" other countries are capable of mounting cyberattacks that would shut down the electric grid and other critical systems in parts of the United States, according to Adm. Michael Rogers, director of the National Security Agency and hea Video provided by AP
Powered by NewsLook.com
You Now 'Get' No-Cost Downloads In Apple's App Store

You Now 'Get' No-Cost Downloads In Apple's App Store

Newsy (Nov. 20, 2014) Apple has changed its App Store wording from "Free" to "Get," as the European Commission and Federal Trade Commission seek to protect consumers. Video provided by Newsy
Powered by NewsLook.com
Google Blocks Its Own Ads With New Contributor Program

Google Blocks Its Own Ads With New Contributor Program

Newsy (Nov. 20, 2014) Google's unveiled a crowdfunding platform dubbed Contributor, which allows people to pay for ad-free sites. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins