Featured Research

from universities, journals, and other organizations

Computerized method for matching images in photos, paintings, sketches created

Date:
December 10, 2011
Source:
Carnegie Mellon University
Summary:
Computers can mimic the human ability to find visually similar images, such as photographs of a fountain in summer and in winter, or a photograph and a painting of the same cathedral, by using a technique that analyzes the uniqueness of images, say researchers. The research team found that their surprisingly simple technique performed well on a number of visual tasks that normally stump computers, including matching sketches of automobiles with photographs of cars.

Computers can mimic the human ability to find visually similar images, such as photographs of a fountain in summer and in winter, or a photograph and a painting of the same cathedral, by using a technique that analyzes the uniqueness of images, say researchers at Carnegie Mellon University's School of Computer Science.

Related Articles


The research team, led by Alexei Efros, associate professor of computer science and robotics, and Abhinav Gupta, assistant research professor of robotics, found that their surprisingly simple technique performed well on a number of visual tasks that normally stump computers, including matching sketches of automobiles with photographs of cars.

The team from the Robotics Institute and Computer Science Department will present its findings on "data-driven uniqueness" on Dec. 14 at SIGGRAPH Asia, a computer graphics and interactive techniques conference in Hong Kong.

Their research paper is available online at http://graphics.cs.cmu.edu/projects/crossDomainMatching/.

Most computerized methods for matching images -- in contrast to image searches based on keywords -- focus on similarities in shapes, colors and composition. That approach has proven effective for finding exact or very close image matches and enabled successful applications such as Google Goggles.

But those methods can fail miserably when applied across different domains -- photographs taken in different seasons or under different lighting conditions, or in different media, such as photographs, color paintings or black-and-white sketches.

"The language of a painting is different than the language of a photograph," Efros explained. "Most computer methods latch onto the language, not on what's being said."

One problem, Gupta said, is that many images have strong elements, such as a cloud-filled sky, that may have superficial similarities to other images, but really only distract from what makes the image interesting to people. He and his collaborators hypothesized that it is instead the unique aspects of an image, in relation to other images being analyzed, that sets it apart and it is those elements that should be used to match it with similar images.

On the pixel level, a photo of a garden statue in the summer or fall will look very different than the same statue photographed in winter, said Abhinav Shrivastava, a master's degree student in robotics and first author of the research paper. But the unique aspects of the statue will carry over from a summer image to a winter image, or from a color photo to a sketch.

Estimating uniqueness is no simple task. The team computes uniqueness based on a very large data set of randomly selected images. Features that are unique are those that best discriminate one image from the rest of the random images. In a photo of a person in front of the Arc de Triomphe in Paris, for instance, the person likely is similar to people in other photos and thus would be given little weight in calculating uniqueness. The Arc itself, however, would be given greater weight because few photos include anything like it.

"We didn't expect this approach to work as well as it did," Efros acknowledged. "We don't know if this is anything like how humans compare images, but it's the best approximation we've been able to achieve."

In addition to automated image searches, this technique has applications to computational rephotography -- the combination of historic photographs with modern-day photos taken from the same perspective. By using the new technique, it may be possible in many cases to eliminate the need for rephotography by simply matching the historic photo with an existing online photo that matches its perspective. Likewise, the technique can be combined with large GPS-tagged photo collections to determine the location where a particular painting of a landmark was painted.

The technique also can be used to assemble a "visual memex" -- a data set that explores the visual similarities and contexts of a set of photos. For instance, the researchers downloaded 200 images of the Medici Fountain in Paris -- paintings, historic photographs and recent snapshots from various seasons and taken from various distances and angles -- and assembled them into a graph, as well as a YouTube video that shows a particular path through the data.

Future work includes using the technique to enhance object detection for computer vision and investigating ways to speed up the computationally intensive matching process.

Tomasz Malisiewicz, a former Ph.D. student who is now a post-doctoral fellow at MIT, also was a member of the research team. This work was supported by the Computer Science Department's Center for Computational Thinking, the Office of Naval Research and Google.

Youtube video.


Story Source:

The above story is based on materials provided by Carnegie Mellon University. Note: Materials may be edited for content and length.


Cite This Page:

Carnegie Mellon University. "Computerized method for matching images in photos, paintings, sketches created." ScienceDaily. ScienceDaily, 10 December 2011. <www.sciencedaily.com/releases/2011/12/111206115246.htm>.
Carnegie Mellon University. (2011, December 10). Computerized method for matching images in photos, paintings, sketches created. ScienceDaily. Retrieved October 24, 2014 from www.sciencedaily.com/releases/2011/12/111206115246.htm
Carnegie Mellon University. "Computerized method for matching images in photos, paintings, sketches created." ScienceDaily. www.sciencedaily.com/releases/2011/12/111206115246.htm (accessed October 24, 2014).

Share This



More Computers & Math News

Friday, October 24, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

The Best Apps to Organize Your Life

The Best Apps to Organize Your Life

Buzz60 (Oct. 23, 2014) — Need help organizing your bills, schedules and other things? Ko Im (@konakafe) has the best apps to help you stay on top of it all! Video provided by Buzz60
Powered by NewsLook.com
Nike And Apple Team Up To Create Wearable ... Something

Nike And Apple Team Up To Create Wearable ... Something

Newsy (Oct. 23, 2014) — For those looking for wearable tech that's significantly less nerdy than Google Glass, Nike CEO Mark Parker says don't worry, It's on the way. Video provided by Newsy
Powered by NewsLook.com
Chameleon Camouflage to Give Tanks Cloaking Capabilities

Chameleon Camouflage to Give Tanks Cloaking Capabilities

Reuters - Innovations Video Online (Oct. 22, 2014) — Inspired by the way a chameleon changes its colour to disguise itself; scientists in Poland want to replace traditional camouflage paint with thousands of electrochromic plates that will continuously change colour to blend with its surroundings. The first PL-01 concept tank prototype will be tested within a few years, with scientists predicting that a similar technology could even be woven into the fabric of a soldiers' clothing making them virtually invisible to the naked eye. Matthew Stock reports. Video provided by Reuters
Powered by NewsLook.com
Internet of Things Aims to Smarten Your Life

Internet of Things Aims to Smarten Your Life

AP (Oct. 22, 2014) — As more and more Bluetooth-enabled devices are reaching consumers, developers are busy connecting them together as part of the Internet of Things. (Oct. 22) Video provided by AP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins