Featured Research

from universities, journals, and other organizations

New method helps computer vision systems decipher outdoor scenes

Date:
September 10, 2010
Source:
Carnegie Mellon University
Summary:
Computer vision systems can struggle to make sense of a single image, but a new method enables computers to gain a deeper understanding of an image by reasoning about the physical constraints of the scene.

Computer vision systems can struggle to make sense of a single image, but a new method devised by computer scientists at Carnegie Mellon University enables computers to gain a deeper understanding of an image by reasoning about the physical constraints of the scene. Here, the computer uses virtual blocks to build a three-dimensional approximation of the image at left that makes sense based on volume and mass.
Credit: Carnegie Mellon University

Computer vision systems can struggle to make sense of a single image, but a new method devised by computer scientists at Carnegie Mellon University enables computers to gain a deeper understanding of an image by reasoning about the physical constraints of the scene.

Related Articles


In much the same way that a child might use a set of toy building blocks to assemble something that looks like a building depicted on the cover of the toy set, the computer would analyze an outdoor scene by using virtual blocks to build a three-dimensional approximation of the image that makes sense based on volume and mass.

"When people look at a photo, they understand that the scene is geometrically constrained," said Abhinav Gupta, a post-doctoral fellow in CMU's Robotics Institute. "We know that buildings aren't infinitely thin, that most towers do not lean, and that heavy objects require support. It might not be possible to know the three-dimensional size and shape of all the objects in the photo, but we can narrow the possibilities. In the same way, if a computer can replicate an image, block by block, it can better understand the scene."

This novel approach to automated scene analysis could eventually be used to understand not only the objects in a scene, but the spaces in between them and what might lie behind areas obscured by objects in the foreground, said Alexei A. Efros, associate professor of robotics and computer science at CMU. That level of detail would be important, for instance, if a robot needed to plan a route where it might walk, he noted.

Gupta presented the research, which he conducted with Efros and Robotics Professor Martial Hebert, at the European Conference on Computer Vision, Sept. 5-11 in Crete, Greece.

Understanding outdoor scenes remains one of the great challenges of artificial intelligence. One approach has been to identify features of a scene, such as buildings, roads and cars, but this provides no understanding of the geometry of the scene, such as the location of walkable surfaces. Another approach, which Hebert and Efros pioneered with former student Derek Hoiem, now of the University of Illinois, Urbana-Champaign, has been to map the planar surfaces of an image to create a rough 3-D depiction of an image, similar to a pop-up book. But that approach can lead to depictions that are highly unlikely and sometimes physically impossible.

In the new method devised by Gupta, Efros and Hebert, the image is first broken into various segments corresponding to objects in the image. Once the ground and sky are identified, other segments are assigned potential geometric shapes. The shapes also are categorized as light or heavy, depending on appearance; a surface that appears to be a brick wall, for instance, would be classified as heavy.

The computer then attempts to reconstruct the image using the virtual blocks. If a heavy block appears unsupported, the computer must substitute an appropriately shaped block, or make assumptions that the original block was obscured in the original image.

Gupta said because this qualitative volumetric approach to scene understanding is so new, no established datasets or evaluation methodologies exist for it. He said in estimating the layout of surfaces, other than sky and ground, the method is better than 70 percent accurate, and its performance is almost as good when comparing its segmentation to ground truth. Overall, Gupta assesses the analysis as very good for 30 to 40 percent of the images and adequate for another 20 to 30 percent.


Story Source:

The above story is based on materials provided by Carnegie Mellon University. Note: Materials may be edited for content and length.


Cite This Page:

Carnegie Mellon University. "New method helps computer vision systems decipher outdoor scenes." ScienceDaily. ScienceDaily, 10 September 2010. <www.sciencedaily.com/releases/2010/09/100909114108.htm>.
Carnegie Mellon University. (2010, September 10). New method helps computer vision systems decipher outdoor scenes. ScienceDaily. Retrieved January 27, 2015 from www.sciencedaily.com/releases/2010/09/100909114108.htm
Carnegie Mellon University. "New method helps computer vision systems decipher outdoor scenes." ScienceDaily. www.sciencedaily.com/releases/2010/09/100909114108.htm (accessed January 27, 2015).

Share This


More From ScienceDaily



More Computers & Math News

Tuesday, January 27, 2015

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

PlayStation Now Smart TV App

PlayStation Now Smart TV App

Rumble (Jan. 27, 2015) PlayStation Now Smart TV app is coming soon and will be available on both Sony and Samsung HDTV, allowing you to play games without even a counsel! Check out the video for more info. Credit to &apos;booredatwork&apos;. Video provided by Rumble
Powered by NewsLook.com
WikiLeaks Accuses Google of Handing Over Emails to US

WikiLeaks Accuses Google of Handing Over Emails to US

AFP (Jan. 27, 2015) Whistleblowing site WikiLeaks accused Google of handing over the emails and electronic data of its senior staff to the US authorities without providing notification until almost three years later. Duration: 01:09 Video provided by AFP
Powered by NewsLook.com
Shark Bite Victim Making Amazing Recovery

Shark Bite Victim Making Amazing Recovery

AP (Jan. 27, 2015) A Texas woman who lost more than five pounds of flesh to a shark in the Bahamas earlier this month could be released from a Florida hospital soon. Experts believe she was bitten by a bull shark while snorkeling. (Jan. 27) Video provided by AP
Powered by NewsLook.com
Cablevision Enters Wi-Fi Phone Fray

Cablevision Enters Wi-Fi Phone Fray

Reuters - Business Video Online (Jan. 26, 2015) The entry by Cablevision and Google could intensify the already heated price wars for mobile phone service. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins