Featured Research

from universities, journals, and other organizations

Photo editing tool enables object images to be manipulated in 3-D

Date:
August 5, 2014
Source:
Carnegie Mellon University
Summary:
Editors of photos routinely resize objects, or move them up, down or sideways, but researchers are adding an extra dimension to photo editing by enabling editors to turn or flip objects any way they want, even exposing surfaces not visible in the original photograph.

An origami bird held in a hand can be made to flap its wings and fly away.
Credit: Image courtesy of Carnegie Mellon University

Editors of photos routinely resize objects, or move them up, down or sideways, but Carnegie Mellon University researchers are adding an extra dimension to photo editing by enabling editors to turn or flip objects any way they want, even exposing surfaces not visible in the original photograph.

Related Articles


A chair in a photograph of a living room, for instance, can be turned around or even upside down in the photo, displaying sides of the chair that would have been hidden from the camera, yet appearing to be realistic.

This three-dimensional manipulation of objects in a single, two-dimensional photograph is possible because 3-D numerical models of many everyday objects -- furniture, cookware, automobiles, clothes, appliances -- are readily available online. The research team led by Yaser Sheikh, associate research professor of robotics, found they could create realistic edits by fitting these models into the geometry of the photo and then applying colors, textures and lighting consistent with the photo.

"In the real world, we're used to handling objects -- lifting them, turning them around or knocking them over," said Natasha Kholgade, a Ph.D. student in the Robotics Institute and lead author of the study. "We've created an environment that gives you that same freedom when editing a photo."

Kholgade will present the team's findings Aug. 13 at the SIGGRAPH 2014 Conference on Computer Graphics and Interactive Techniques in Vancouver, Canada. A video demonstrating the new system is available on the project website at http://www.cs.cmu.edu/~om3d/.

Though the system is designed for use with digital imagery, it enables the same 3-D manipulation of objects in paintings and in historical photos. Objects that can be manipulated in photos also can be animated; the researchers demonstrated that an origami bird held in a hand can be made to flap its wings and fly away, or a taxi cab shown in a street scene can levitate, flip over to reveal its undercarriage and zip off into the heavens.

"Instead of simply editing 'what we see' in the photograph, our goal is to manipulate 'what we know' about the scene behind the photograph," Kholgade said.

Other researchers have used depth-based segmentation to perform viewpoint changes in photos or have used modeling of photographed objects, but neither approach enables hidden areas to be revealed. Another alternative is to insert a new 2-D or 3-D object into a photo, but those approaches discard information from the original photo regarding lighting and appearance, so the results are less than seamless.

One of the catches to using publicly available 3-D models is that the models seldom, if ever, fit a photo exactly. Variations occur between the models and the physical objects; real-life objects such as seat cushions and backpacks are sometimes deformed as they are used, and appearances may change because of aging, weathering or lighting.

To fix these variations, the researchers developed a technique to semi-automatically align the model to the geometry of the object in the photo while preserving the symmetries in the object. The system then automatically estimates the environmental illumination and appearance of the hidden parts of the object -- the visible side of a seat cushion or of a banana is used to create a plausible appearance for the opposite side. If the photo doesn't contain pertinent appearance information -- such as the underside of a taxicab -- the system uses the appearance of the stock 3-D model.

Though a wide variety of stock models is available online, models are not available for every object in a photo. But that limitation is likely to subside, particularly as 3-D scanning and printing technologies become ubiquitous. "The more pressing question will soon be, not whether a particular model exists online, but whether the user can find it," Sheikh said. One thrust of future research will thus need to be automating the search for 3-D models in a database of millions.

This research was sponsored, in part, by a Google Research Award. In addition to Kholgade and Sheikh, the team included Tomas Simon, a Ph.D. student in the Robotics Institute, and Alexei Efros, a former CMU faculty member now an associate professor of electrical engineering and computer science at the University of California, Berkeley.


Story Source:

The above story is based on materials provided by Carnegie Mellon University. Note: Materials may be edited for content and length.


Cite This Page:

Carnegie Mellon University. "Photo editing tool enables object images to be manipulated in 3-D." ScienceDaily. ScienceDaily, 5 August 2014. <www.sciencedaily.com/releases/2014/08/140805132148.htm>.
Carnegie Mellon University. (2014, August 5). Photo editing tool enables object images to be manipulated in 3-D. ScienceDaily. Retrieved November 23, 2014 from www.sciencedaily.com/releases/2014/08/140805132148.htm
Carnegie Mellon University. "Photo editing tool enables object images to be manipulated in 3-D." ScienceDaily. www.sciencedaily.com/releases/2014/08/140805132148.htm (accessed November 23, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Sunday, November 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

European Parliament Might Call For Google's Break-Up

European Parliament Might Call For Google's Break-Up

Newsy (Nov. 22, 2014) This is the latest development in an antitrust investigation accusing Google of unfairly prioritizing own products and services in search results. Video provided by Newsy
Powered by NewsLook.com
Google Announces Improvements To Balloon-Borne Wi-Fi Project

Google Announces Improvements To Balloon-Borne Wi-Fi Project

Newsy (Nov. 21, 2014) In a blog post, Google said its balloons have traveled 3 million kilometers since the start of Project Loon. Video provided by Newsy
Powered by NewsLook.com
Is Nintendo Making A Comeback With 'Super Smash Bros.'?

Is Nintendo Making A Comeback With 'Super Smash Bros.'?

Newsy (Nov. 21, 2014) Nintendo released new "Super Smash Bros." Friday, and it's getting great reviews. Could this mean a comeback for the gaming company? Video provided by Newsy
Powered by NewsLook.com
NSA Director: China Can Damage US Power Grid

NSA Director: China Can Damage US Power Grid

AP (Nov. 20, 2014) China and "one or two" other countries are capable of mounting cyberattacks that would shut down the electric grid and other critical systems in parts of the United States, according to Adm. Michael Rogers, director of the National Security Agency and hea Video provided by AP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins