Featured Research

from universities, journals, and other organizations

Hundreds of videos used to reconstruct 3-D motion without markers

Date:
July 17, 2014
Source:
Carnegie Mellon University
Summary:
Researchers have developed techniques for combining the views of 480 video cameras mounted in a two-story geodesic dome to perform large-scale 3D motion reconstruction, including volleyball games, the swirl of air currents and even a cascade of confetti.

Carnegie Mellon University researchers have developed techniques for combining the views of 480 video cameras mounted in a two-story geodesic dome to perform large-scale 3D motion reconstruction, including volleyball games, the swirl of air currents and even a cascade of confetti.

Though the research was performed in a specialized, heavily instrumented video laboratory, Yaser Sheikh, an assistant research professor of robotics who led the research team, said the techniques might eventually be applied to large-scale reconstructions of sporting events or performances captured by hundreds of cameras wielded by spectators.

In contrast to most previous work, which typically has involved just 10 to 20 video feeds, the Carnegie Mellon researchers didn't have to worry about filling in gaps in data; their camera system can track 100,000 points at a time. Rather, they have to figure out how to choose which of the hundreds of video trajectories can see each of those points and select only those camera views for the reconstruction.

"At some point, extra camera views just become 'noise,'" said Hanbyul Joo, a Ph.D. student in the Robotics Institute. "To fully leverage hundreds of cameras, we need to figure out which cameras can see each target point at any given time."

The research team developed a technique for estimating visibility that uses motion as a cue. In contrast to motion capture systems that use balls or other markers, the researchers used established techniques for automatically identifying and tracking points based on appearance features -- in this case, distinctive patterns. For each point, the system then seeks to determine which cameras see motion that is consistent with that point.

For instance, if a point on a person's chest is being tracked and most cameras show that point is moving to the right, a camera that picks up motion in the opposite direction is probably seeing a person or object that is in between the target and the camera. Or it may indicate the person has turned and the chest is no longer visible to the camera. In either case, the system knows that camera cannot see the target point and that its video feed is not useful for 3D reconstruction involving that point.

Other researchers have been able to use images from a large number of cameras, such as smartphones, to create 3D reconstructions of still images, Joo noted. But without methods such as the visibility estimation technique, 3D motion reconstruction at such a large scale has not been possible.

In their laboratory, called the Panoptic Studio, the researchers have 480 video cameras, plus an additional 30 high-definition video cameras, arrayed all around and halfway up the walls of a geodesic dome that can easily accommodate 10 people.

Such a dense array of cameras enables the researchers to perform 3D motion reconstructions not previously possible. These include 3D reconstructions of a person tossing confetti into the air, with each piece of paper tracked until it reaches the floor. In another case, confetti is fed into a fan, enabling a motion capture of the air flow. "You couldn't put markers on the paper without changing the flow," Joo explained.

Likewise, such techniques might be used for reconstruction of the motion of animals, which typically can't be instrumented. The CMU researchers have used the Panoptic Studio to capture the fine details of people interacting, whether it be college students casually conversing or a child being evaluated by a psychologist for signs of autism.

A video of the 3D reconstructions and links to the team's research paper are available on the project website, http://www.cs.cmu.edu/~hanbyulj/14/visibility.html.

The findings were presented at the Computer Vision and Pattern Recognition conference, June 24-27, in Columbus, Ohio. In addition to Sheikh and Joo, the research team included Hyun Soo Park, who this year completed his Ph.D. in mechanical engineering at CMU and is now a post-doctoral researcher at the University of Pennsylvania.


Story Source:

The above story is based on materials provided by Carnegie Mellon University. Note: Materials may be edited for content and length.


Cite This Page:

Carnegie Mellon University. "Hundreds of videos used to reconstruct 3-D motion without markers." ScienceDaily. ScienceDaily, 17 July 2014. <www.sciencedaily.com/releases/2014/07/140717124950.htm>.
Carnegie Mellon University. (2014, July 17). Hundreds of videos used to reconstruct 3-D motion without markers. ScienceDaily. Retrieved October 21, 2014 from www.sciencedaily.com/releases/2014/07/140717124950.htm
Carnegie Mellon University. "Hundreds of videos used to reconstruct 3-D motion without markers." ScienceDaily. www.sciencedaily.com/releases/2014/07/140717124950.htm (accessed October 21, 2014).

Share This



More Matter & Energy News

Tuesday, October 21, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Gulfstream G500, G600 Unveiling

Gulfstream G500, G600 Unveiling

Flying (Oct. 20, 2014) Watch Gulfstream's public launch of the G500 and G600 at their headquarters in Savannah, Ga., along with a surprise unveiling of the G500, which taxied up under its own power. Video provided by Flying
Powered by NewsLook.com
Japanese Scientists Unveil Floating 3D Projection

Japanese Scientists Unveil Floating 3D Projection

Reuters - Innovations Video Online (Oct. 20, 2014) Scientists in Tokyo have demonstrated what they say is the world's first 3D projection that floats in mid air. A laser that fires a pulse up to a thousand times a second superheats molecules in the air, creating a spark which can be guided to certain points in the air to shape what the human eye perceives as an image. Matthew Stock reports. Video provided by Reuters
Powered by NewsLook.com
Hey, Doc! Sewage, Beer and Food Scraps Can Power Chevrolet’s Bi-Fuel Impala

Hey, Doc! Sewage, Beer and Food Scraps Can Power Chevrolet’s Bi-Fuel Impala

3BL Media (Oct. 20, 2014) Hey, Doc! Sewage, Beer and Food Scraps Can Power Chevrolet’s Bi-fuel Impala Video provided by 3BL
Powered by NewsLook.com
What We Know About Microsoft's Rumored Smartwatch

What We Know About Microsoft's Rumored Smartwatch

Newsy (Oct. 20, 2014) Microsoft will reportedly release a smartwatch that works across different mobile platforms, has a two-day battery life and tracks heart rate. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins