Featured Research

from universities, journals, and other organizations

UCSD Computer Scientists Develop Ubiquitous Video Application For 3D Environments

June 13, 2005
University of California - San Diego
Computer scientists at UC San Diego unveiled a new technique for mixing images and live video feeds from roving cameras to provide remote viewers with a virtual window into a physical environment.

The RealityFlythrough software stitches together images and live video feeds to simulate the 3D environment.
Credit: Image courtesy of University of California - San Diego

Computer scientists at the University of California, San Diego have taken the wraps off a new technique for mixing images and video feeds from mobile cameras in the field to provide remote viewers with a virtual window into a physical environment. Dubbed 'RealityFlythrough,' the application constructs a 3D virtual environment dynamically out of the live video streams.

Related Articles

"Instead of watching all the feeds simultaneously on a bank of monitors, the viewer can navigate an integrated, interactive environment as if it were a video game," said UCSD computer science and engineering professor Bill Griswold, who is working on the project with Ph.D. candidate Neil McCurdy. "RealityFlythrough creates the illusion of complete live camera coverage in a physical space. It's a new form of situational awareness, and we designed a system that can work in unforgiving environments with intermittent network connectivity."

The researchers at UCSD's Jacobs School of Engineering have already begun testing the software for homeland security and emergency response, but they say that the technology has other potential consumer uses as well. "With virtual tourism, for instance, you could walk down the streets of Bangkok to see what it will be like before getting there," said McCurdy. "Another really cool application is pre-drive driving instructions. Imagine going to your favorite mapping website, where currently you get a set of instructions to turn left here or right there, and instead, you can 'fly' through the drive before doing it."

On June 6 at MobiSys 2005 in Seattle, McCurdy presented a joint paper* with Griswold about RealityFlythrough and a "systems architecture for ubiquitous video." The third international conference on mobile systems, applications and services brings together academic and industry researchers in the area of mobile and wireless systems.

Griswold and McCurdy are testing their new system as part of the WIISARD (Wireless Internet Information System for Medical Response in Disasters) project, which is funded by NIH's National Library of Medicine. During a May 12 disaster drill organized by San Diego's Metropolitan Medical Strike Team, the researchers shadowed a hazmat team responding to a simulated terrorist attack. They wore cameras mounted on their hardhats, tilt sensors with magnetic compasses, and global positioning (GPS) devices. Walking through the simulated disaster scene at the city's Cruise Ship Terminal, McCurdy and Griswold captured continuous video to be fed over an ad hoc wireless network to a makeshift command post nearby.

The RealityFlythrough software automatically stitches the feeds together, by integrating the visual data with the camera's location and direction it is pointing. "Our system works in ubiquitous and dynamic environments, and the cameras themselves are moving and shifting," said McCurdy, who expects to finish his Ph.D. in 2006. "RealityFlythrough situates still photographs or live video in a three-dimensional environment, making the transition between two cameras while projecting the images onto the screen. We're cheating and flattening space into two dimensions and then re-projecting the images in 3D space."

The UCSD researchers say the biggest research challenge was to overcome the limitation of incomplete coverage of live video streams. "Every square meter of a space cannot be viewed from every angle with a live video stream at any given moment," said Griswold, an academic participant in the California Institute for Telecommunications and Information Technology (Calit2). "We had to find a way to fill in the empty space that would give the user a sense of how the video streams relate to one another spatially."

Their solution: RealityFlythrough fills in the gaps in coverage with the most recent still images captured during camera pans. The software then blends the imagery with smooth transitions that simulate the sensation of a human performing a walking camera pan - even when one of the images is a still frame. If older images are not desirable (e.g. in some security applications), the fill-in images can be omitted, or shown in sepia, or include an icon displaying how old the photo is.

The fundamental research finding to date, according to McCurdy, is that some of the processing can be offloaded to the human. "We take advantage of a principle called closure, which allows our brains to make sense of incomplete information. The visual cortex does this all the time when it corrects for blind spots in our vision, for example," explained the graduate student. "RealityFlythrough supplies as much information as possible to the human operator, and the operator can easily fill in the blanks."

Human input is especially important indoors, where GPS cannot provide adequate location information. McCurdy carried a 'dead reckoning' device on his back during the May 12 disaster drill. The device uses gyros and other components to track body movement directions and footsteps from the moment the user enters an indoor area.

Since dead-reckoning systems lose accuracy over time, the researchers implemented a system that allows the camera operators to periodically correct their locations. "We created a Wizard-of-Oz approach to correcting inadequate location information," explained McCurdy. "Since we're combining this self-reporting technology with GPS or dead reckoning, it only has to be done occasionally. From all the footage we got from the May 12 drill, I only had to put in four corrections, and that was sufficient to give us pretty good accuracy indoors."

McCurdy will work on refining the system for his dissertation. And if consumers start to show interest in RealityFlythrough, he holds open the possibility of starting up a company to commercialize the technology -- but only after finishing his Ph.D. in 2006.

* "A Systems Architecture for Ubiquitous Video," Neil J. McCurdy and William G. Griswold. Proceedings of MobiSys 2005.

Story Source:

The above story is based on materials provided by University of California - San Diego. Note: Materials may be edited for content and length.

Cite This Page:

University of California - San Diego. "UCSD Computer Scientists Develop Ubiquitous Video Application For 3D Environments." ScienceDaily. ScienceDaily, 13 June 2005. <www.sciencedaily.com/releases/2005/06/050613062242.htm>.
University of California - San Diego. (2005, June 13). UCSD Computer Scientists Develop Ubiquitous Video Application For 3D Environments. ScienceDaily. Retrieved January 28, 2015 from www.sciencedaily.com/releases/2005/06/050613062242.htm
University of California - San Diego. "UCSD Computer Scientists Develop Ubiquitous Video Application For 3D Environments." ScienceDaily. www.sciencedaily.com/releases/2005/06/050613062242.htm (accessed January 28, 2015).

Share This

More From ScienceDaily

More Computers & Math News

Wednesday, January 28, 2015

Featured Research

from universities, journals, and other organizations

Featured Videos

from AP, Reuters, AFP, and other news services

PlayStation Now Smart TV App

PlayStation Now Smart TV App

Rumble (Jan. 27, 2015) PlayStation Now Smart TV app is coming soon and will be available on both Sony and Samsung HDTV, allowing you to play games without even a counsel! Check out the video for more info. Credit to &apos;booredatwork&apos;. Video provided by Rumble
Powered by NewsLook.com
WikiLeaks Accuses Google of Handing Over Emails to US

WikiLeaks Accuses Google of Handing Over Emails to US

AFP (Jan. 27, 2015) Whistleblowing site WikiLeaks accused Google of handing over the emails and electronic data of its senior staff to the US authorities without providing notification until almost three years later. Duration: 01:09 Video provided by AFP
Powered by NewsLook.com
Shark Bite Victim Making Amazing Recovery

Shark Bite Victim Making Amazing Recovery

AP (Jan. 27, 2015) A Texas woman who lost more than five pounds of flesh to a shark in the Bahamas earlier this month could be released from a Florida hospital soon. Experts believe she was bitten by a bull shark while snorkeling. (Jan. 27) Video provided by AP
Powered by NewsLook.com
Cablevision Enters Wi-Fi Phone Fray

Cablevision Enters Wi-Fi Phone Fray

Reuters - Business Video Online (Jan. 26, 2015) The entry by Cablevision and Google could intensify the already heated price wars for mobile phone service. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.


Breaking News:

Strange & Offbeat Stories

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News


Free Subscriptions

Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile

Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?

Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins