Room's Ambience Fingerprinted By Phone

Duke University computer engineers have made use of standard cell phone features – accelerometers, cameras and microphones – to turn the unique properties of a particular space into a distinct fingerprint. While standard global positioning systems (GPS) are only accurate to 10 meters (32 feet) and do not work indoors, the new application is designed to work indoors and can be as precise as telling if a user is on one side of an interior wall or another.

The system, dubbed SurroundSense, uses the phone's built-in camera and microphone to record sound, light and colors, while the accelerometer records movement patterns of the phone's user. This information is sent to a server, which knits the disparate information together into a single fingerprint.

"You can't tell much from any of the measurements individually, but when combined, the optical, acoustic and motion information creates a unique fingerprint of the space," said Ionut Constandache, graduate student in computer science. He presented the details of SurroundSense at the 15th International Conference on Mobile Computing and Networking in Bejing on Sept. 25.

For example, in a bar, people spend little time moving and most time sitting, while the room is typically dark and noisy. In contrast, a Target store will be brightly lit with vibrant colors – especially red – with movement up and down aisles. SurroundSense can tell these differences.

Students of Romit Roy Chouhury, Duke assistant professor of electrical and computer engineering and senior member of the research team, fanned out across Durham, N.C. with their cell phones, collecting data in different types of businesses. So that they would not bias the measurements, the students "mirrored" the actions of selected customers.

"We went to 51 different stores and found that SurroundSense achieved an average accuracy of about 87 percent when all of the sensing capabilities were used," Constandache said.

As more people use the application, it gets "smarter."

"As the system collects and analyzes more and more information about a particular site, the fingerprint becomes that much more precise," said Roy Choudhury. "Not only is the ambience different at different locations, but also can be different at different times at the same location."

SurroundSense collects data at different time points, so it would be able to distinguish a Starbucks store at the morning rush when there are many customers from the slower period in mid-afternoon.

"We believe that SurroundSense is an early step toward a long-standing challenge of improving indoor localization," Roy Choudhury said.

Currently, in order for the phone to collect data, it must be held with the camera facing down, though the researchers are working on strategies for the application to work if the phone is in a pocket, case or handbag. However, as the researchers pointed out, phones are now coming onto the market that are worn on the wrist or around the neck on a necklace.

As in many technical advances, it appears that batteries can be an Achilles' heel. The Duke researchers are now considering the tradeoffs between having the application "on" all the time, which drains the battery faster, or having it take measurements at regular intervals. They are also trying to determine whether the entire application should be housed on the server, the phone, or some combination of the two.

Roy Choudhury's research is supported by the National Science Foundation, Nokia, Verizon and Microsoft Research. Duke undergraduate Martin Azizyan also participated in the project.