Featured Research

from universities, journals, and other organizations

Computer Model Mimicks How Brain Recognizes Street Scenes

Date:
February 16, 2007
Source:
McGovern Institute for Brain Research
Summary:
Scientists have developed a computational model of how the brain processes visual information and applied it to a complex, real world task: Recognizing the objects in a busy street scene. The researchers were pleasantly surprised at the power of this first application of a biologically inspired computer model for artificial vision, which has many potential practical applications.

The Poggio model for object recognition takes as input the unlabled images of digital photographs from the Street Scene Database (top) and generates automatic annotations of the type shown in the bottom row. The orange bounding boxes are for pedestrians ('ped') and cars ('car'). The system would have also detected bicycles if present. For sky, buildings, trees, and road, the system uses color coding (blue, brown, green, and grey). Note the false detection in the image on the right. A construction sign was mistaken for a pedestrian. (Graphic courtesy Stanley Bileschi, Ph.D., McGovern Institute for Brain Research at MIT)

At last, neuroscience is having an impact on computer science and artificial intelligence (AI). For the first time, scientists in Tomaso Poggio's laboratory at the McGovern Institute for Brain Research at MIT applied a computational model of how the brain processes visual information to a complex, real world task: recognizing the objects in a busy street scene. The researchers were pleasantly surprised at the power of this new approach.

Related Articles


"People have been talking about computers imitating the brain for a long time," said Poggio, who is also the Eugene McDermott Professor in the Department of Brain and Cognitive Sciences and the co-director of the Center for Biological and Computational Learning at MIT. "That was Alan Turing's original motivation in the 1940s. But in the last 50 years, computer science and AI have developed independently of neuroscience. Our work is biologically inspired computer science."

"We developed a model of the visual system that was meant to be useful for neuroscientists in designing and interpreting experiments, but that also could be used for computer science," said Thomas Serre, a former PhD student and now a post-doctoral researcher in Poggio's lab and lead author a paper about the street scene application in the 2007 IEEE Transactions on Pattern Analysis and Machine Intelligence. "We chose street scene recognition as an example because it has a restricted set of object categories, and it has practical social applications."

Near-term applications include surveillance and automobile driver's assistance, and eventually visual search engines, biomedical imaging analysis, robots with realistic vision. On the neuroscience end, this research is essential for designing augmented sensory prostheses, such as one that could replicate the computations carried by damaged nerves from the retina. "And once you have a good model of how the human brain works," Serre explained, "you can break it to mimic a brain disorder." One brain disorder that involves distortions in visual perception is schizophrenia, but nobody understands the neurobiological basis for those distortions.

"The versatility of the biological model turns computer vision from a trick into something really useful," said co-author Stanley Bileschi, a post-doctoral researcher in the Poggio lab. He and co-author Lior Wolf, a former post-doctoral associate who is now on the faculty of the Computer Science Department at Tel-Aviv University, are working with the MIT entrepreneur office, the Deshpande Center in the Sloan School. This center helps MIT students and professors bridge the gap between an intriguing idea or technology and a commercially viable concept.

Recognizing Scenes

The IEEE paper describes how the team "showed" the model randomly selected images so that it could "learn" to identify commonly occurring features in real-word objects, such as trees, cars, and people. In so-called supervised training sessions, the model used those features to label by category the varied examples of objects found in digital photographs of street scenes: buildings, cars, motorcycles, airplanes, faces, pedestrians, roads, skies, trees, and leaves. The photographs derive from a Street Scene Database compiled by Bileschi.

Compared to traditional computer-vision systems, the biological model was surprisingly versatile. Traditional systems are engineered for specific object classes. For instance, systems engineered to detect faces or recognize textures are poor at detecting cars. In the biological model, the same algorithm can learn to detect widely different types of objects.

To test the model, the team presented full street scenes consisting of previously unseen examples from the Street Scene Database. The model scanned the scene and, based on its supervised training, recognized the objects in the scene. The upshot is that the model learned from examples, which, according to Poggio, is a hallmark of artificial intelligence.

Modeling Object Recognition

Teaching a computer how to recognize objects has been exceedingly difficult because a computer model has two paradoxical goals. It needs to create a representation for a particular object that is very specific, such as a horse as opposed to a cow or a unicorn. At the same time the representation must be sufficiently "invariant" so as to discard meaningless changes in pose, illumination, size, position, and many other variations in appearances.

Even a child's brain handles these contradictory tasks easily in rapid object recognition. Pixel-like information enters from the retina and passes in a fast feed-forward, bottom-up sweep through the hierarchical architecture of the visual cortex. What makes the Poggio lab's model so innovative and powerful is that, computationally speaking, it mimics the brain's own hierarchy. Specifically, the "layers" within the model replicate the way neurons process input and output stimuli -- according to neural recordings in physiological labs. Like the brain, the model alternates several times between computations that help build an object representation that is increasingly invariant to changes in appearances of an object in the visual field and computations that help build an object representation that is increasingly complex and specific to a given object.

The model's success validates work in physiology labs that have measured the tuning properties of neurons throughout visual cortex. By necessity, most of those experiments are made with simplistic artificial stimuli, such as gratings, bars, and line drawings that bear little resemblance to real-world images. "We put together a system that mimics as closely as possible how cortical cells respond to simple stimuli like the ones that are used in the physiology lab," said Serre. "The fact that this system seems to work on realistic street scene images is a concept proof that the activity of neurons as measured in the lab is sufficient to explain how brains can perform complex recognition tasks."

Making it More Useful

The model used in the street scene application mimics only the computations the brain uses for rapid object recognition. The lab is now elaborating the model to include the brain's feedback loops from the cognitive centers. This slower form of object recognition provides time for context and reflection, such as: if I see a car, it must be on the road not in the sky. Giving the model the ability to recognize such semantic features will empower it for broader applications, including managing seemingly insurmountable amounts of data, work tasks, or even email. The team is also working on a model for recognizing motions and actions, such as walking or talking, which could be used to filter videos for anomalous behaviors -- or for smarter movie editing.

The Street Scene database is freely available at Center for Biological and Computational Learning website (http://cbcl.mit.edu/). Co-author Maximilian Riesenbuber began the work on the model for visual recognition for his Ph.D. dissertation in Poggio's lab and continues this work as assistant professor of neuroscience at the Georgetown University Medical Center. This research was partially funded by the U.S. Defense Advanced Research Projects Agency (DARPA), U.S. Office of Naval Research, and the U.S. National Science Foundation-National Institutes of Health.


Story Source:

The above story is based on materials provided by McGovern Institute for Brain Research. Note: Materials may be edited for content and length.


Cite This Page:

McGovern Institute for Brain Research. "Computer Model Mimicks How Brain Recognizes Street Scenes." ScienceDaily. ScienceDaily, 16 February 2007. <www.sciencedaily.com/releases/2007/02/070207171829.htm>.
McGovern Institute for Brain Research. (2007, February 16). Computer Model Mimicks How Brain Recognizes Street Scenes. ScienceDaily. Retrieved December 20, 2014 from www.sciencedaily.com/releases/2007/02/070207171829.htm
McGovern Institute for Brain Research. "Computer Model Mimicks How Brain Recognizes Street Scenes." ScienceDaily. www.sciencedaily.com/releases/2007/02/070207171829.htm (accessed December 20, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Saturday, December 20, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Building Google Into Cars

Building Google Into Cars

Reuters - Business Video Online (Dec. 19, 2014) — Google's next Android version could become the standard that'll power your vehicle's entertainment and navigation features, Reuters has learned. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com
After Sony Hack, What's Next?

After Sony Hack, What's Next?

Reuters - US Online Video (Dec. 19, 2014) — The hacking attack on Sony Pictures has U.S. government officials weighing their response to the cyber-attack. Linda So reports. Video provided by Reuters
Powered by NewsLook.com
Navy Unveils Robot Fish

Navy Unveils Robot Fish

Reuters - Light News Video Online (Dec. 18, 2014) — The U.S. Navy unveils an underwater device that mimics the movement of a fish. Tara Cleary reports. Video provided by Reuters
Powered by NewsLook.com
How 2014 Shaped The Future Of The Internet

How 2014 Shaped The Future Of The Internet

Newsy (Dec. 18, 2014) — It has been a long, busy year for Net Neutrality. The stage is set for an expected landmark FCC decision sometime in 2015. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins