Distinguishing How The Visual System Perceives Approaching Objects

In an article published in the April 12, 2001, issue of the journal Nature, Howard Hughes Medical Institute investigator Eero P. Simoncelli at New York University and colleagues Paul R. Schrater and David C. Knill reported that the visual system can extract information about forward motion by estimating the rate of change in size of approaching objects. Schrater is at the University of Minnesota and Knill is at the University of Rochester. The authors collaborated on the research while they were at the University of Pennsylvania.

When an observer moves toward an object, the image expands on his or her retina. The pattern of local motions on the retina is known as optic flow. Schrater, Knill and Simoncelli showed that the visual system does not rely solely on optic flow to perceive information about objects as they move closer. "We think of these expanding patterns as a single phenomenon," said Simoncelli. "But there are really two sources of visual information—the size change and the optic flow. So, in our work we sought to design visual stimuli that could isolate size changes from optic flow."

The stimuli the scientists created to test the perceptions of human subjects consisted of a textured visual field of random light and dark areas. To simulate both size change and optic flow, the scientists created an animation in which the elements changed size and separated, as objects do when an observer moves toward them.

To simulate size change alone, the scientists created an animation in which the texture elements increased in scale. In order to eliminate any perception of optic flow, however, the texture elements randomly changed position as the scale increased.

The scientists also used an optic flow animation developed by other scientists, in which randomly placed dots moved away from one another. Because the dots appeared for short intervals and then disappeared, the animation eliminated the possibility that observers could use size-change information to see the expansion. "This animation prevented people from seeing clusters of the points, grouping them together, and thinking of them as expanding objects," said Simoncelli.

The researchers used the animations to do two basic experiments using human subjects. "In the first experiment, we asked subjects to compare the expansion rate they perceived in the animations of pure size change and pure optic flow. That is, they just told us which one of the two stimuli appeared to grow faster," Simoncelli said. "We found that the subjects were able to perceive expansion in the animation that showed only size change without optic flow.

"This wasn't a surprise to us because we could just look at the animations ourselves and see the expansion," said Simoncelli. "But we also found that subjects were good at judging the rate of expansion from size change alone. They were not quite as good at making these judgments as they were with stimuli that also contained optic flow information. But the size change alone was sufficient for them to make the judgments."

However, if perceptions were truly due to early stage visual-system processing, said Simoncelli, the subjects should experience a well-known visual illusion known as the "waterfall illusion."

"It's been known for centuries that when you stare at something moving for a long time and then look at a stationary object, it will appear to move in the opposite direction," explained Simoncelli. "For example, when you stare at a waterfall for a while and then move your gaze to the adjacent rocks, they will appear to move upwards." In a similar way, when the visual system is exposed to an expanding animation, it should perceive a subsequently viewed stationary pattern as contracting, he said. The researchers found that when subjects were exposed to the expanding size-change animation, their judgments of mixed-motion animations did indeed show such visual after-effects.

"The after-effect provides more compelling evidence that there's a specific mechanism involved in sensing size change information," said Simoncelli.

While these kinds of experiments can reveal that a visual mechanism is devoted to perceiving size change, said Simoncelli, extensive animal studies will be needed to understand the neural circuitry involved. "It has been well known for some time that there are mechanisms in the mammalian visual cortex that are specialized for motion perception," he said. "But just saying that there are a set of neurons that respond to expanding stimuli doesn't specify the cascade of computations that are done, starting with the retina and propagating through the system to the responses of those neurons."