Saliency and temporal integration for natural scenes

Claudia Wilimzig

Koch Lab, California Institute of Technology

It has been known for some time that for two differently colored lights flashed subsequently,
people report a blended version of both stimuli as if the process underlying perception
integrates over both stimuli (Efron,1967). We explore temporal integration processes for
natural scenes by briefly flashing (e.g., 10 ms) a natural scene followed by a brief flash
of a negative version of the same scene. The stimulus and its inverse version are inversely
related like a photo and its negative and can be obtained by subtracting the stimulus from the
maximum palette entry ineach respective color channel. Since presenting the inverse after
the stimulus reduces the visibility of the stimulus, it can be regarded as a "mask" for natural
scenes.

If the visual system perfectly averaged over stimulus and mask, subjects would report seeing
a uniform gray patch. Contrariwise, in some areas of the picture subjects reported deviations
from mere gray perception. We show (1.) that these areas are systematically related to
predictions of computational approaches to saliency (Itti & Koch, 2001, e. g.) such that
as a masking technique our paradigm masks everything but the most salient regions of the
image; (2.) that this masking effect is weakened but not abolished in a dichoptic version of
the experiment showing that retinalmechanisms may contribute but are not sufficient to
explain the results;(3.) that the spatial array of the percept is highly correlated with the
output of computational approaches to saliency; (4.) that the results are better accounted for
by saliency as a combination of different channels (color,intensity, flicker, orientation) rather
than intensity alone; (5.) that -comparable to the results for temporal integration of simple
stimuli-  the time course depends on spatial frequencies in accordance with a "from
coarse to fine grained"-hypothesis. While the role of saliency in natural scenes for eye
movements is well-known, these results show that saliency also plays an important role
in determining the time course of temporal integration for natural scenes.