Memory for scenes: May I have the spatial envelope, please?

Aude Oliva & Jeremy M Wolfe

Harvard Medical SchoolWe had subjects perform picture memory and boundary recognition experiments with real world scenes to determine how spatial envelope properties affect scene memory. The spatial envelope (Oliva and Torralba, 2001, Int.J.Comp.Vision, 42,145-175) characterizes scenes on various spatial dimensions including rankings from open to enclosed spaces, from flat to perspective views, from low to high complexity (e.g. roughness) and from close-up to far views. These space properties represent the scene holistically and do not require object segmentation or recognition. In the picture memory experiment, 20 Ss judged test scenes as old or new. New stimuli could be similar or dissimilar from old in spatial envelope "space". When new stimuli were dissimilar from old, performance could be predicted by ranking scenes on the complexity dimension of the spatial envelope. Performance increased with visual complexity (from d'=1.1 to d'=2.1). When the spatial envelope of each new scene was similar to one of the old scenes and when the semantic categories of these pairs were the same (e.g. two kitchens), performance was poor for relatively simple scenes (d' = 0.5) but better for more complex scenes (d' = 1.15). In the boundary recognition experiment, Ss drew or adjusted boundaries of scenes after viewing them for a few seconds. Intraub's classic finding is that Ss draw more than they have seen. Our results suggest boundary extension is influenced by the spatial envelope of the scene. Ss seemed to fill out truncated spaces in relatively closed/rough spaces and to remove voids in relatively open spaces. We conclude that some of the scene properties captured by the spatial envelope model are systematically related to the properties that are preserved when a visual image is compressed into a memory trace.