Semantic Guidance of Eye Movement in Real-World Scenes

Marc Pomplun

Our visual world is composed of objects with not only low-level visual features such as shape and color, but also high-level features such as meaning and semantic relations among them. While it has been shownthat low-level features in real-world scenes guide eye movements during scene inspection and search, the influence of semantic similarity among scene objects on eye movements in such situations is unknown. I will present a study of semantic guidance of eye movements during real-world scene inspection and search. By selecting scenes from the LabelMe object-annotated image database and applying Latent Semantic Analysis (LSA) to the object labels, we generated semantic saliency (similarity) maps of real-world scenes. An ROC analysis ofthese maps as predictors of subjects’ gaze transitions between objects revealed subjects’ tendency to transition to objects that were semantically similar to the currently inspected one. Furthermore, during scene search, subjects’ eye movements were guided toward objects that were semantically similar to the search target. These findings demonstrate substantial semantic guidance of eye movements in real-world scenes and motivate its further study for a more comprehensive understanding of attentional control.