Gaze Guidance by Scene Semantics and Syntax

Melissa Le­Hoa Võ
Visual Attention Lab
Harvard Medical School

The world around us does not constitute a random aggregation of 
isolated objects. Rather, the arrangement of objects in naturalistic 
scenes conforms to certain semantic (i.e., content related) and 
syntactic (i.e., structure related) rules. For example, we know that 
a kitchen tends to contain pots and that a pot is often found on top 
of a stove. In this talk, I will present a series of eye tracking 
experiments that speak for the key role of scene semantics and syntax 
in guiding gaze during scene viewing. In particular, I will show that 
1) both semantic and syntactic object­-scene inconsistencies attract 
gaze, however, only after their foveal processing. 2) General 
knowledge of scene semantics and syntax can guide search, but needs 
time to be integrated with the current visual information. 3) When 
repeatedly searching for different objects in the same scene, gaze 
seems to continue to be guided by semantic and syntactic scene 
knowledge rather than by episodic scene memory.