Control of attention and gaze in complex environments

Dealing with natural, complex scenes in everyday behavior, where one is surrounded by a variety of potentially relevant stimuli, poses an important problem for our visual system. Given the constraints set by attentional and working memory limitations in acquiring and retaining information, how does our visual system solve the problem of selecting appropriate information when it's needed, in the context of visually guided behavior? Though incomplete, overt fixations carry much information about current attentional state, and are a revealing indicator of this selection. What controls allocation of gaze and attention in natural environments? Traditionally, attention was thought to be attracted exogenously by the properties of the stimulus. Studies done using 2D experimental displays or viewing of scenes, showed that properties such as contrast or chromatic salience can explain some regularities in fixation patterns. These, however can account for only a modest proportion of the variance. Further, experimental contexts examined may not reflect the challenges of natural visually guided behavior. Complexity of the environment and the ongoing behavior make it necessary to look at natural behavior when investigating control of gaze. Recent work in natural tasks has demonstrated that the observer's cognitive goals play a critical role in the distribution of gaze during ongoing natural behavior.; The goal of this thesis is to understand the mechanisms that control the deployment of gaze in natural environments. Though fixation patterns in natural behavior are largely determined by the momentary task, it is not clear how effective top---down control is in dynamic environments because of the difficulty of dealing with unexpected events. To address this problem we studied gaze patterns in both real and virtual walking environments where subjects were occasionally exposed to potentially colliding pedestrians. Our results indicate potential collisions do not automatically attract attention and are usually detected by active search rather than by reacting to looming. If, however, a collider is detected, fixations on all pedestrians are increased in the subsequent few seconds, indicating that subjects learn the structure and dynamic properties of the world in order to fixate critical regions at the right time. We also investigated whether an addition of another perceptually demanding task interferes with the detection of potential collisions. In a situation of walking, while also following a leader pedestrian, detection of colliders decreased significantly indicating that subjects learn how to allocate attention and gaze to satisfy competing demands. In a real environment we investigated whether manipulation of the probability of a potential collision with pedestrians in predetermined roles, is accompanied by a corresponding change in gaze allocation. We demonstrated that fixation patterns adjust very quickly to changes in the probabilistic structure of the environment that indicate different priorities for gaze allocation. Based on our results, it appears that observers learn to represent sufficient structure about the visual environment in order to guide eye movement in a pro-active manner, in anticipation of events that are likely to occur in the scene To investigate the importance of behavioral relevance we compared fixation durations when walkers stopped instead of going on a collision path. Other than the reduction in fixation probabilities of about 20%, the pattern remained the same. This supports the idea that gaze behavior takes into account the risk (or reward) value of particular information, and is consistent with reinforcement learning models of gaze as well as with the neuro-physiological findings on the importance of reward. Finally we made a comparison of performance in real and virtual environments in order to evaluate the validity of the latter. The results in virtual reality walking strengthen our result in the real walking...