Three Elements of Visual Decision Making

Decisions made on the basis of visual information are ubiquitous and often of critical importance. The process of visual decision making can be broken down into three essential elements: 1) An encoding phase in which a representation of the image in terms of neural responses is formed, 2) A decision phase in which the encoded image is considered along with the probabilities and potential gains relevant to the task in order to make a decision, 3) a refixation phase in which the decision is delayed in order to allow for eye movements and hence, multiple encodings of the image. In each chapter that follows, we outline one of these elements. We apply probabilistic models to each process in order to generate novel hypotheses as to how each is carried out, and compare these models to the performance of human subjects in a series of experiments. In Chapter 1, subjects performed a visual search task involving complex stimuli with an uncertain angle of rotation. We show that their performance is predicted by a model that represents the stimuli in terms of natural image statistics and not in terms of luminance intensities. In Chapter 2, subjects performed a visual search task in which the probabilities of target occurrence and the rewards for a correct response were varied across potential target locations. In order to maximize their rewards, subject's should bias their choice of target location in line with the probabilities and rewards. We find a commonly observed pattern of suboptimal response bias in which subject do not bias their responses to the degree that they should and thus fail to maximize rewards. We show that this suboptimality is predicted by the subjective, nonlinear weighting of the explicit probabilities and rewards on the part of the subject. In Chapter 3, observers performed a visual search task in which the reward for correct detection was varied across potential target locations. Subject's choice of eye position in the search task differed significantly from that of an ideal observer that chooses the eye position on every trial that maximizes expected gain.