Visual working memory in the context of ongoing natural behaviors

How is information represented in working memory as we perform simple visuo-motor tasks? To study this we had subjects perform a copying task in which the goal was to make a duplicate of a pattern of objects on a computer monitor. Previous work in this lab suggested that observers seek information as they need it during the task, and avoid using complex visual representations. The experiments presented here are an attempt to use this technique to extend these findings and ask specific questions about visual working memory function. One set of experiments was designed to generalize the results from previous block copying experiments to a wider range of experimental parameters. We allowed subjects to manipulate more than one object at a time and used digitized images of real objects instead of colored blocks. We found that, despite these new parameters, subjects still performed the task in the same way, breaking the task down into small subtasks and frequently inspecting the pattern as they copy it. In another set of experiments we tested whether the eye movements observed in these experiments are obligatory or a consequence of the eyes being faster than the hand. We made the pattern invisible for a significant portion of the task. Subjects reworked their strategies for performing the task such that they spent the same amount of time looking at the pattern in the blanked condition. This supports the idea that subjects rely on frequent inspections of the model pattern in order to complete this task. We also changed the order of how the items were placed while making the copy of the pattern. The items were either placed in the same order as they were picked up or in the reverse order. There was no difference in performance between these two conditions. This suggests that there is not preset structure for serial order in working memory. In another set of experiments we changed the color of one or more blocks during a saccade to the pattern. The duration of fixations following the saccade changed, depending on the point in the task that the change was made. This is consistent with the idea that visual representations are limited and task dependent. There was a greater increase when more than one block was changed. This suggests that some global aspects of the pattern were retained across different fixations. In the last experiment we examined how eye movement strategies change when information gets compressed in memory as subjects become familiar with patterns through repeated exposures. We observed that fixations are a good indicator of the degree of compression in memory. These experiments support a model of top-down vision in which eye movements are an important tool in performing common visuo-motor tasks. These experiments are consistent with a view of working memory that relies on external information and maintains minimal internal visual representations.