Learning cue validity through performance feedback.

Targets of a visual search are often not randomly positioned within a scene, but may be more likely to co-occur adjacent to other objects or background properties. Studies on target-cue co-occurrence (e.g. cue validity) suggest that observers can exploit this knowledge to increase performance in detection and localization tasks. However, little is known regarding how observers learn this co-occurrence. The present experiment sought to determine if observers were capable of learning the probability of cue validity, and determine how this learning is shaped by feedback. Separate groups of subjects performed a search task using one of three different feedback conditions providing varying degrees of information: unsupervised feedback, response reinforcement, or supervised feedback. Results show that saccadic and perceptual decisions reflect larger cueing effects as feedback information increased. This suggests that internal signals generated from response selection are insufficient for exploiting cue validity, but that reinforcement may be sufficient. However, final explicit estimates of cue validity were independent of feedback condition, suggesting that implicit behaviors are subject to unique learning constraints. Comparison to an ideal observer reveals that the rate at which participants learned cue validity was suboptimal, which may have impaired performance during initial familiarization with scene statistics.

Statistical decision theory to relate neurons to behavior in the study of covert visual attention.

Scrutiny of the numerous physiology and imaging studies of visual attention reveal that integration of results from neuroscience with the classic theories of visual attention based on behavioral work is not simple. The different subfields have pursued different questions, used distinct experimental paradigms and developed diverse models. The purpose of this review is to use statistical decision theory and computational modeling to relate classic theories of attention in psychological research to neural observables such as mean firing rate or functional imaging BOLD response, tuning functions, Fano factor, neuronal index of detectability and area under the receiver operating characteristic (ROC). We focus on cueing experiments and attempt to distinguish two major leading theories in the study of attention: limited resources model/increased sensitivity vs. selection/differential weighting. We use Bayesian ideal observer (BIO) modeling, in which predictive cues or prior knowledge change the differential weighting (prior) of sensory information to generate predictions of behavioral and neural observables based on Gaussian response variables and Poisson process neural based models. The ideal observer model can be modified to represent a number of classic psychological theories of visual attention by including hypothesized human attentional limited resources in the same way sequential ideal observer analysis has been used to include physiological processing components of human spatial vision (Geisler, W. S. (1989). Sequential ideal-observer analysis of visual discrimination. Psychological Review 96, 267-314.). In particular we compare new biologically plausible implementations of the BIO and variant models with limited resources. We find a close relationship between the behavioral effects of cues predicted by the models developed in the field of human psychophysics and their neuron-based analogs. Critically, we show that cue effects on experimental observables such as mean neural activity, variance, Fano factor and neuronal index of detectability can be consistent with the two major theoretical models of attention depending on whether the neuron is assumed to be computing likelihoods, log-likelihoods or a simple model operating directly on the Poisson variable. Change in neuronal tuning functions can also be consistent with both theories depending on whether the change in tuning is along the dimension being experimentally cued or a different dimension. We show that a neuron's sensitivity appropriately measured using the area under the Receive Operating Characteristic curve can be used to distinguish across both theories and is robust to the many transformations of the decision variable. We provide a summary table with the hope that it might provide some guidance in interpreting past results as well as planning future studies.

Trade-offs between gaze and working memory use.

Eye movements during natural tasks suggest that observers do not use working memory to capacity but instead use eye movements to acquire relevant information immediately before needed. Results here however, show that this strategy is sensitive to memory load and to observers' expectations about what information will be relevant. Depending upon the predictability of what object features would be needed in a brick sorting task, subjects spontaneously modulated the order in which they sampled and stored visual information using working memory more when the task was predictable and reverting to a just-in-time strategy when the task was unpredictable and the memory load was higher. This self organization was evidenced by subjects' sequence of eye movements and also their sorting decisions following missed feature changes. These results reveal that attentional selection, fixations, and use of working memory reflect a dynamic optimization with respect to a set of constraints, such as task predictablity and memory load. They also reveal that change blindness depends critically on the local task context, by virtue of its influence on the information selected for storage in working memory.

Learning where to direct gaze during change detection.

Where do observers direct their attention in complex scenes? Previous work on the cognitive control of fixation patterns in natural environments suggests that subjects must learn where to direct attention and gaze. We examined this question in the context of a change blindness paradigm, where some objects were more likely to undergo a change in orientation than others. The experiments revealed that observers are capable of learning the frequency with which objects undergo a change, and that this learning is manifested in the distribution of gaze among objects in the scene, as well as in the reaction time for detecting visual changes, and the frequency of localizing changing objects. However, observers were much less sensitive to the conditional probability of a second feature, border color, predicting a change in orientation. We conclude that striking demonstrations of change blindness may reflect not only the constraints of attention and working memory, but also what observers have learnt about what information to attend and select for storage during the task of change detection. Such exploitation of the frequency of change suggests that gaze allocation is sensitive to the probabilistic structure of the environment.

Task demands control acquisition and storage of visual information.

Attention and working memory limitations set strict limits on visual representations, yet researchers have little appreciation of how these limits constrain the acquisition of information in ongoing visually guided behavior. Subjects performed a brick sorting task in a virtual environment. A change was made to 1 of the features of the brick being held on about 10% of trials. Rates of change detection for feature changes were generally low and depended on the pick-up and put-down relevance of the feature to the sorting task. Subjects' sorting decision suggests that changes may be missed because of a failure to update the changed feature. The authors also explore how hand and eye behavior are coordinated for strategic acquisition and storage of visual information throughout the task.

Activity of neurons in cortical area MT during a memory for motion task.

We recorded the activity of middle temporal (MT) neurons in 2 monkeys while they compared the directions of motion in 2 sequentially presented random-dot stimuli, sample and test, and reported them as the same or different by pressing one of 2 buttons. We found that MT neurons were active not only in response to the sample and test stimuli but also during the 1,500-ms delay separating them. Most neurons showed a characteristic pattern of activity consisting of a small burst of firing early in the delay, followed by a period of suppression and a subsequent increase in firing rate immediately preceding the presentation of the test stimulus. In a third of the neurons, the activity early in the delay not only reflected the direction of the sample stimulus, but was also related to the range of local directions it contained. During the middle of the delay the majority of neurons were suppressed, consistent with a gating mechanism that could be used to ignore task-irrelevant stimuli. Late in the delay, most neurons showed an increase in response, probably in anticipation of the upcoming test. Throughout most of the delay there was a directional signal in the population of MT neurons, manifested by higher firing rates following the sample moving in the antipreferred direction. Whereas some of these effects may be related to sensory adaptation, others are more likely to represent a more active task-related process. These results support the hypothesis that MT neurons actively participate in the successful execution of all aspects of the task requiring processing and remembering visual motion.