Temporal attention recruits fronto-cingulate cortex to amplify stimulus representations.

The human brain receives a continuous stream of input, but it faces significant constraints in its ability to process every item in a sequence of stimuli. Voluntary temporal attention can alleviate these constraints by using information about upcoming stimulus timing to selectively prioritize a task-relevant item over others in a sequence. But the neural mechanisms underlying this ability remain unclear. Here, we manipulated temporal attention to successive stimuli in a two-target temporal cueing task, while controlling for temporal expectation by using fully predictable stimulus timing. We recorded magnetoencephalography (MEG) in human observers and measured the effects of temporal attention on orientation representations of each stimulus using time-resolved multivariate decoding in both sensor and source space. Voluntary temporal attention enhanced the orientation representation of the first target 235-300 milliseconds after target onset. Unlike previous studies that did not isolate temporal attention from temporal expectation, we found no evidence that temporal attention enhanced early visual evoked responses. Instead, and unexpectedly, the primary source of enhanced decoding for attended stimuli in the critical time window was a contiguous region spanning left frontal cortex and cingulate cortex. The results suggest that voluntary temporal attention recruits cortical regions beyond the ventral stream at an intermediate processing stage to amplify the representation of a target stimulus, which may serve to protect it from subsequent interference by a temporal competitor.

Studying the neural representations of uncertainty.

The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.

Tasks and their role in visual neuroscience.

Vision is widely used as a model system to gain insights into how sensory inputs are processed and interpreted by the brain. Historically, careful quantification and control of visual stimuli have served as the backbone of visual neuroscience. There has been less emphasis, however, on how an observer's task influences the processing of sensory inputs. Motivated by diverse observations of task-dependent activity in the visual system, we propose a framework for thinking about tasks, their role in sensory processing, and how we might formally incorporate tasks into our models of vision.

Exogenous temporal attention varies with temporal uncertainty.

Temporal attention is the selection and prioritization of information at a specific moment. Exogenous temporal attention is the automatic, stimulus driven deployment of attention. The benefits and costs of exogenous temporal attention on performance have not been isolated. Previous experimental designs have precluded distinguishing the effects of attention and expectation about stimulus timing. Here, we manipulated exogenous temporal attention and the uncertainty of stimulus timing independently and investigated visual performance at the attended and unattended moments with different levels of temporal uncertainty. In each trial, two Gabor patches were presented consecutively with a variable stimulus onset. To drive exogenous attention and test performance at attended and unattended moments, a task-irrelevant, brief cue was presented 100 ms before target onset, and an independent response cue was presented at the end of the trial. Exogenous temporal attention slightly improved accuracy, and the effects varied with temporal uncertainty, suggesting a possible interaction of temporal attention and expectations in time.

An auditory-visual tradeoff in susceptibility to clutter.

Sensory cortical mechanisms combine auditory or visual features into perceived objects. This is difficult in noisy or cluttered environments. Knowing that individuals vary greatly in their susceptibility to clutter, we wondered whether there might be a relation between an individual's auditory and visual susceptibilities to clutter. In auditory masking, background sound makes spoken words unrecognizable. When masking arises due to interference at central auditory processing stages, beyond the cochlea, it is called informational masking. A strikingly similar phenomenon in vision, called visual crowding, occurs when nearby clutter makes a target object unrecognizable, despite being resolved at the retina. We here compare susceptibilities to auditory informational masking and visual crowding in the same participants. Surprisingly, across participants, we find a negative correlation (R = -0.7) between susceptibility to informational masking and crowding: Participants who have low susceptibility to auditory clutter tend to have high susceptibility to visual clutter, and vice versa. This reveals a tradeoff in the brain between auditory and visual processing.

A dynamic normalization model of temporal attention.

Vision is dynamic, handling a continuously changing stream of input, yet most models of visual attention are static. Here, we develop a dynamic normalization model of visual temporal attention and constrain it with new psychophysical human data. We manipulated temporal attention-the prioritization of visual information at specific points in time-to a sequence of two stimuli separated by a variable time interval. Voluntary temporal attention improved perceptual sensitivity only over a specific interval range. To explain these data, we modelled voluntary and involuntary attentional gain dynamics. Voluntary gain enhancement took the form of a limited resource over short time intervals, which recovered over time. Taken together, our theoretical and experimental results formalize and generalize the idea of limited attentional resources across space at a single moment to limited resources across time at a single location.

Modeling pupil responses to rapid sequential events.

Pupil size is an easily accessible, noninvasive online indicator of various perceptual and cognitive processes. Pupil measurements have the potential to reveal continuous processing dynamics throughout an experimental trial, including anticipatory responses. However, the relatively sluggish (~2 s) response dynamics of pupil dilation make it challenging to connect changes in pupil size to events occurring close together in time. Researchers have used models to link changes in pupil size to specific trial events, but such methods have not been systematically evaluated. Here we developed and evaluated a general linear model (GLM) pipeline that estimates pupillary responses to multiple rapid events within an experimental trial. We evaluated the modeling approach using a sample dataset in which multiple sequential stimuli were presented within 2-s trials. We found: (1) Model fits improved when the pupil impulse response function (PuRF) was fit for each observer. PuRFs varied substantially across individuals but were consistent for each individual. (2) Model fits also improved when pupil responses were not assumed to occur simultaneously with their associated trial events, but could have non-zero latencies. For example, pupil responses could anticipate predictable trial events. (3) Parameter recovery confirmed the validity of the fitting procedures, and we quantified the reliability of the parameter estimates for our sample dataset. (4) A cognitive task manipulation modulated pupil response amplitude. We provide our pupil analysis pipeline as open-source software (Pupil Response Estimation Toolbox: PRET) to facilitate the estimation of pupil responses and the evaluation of the estimates in other datasets.

The Confidence Database.

Understanding how people rate their confidence is critical for the characterization of a wide range of perceptual, memory, motor and cognitive processes. To enable the continued exploration of these processes, we created a large database of confidence studies spanning a broad set of paradigms, participant populations and fields of study. The data from each study are structured in a common, easy-to-use format that can be easily imported and analysed using multiple software packages. Each dataset is accompanied by an explanation regarding the nature of the collected data. At the time of publication, the Confidence Database (which is available at https://osf.io/s46pr/) contained 145 datasets with data from more than 8,700 participants and almost 4 million trials. The database will remain open for new submissions indefinitely and is expected to continue to grow. Here we show the usefulness of this large collection of datasets in four different analyses that provide precise estimations of several foundational confidence-related effects.

Temporal attention improves perception similarly at foveal and parafoveal locations.

Temporal attention, the prioritization of information at a specific point in time, improves visual performance, but it is unknown whether it does so to the same extent across the visual field. This knowledge is necessary to establish whether temporal attention compensates for heterogeneities in discriminability and speed of processing across the visual field. Discriminability and rate of information accrual depend on eccentricity as well as on polar angle, a characteristic known as performance fields. Spatial attention improves speed of processing more at locations at which discriminability is lower and information accrual is slower, but it improves discriminability to the same extent across isoeccentric locations. Here we asked whether temporal attention benefits discriminability in a similar or differential way across the visual field. Observers were asked to report the orientation of one of two targets presented at different points in time at the same spatial location (fovea, right horizontal meridian, or upper vertical meridian, blocked). Temporal attention improved discriminability and shortened reaction times at the foveal and each parafoveal location similarly. These results provide evidence that temporal attention is similarly effective at multiple locations in the visual field. Consequently, at the tested locations, performance fields are preserved with temporal orienting of attention.

Directing Voluntary Temporal Attention Increases Fixational Stability.

Our visual input is constantly changing, but not all moments are equally relevant. Visual temporal attention, the prioritization of visual information at specific points in time, increases perceptual sensitivity at behaviorally relevant times. The dynamic processes underlying this increase are unclear. During fixation, humans make small eye movements called microsaccades, and inhibiting microsaccades improves perception of brief stimuli. Here, we investigated whether temporal attention changes the pattern of microsaccades in anticipation of brief stimuli. Human observers (female and male) judged stimuli presented within a short sequence. Observers were given either an informative precue to attend to one of the stimuli, which was likely to be probed, or an uninformative (neutral) precue. We found strong microsaccadic inhibition before the stimulus sequence, likely due to its predictable onset. Critically, this anticipatory inhibition was stronger when the first target in the sequence (T1) was precued (task-relevant) than when the precue was uninformative. Moreover, the timing of the last microsaccade before T1 and the first microsaccade after T1 shifted such that both occurred earlier when T1 was precued than when the precue was uninformative. Finally, the timing of the nearest pre- and post-T1 microsaccades affected task performance. Directing voluntary temporal attention therefore affects microsaccades, helping to stabilize fixation at the most relevant moments over and above the effect of predictability. Just as saccading to a relevant stimulus can be an overt correlate of the allocation of spatial attention, precisely timed gaze stabilization can be an overt correlate of the allocation of temporal attention.SIGNIFICANCE STATEMENT We pay attention at moments in time when a relevant event is likely to occur. Such temporal attention improves our visual perception, but how it does so is not well understood. Here, we discovered a new behavioral correlate of voluntary, or goal-directed, temporal attention. We found that the pattern of small fixational eye movements called microsaccades changes around behaviorally relevant moments in a way that stabilizes the position of the eyes. Microsaccades during a brief visual stimulus can impair perception of that stimulus. Therefore, such fixation stabilization may contribute to the improvement of visual perception at attended times. This link suggests that, in addition to cortical areas, subcortical areas mediating eye movements may be recruited with temporal attention.

Humans incorporate attention-dependent uncertainty into perceptual decisions and confidence.

Perceptual decisions are better when they take uncertainty into account. Uncertainty arises not only from the properties of sensory input but also from cognitive sources, such as different levels of attention. However, it is unknown whether humans appropriately adjust for such cognitive sources of uncertainty during perceptual decision-making. Here we show that, in a task in which uncertainty is relevant for performance, human categorization and confidence decisions take into account uncertainty related to attention. We manipulated uncertainty in an orientation categorization task from trial to trial using only an attentional cue. The categorization task was designed to disambiguate decision rules that did or did not depend on attention. Using formal model comparison to evaluate decision behavior, we found that category and confidence decision boundaries shifted as a function of attention in an approximately Bayesian fashion. This means that the observer's attentional state on each trial contributed probabilistically to the decision computation. This responsiveness of an observer's decisions to attention-dependent uncertainty should improve perceptual decisions in natural vision, in which attention is unevenly distributed across a scene.

Illusory occlusion affects stereoscopic depth perception.

When occlusion and binocular disparity cues conflict, what visual features determine how they combine? Sensory cues, such as T-junctions, have been suggested to be necessary for occlusion to influence stereoscopic depth perception. Here we show that illusory occlusion, with no retinal sensory cues, interacts with binocular disparity when perceiving depth. We generated illusory occlusion using stimuli filled in across the retinal blind spot. Observers viewed two bars forming a cross with the intersection positioned within the blind spot. One of the bars was presented binocularly with a disparity signal; the other was presented monocularly, extending through the blind spot, with no defined disparity. When the monocular bar was perceived as filled in through the blind spot, it was perceived as occluding the binocular bar, generating illusory occlusion. We found that this illusory occlusion influenced perceived stereoscopic depth: depth estimates were biased to be closer or farther, depending on whether a bar was perceived as in front of or behind the other bar, respectively. Therefore, the perceived relative depth position, based on filling-in cues, set boundaries for interpreting metric stereoscopic depth cues. This suggests that filling-in can produce opaque surface representations that can trump other depth cues such as disparity.

Recent cross-modal statistical learning influences visual perceptual selection.

Incoming sensory signals are often ambiguous and consistent with multiple perceptual interpretations. Information from one sensory modality can help to resolve ambiguity in another modality, but the mechanisms by which multisensory associations come to influence the contents of conscious perception are unclear. We asked whether and how novel statistical information about the coupling between sounds and images influences the early stages of awareness of visual stimuli. We exposed subjects to consistent, arbitrary pairings of sounds and images and then measured the impact of this recent passive statistical learning on subjects' initial conscious perception of a stimulus by employing binocular rivalry, a phenomenon in which incompatible images presented separately to the two eyes result in a perceptual alternation between the two images. On each trial of the rivalry test, subjects were presented with a pair of rivalrous images (one of which had been consistently paired with a specific sound during exposure while the other had not) and an accompanying sound. We found that, at the onset of binocular rivalry, an image was significantly more likely to be perceived, and was perceived for a longer duration, when it was presented with its paired sound than when presented with other sounds. Our results indicate that recently acquired multisensory information helps resolve sensory ambiguity, and they demonstrate that statistical learning is a fast, flexible mechanism that facilitates this process.

Suboptimality in perceptual decision making.

Human perceptual decisions are often described as optimal. Critics of this view have argued that claims of optimality are overly flexible and lack explanatory power. Meanwhile, advocates for optimality have countered that such criticisms single out a few selected papers. To elucidate the issue of optimality in perceptual decision making, we review the extensive literature on suboptimal performance in perceptual tasks. We discuss eight different classes of suboptimal perceptual decisions, including improper placement, maintenance, and adjustment of perceptual criteria; inadequate tradeoff between speed and accuracy; inappropriate confidence ratings; misweightings in cue combination; and findings related to various perceptual illusions and biases. In addition, we discuss conceptual shortcomings of a focus on optimality, such as definitional difficulties and the limited value of optimality claims in and of themselves. We therefore advocate that the field drop its emphasis on whether observed behavior is optimal and instead concentrate on building and testing detailed observer models that explain behavior across a wide range of tasks. To facilitate this transition, we compile the proposed hypotheses regarding the origins of suboptimal perceptual decisions reviewed here. We argue that verifying, rejecting, and expanding these explanations for suboptimal behavior - rather than assessing optimality per se - should be among the major goals of the science of perceptual decision making.

Behavior is sensible but not globally optimal: Seeking common ground in the optimality debate.

The disagreements among commentators may appear substantial, but much of the debate seems to stem from inconsistent use of the term optimality. Optimality can be used to indicate sensible behavior (adapted to the environment), globally optimal behavior (fully predicted from optimality considerations alone), locally optimal behavior (conforming to a specific model), and optimality as an empirical strategy (a tool for studying behavior). Distinguishing among these different concepts uncovers considerable common ground in the optimality debate.

Feature reliability determines specificity and transfer of perceptual learning in orientation search.

Training can modify the visual system to produce a substantial improvement on perceptual tasks and therefore has applications for treating visual deficits. Visual perceptual learning (VPL) is often specific to the trained feature, which gives insight into processes underlying brain plasticity, but limits VPL's effectiveness in rehabilitation. Under what circumstances VPL transfers to untrained stimuli is poorly understood. Here we report a qualitatively new phenomenon: intrinsic variation in the representation of features determines the transfer of VPL. Orientations around cardinal are represented more reliably than orientations around oblique in V1, which has been linked to behavioral consequences such as visual search asymmetries. We studied VPL for visual search of near-cardinal or oblique targets among distractors of the other orientation while controlling for other display and task attributes, including task precision, task difficulty, and stimulus exposure. Learning was the same in all training conditions; however, transfer depended on the orientation of the target, with full transfer of learning from near-cardinal to oblique targets but not the reverse. To evaluate the idea that representational reliability was the key difference between the orientations in determining VPL transfer, we created a model that combined orientation-dependent reliability, improvement of reliability with learning, and an optimal search strategy. Modeling suggested that not only search asymmetries but also the asymmetric transfer of VPL depended on preexisting differences between the reliability of near-cardinal and oblique representations. Transfer asymmetries in model behavior also depended on having different learning rates for targets and distractors, such that greater learning for low-reliability distractors facilitated transfer. These findings suggest that training on sensory features with intrinsically low reliability may maximize the generalizability of learning in complex visual environments.

Filling-in rivalry: Perceptual alternations in the absence of retinal image conflict.

During perceptual rivalry, an observer's perceptual experience alternates over time despite constant sensory stimulation. Perceptual alternations are thought to be driven by conflicting or ambiguous retinal image features at a particular spatial location and modulated by global context from surrounding locations. However, rivalry can also occur between two illusory stimuli-such as two filled-in stimuli within the retinal blind spot. In this "filling-in rivalry," what observers perceive in the blind spot changes in the absence of local stimulation. It remains unclear if filling-in rivalry shares common mechanisms with other types of rivalry. We measured the dynamics of rivalry between filled-in percepts in the blind spot, finding a high degree of exclusivity (perceptual dominance of one filled-in percept, rather than a perception of transparency), alternation rates that were highly consistent for individual observers, and dynamics that closely resembled other forms of perceptual rivalry. The results suggest that mechanisms common to a wide range of rivalry situations need not rely on conflicting retinal image signals.

Attention flexibly trades off across points in time.

Sensory signals continuously enter the brain, raising the question of how perceptual systems handle this constant flow of input. Attention to an anticipated point in time can prioritize visual information at that time. However, how we voluntarily attend across time when there are successive task-relevant stimuli has been barely investigated. We developed a novel experimental protocol that allowed us to assess, for the first time, both the benefits and costs of voluntary temporal attention when perceiving a short sequence of two or three visual targets with predictable timing. We found that when humans directed attention to a cued point in time, their ability to perceive orientation was better at that time but also worse earlier and later. These perceptual tradeoffs across time are analogous to those found across space for spatial attention. We concluded that voluntary attention is limited, and selective, across time.

Perceptual suppression of predicted natural images.

Perception is shaped not only by current sensory inputs but also by expectations generated from past sensory experience. Humans viewing ambiguous stimuli in a stable visual environment are generally more likely to see the perceptual interpretation that matches their expectations, but it is less clear how expectations affect perception when the environment is changing predictably. We used statistical learning to teach observers arbitrary sequences of natural images and employed binocular rivalry to measure perceptual selection as a function of predictive context. In contrast to previous demonstrations of preferential selection of predicted images for conscious awareness, we found that recently acquired sequence predictions biased perceptual selection toward unexpected natural images and image categories. These perceptual biases were not associated with explicit recall of the learned image sequences. Our results show that exposure to arbitrary sequential structure in the environment impacts subsequent visual perceptual selection and awareness. Specifically, for natural image sequences, the visual system prioritizes what is surprising, or statistically informative, over what is expected, or statistically likely.

Functional mapping of the magnocellular and parvocellular subdivisions of human LGN.

The magnocellular (M) and parvocellular (P) subdivisions of primate LGN are known to process complementary types of visual stimulus information, but a method for noninvasively defining these subdivisions in humans has proven elusive. As a result, the functional roles of these subdivisions in humans have not been investigated physiologically. To functionally map the M and P subdivisions of human LGN, we used high-resolution fMRI at high field (7 T and 3 T) together with a combination of spatial, temporal, luminance, and chromatic stimulus manipulations. We found that stimulus factors that differentially drive magnocellular and parvocellular neurons in primate LGN also elicit differential BOLD fMRI responses in human LGN and that these responses exhibit a spatial organization consistent with the known anatomical organization of the M and P subdivisions. In test-retest studies, the relative responses of individual voxels to M-type and P-type stimuli were reliable across scanning sessions on separate days and across sessions at different field strengths. The ability to functionally identify magnocellular and parvocellular regions of human LGN with fMRI opens possibilities for investigating the functions of these subdivisions in human visual perception, in patient populations with suspected abnormalities in one of these subdivisions, and in visual cortical processing streams arising from parallel thalamocortical pathways.