AttentionMNIST: a mouse-click attention tracking dataset for handwritten numeral and alphabet recognition.

Multiple attention-based models that recognize objects via a sequence of glimpses have reported results on handwritten numeral recognition. However, no attention-tracking data for handwritten numeral or alphabet recognition is available. Availability of such data would allow attention-based models to be evaluated in comparison to human performance. We collect mouse-click attention tracking data from 382 participants trying to recognize handwritten numerals and alphabets (upper and lowercase) from images via sequential sampling. Images from benchmark datasets are presented as stimuli. The collected dataset, called AttentionMNIST, consists of a sequence of sample (mouse click) locations, predicted class label(s) at each sampling, and the duration of each sampling. On average, our participants observe only 12.8% of an image for recognition. We propose a baseline model to predict the location and the class(es) a participant will select at the next sampling. When exposed to the same stimuli and experimental conditions as our participants, a highly-cited attention-based reinforcement model falls short of human efficiency.

Moral outrage drives the interaction of harm and culpable intent in third-party punishment decisions.

The willingness of humans to engage in third-party punishment (TPP)-a lynchpin of our society-critically depends on the interaction between the wrongdoer's intent and the harm that he caused. But what compels us to punish such individuals when we are unaffected by their harms? Inconsistent with the idealized notion that TPP decisions are based on purely cognitive reasoning, intended harmful acts elicit strong emotional reactions in third-party decision makers. While these emotional responses are now believed to be a driving force in TPP decision making, there is debate about what emotions may be motivating this behavior. Here we show that-unlike anger, contempt, and disgust-moral outrage is evoked by the integration of culpable intent and severe harm, and that the expression of moral outrage alone mediates the relationship between this integrative process and punishment decisions. Sadness had the opposite effect of dampening punishment in response to accidental harms. We take these findings to indicate that moral outrage expresses the interaction of intent and harm in driving third-party punishment behavior. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Conscious perception can be both graded and discrete.

The attentional blink (AB) paradigm has been used to address an enduring debate about the nature of conscious perception: Does the temporary impairment in conscious perception of the second (T2) of two serially presented targets result from a probabilistic all-or-none loss of information, or does T2 transition into consciousness along a continuum of perceptual fidelity? To investigate this question, we presented noisy orientation patterns as targets embedded in a rapid serial sequence of nonoriented noise distractors, and evaluated perception of T2 orientation using a continuous report paradigm. Using discrete mixture models and variable resource models, we evaluated the effects of manipulating both perceptual and central demands on the precision of T2 responses and the estimated frequency of random guessing. When perceptual competition between targets was emphasized by their sharing of a common visual feature (i.e., orientation), the attentional blink was associated with degraded precision of T2 perception. By contrast, when the task required switching between different attended features across two visually distinct targets, T2 awareness was impaired in an all-or-none manner as evidenced by significant increases in guessing responses. Both statistical and model comparison analyses indicated that loss of target information can be graded or discrete, depending on whether perceptual or higher central stages are taxed by processing demands. Our findings provide new insights into the mechanisms underlying the attentional blink and help reconcile conflicting views regarding how information can be lost from awareness. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Increase in internetwork functional connectivity in the human brain with attention capture.

Attention is often extolled for its selective neural properties. Yet, when powerfully captured by a salient unexpected event, attention can give rise to a broad cascade of systemic effects for evaluating and adaptively responding to the event. Using graph theory analysis combined with fMRI, we show here that the extensive psychophysiological and cognitive changes associated with such attention capture are related to large-scale distributed changes in the brain's functional connectivity. Novel task-irrelevant "oddball" stimuli presented to subjects during the performance of a target-search task triggered an increase in internetwork functional connectivity that degraded the brain's network modularity, thereby facilitating the integration of information. Furthermore, this phenomenon habituated with repeated oddball presentations, mirroring the behavior. These functional network connectivity changes are remarkably consistent with those previously obtained with conscious target perception, thus raising the possibility that large-scale internetwork connectivity changes triggered by attentional capture and awareness rely on common neural network dynamics.NEW & NOTEWORTHY The selective properties of attention have been extensively studied. There are some circumstances in which attention can have widespread and systemic effects, however, such as when it is captured by an unexpected, salient stimulus or event. How are such effects propagated in the human brain? Using graph theory analysis of fMRI data, we show here that salient task-irrelevant events produced a global increase in the functional integration of the brain's neural networks.

Distinct affective responses to second- and third-party norm violations.

Social norm violations provoke strong emotional reactions that often culminate in punishment of the wrongdoer. This is true not only when we are the victims of the norm violation (second-party), but also when witnessing a complete stranger being victimized (third-party). What remains unclear, however, is whether second- and third-party punishments are associated with different emotions. To address this question, here we examine how subjects respond affectively to both second- and third-party norm violations in an economic game. Our results indicate that while second- and third-parties respond to norm violations by punishing wrongdoers similarly, they report experiencing distinct emotional states as a result of the violation. Specifically, we observed a cross-over interaction between anger and moral outrage depending on the party's context: while anger was more frequently reported for second- than for third-party violations, moral outrage was more evoked by third-party than second-party violations. Disgust and sadness were the most prevalently reported emotions, but their prevalence were unaffected by party contexts. These results indicate that while responses to second- and third-party violations result in similar punishment, they are associated with the expression of distinct affective palettes. Further, our results provide additional evidence that moral outrage is a critical experience in the evaluation of third-party wrongdoings.

Functional Fractionation of the Cingulo-opercular Network: Alerting Insula and Updating Cingulate.

The anterior insula (AI) and dorsal anterior cingulate cortex (dACC) are engaged in various cognitive and affective processes. An influential account posits that the AI and dACC's ubiquitous engagements reflect their role in the transient capture of attention by salient stimuli. Using fMRI here we tested this claim and functionally dissociated these regions. In the first experiment, we compared these regions' responses to emotion-laden and emotion-neutral salient "oddball" movie events. We found that while the AI only responded transiently to the onset and offset of neutral events, its response to affective events was sustained, challenging the transient attention capture account. By contrast, dACC remained transient regardless of event type. A second experiment distinguished the information encoded by these brain regions with the presentation of behaviorally salient events that require either maintaining the current task set or updating to a different one; the AI was found to signal the presence of the behaviorally relevant events, while the dACC was associated with switching of attention settings in response to the events. We conclude that AI and dACC are involved in signaling the presence of potentially or de facto behaviorally significant events and updating internal attention settings in response to these events, respectively.

Functional dissociation of the inferior frontal junction from the dorsal attention network in top-down attentional control.

The posterior lateral prefrontal cortex-specifically, the inferior frontal junction (IFJ)-is thought to exert a key role in the control of attention. However, the precise nature of that role remains elusive. During the voluntary deployment and maintenance of visuospatial attention, the IFJ is typically coactivated with a core dorsal network consisting of the frontal eye field and superior parietal cortex. During stimulus-driven attention, IFJ instead couples with a ventrolateral network, suggesting that IFJ plays a role in attention distinct from the dorsal network. Because IFJ rapidly switches activation patterns to accommodate conditions of goal-directed and stimulus-driven attention (Asplund CL, Todd JJ, Snyder AP, Marois R. Nat Neurosci 13: 507-512, 2010), we hypothesized that IFJ's primary role is to dynamically reconfigure attention rather than to maintain attention under steady-state conditions. This hypothesis predicts that in a goal-directed visuospatial cuing paradigm IFJ would transiently deploy attention toward the cued location, whereas the dorsal attention network would maintain attentional weights during the delay between cue and target presentation. Here we tested this hypothesis with functional magnetic resonance imaging while subjects were engaged in a Posner cuing task with variable cue-target delays. Both IFJ and dorsal network regions were involved in transient processes, but sustained activity was far more evident in the dorsal network than in IFJ. These results support the account that IFJ primarily acts to shift attention whereas the dorsal network is the main locus for the maintenance of stable attentional states. NEW & NOTEWORTHY Goal-directed visuospatial attention is controlled by a dorsal fronto-parietal network and lateral prefrontal cortex. However, the relative roles of these regions in goal-directed attention are unknown. Here we present evidence for their dissociable roles in the transient reconfiguration and sustained maintenance of attentional settings: while maintenance of attentional settings is confined to the dorsal network, the configuration of these settings at the beginning of an attentional episode is a function of lateral prefrontal cortex.

A common source of attention for auditory and visual tracking.

Tasks that require tracking visual information reveal the severe limitations of our capacity to attend to multiple objects that vary in time and space. Although these limitations have been extensively characterized in the visual domain, very little is known about tracking information in other sensory domains. Does tracking auditory information exhibit characteristics similar to those of tracking visual information, and to what extent do these two tracking tasks draw on the same attention resources? We addressed these questions by asking participants to perform either single or dual tracking tasks from the same (visual-visual) or different (visual-auditory) perceptual modalities, with the difficulty of the tracking tasks being manipulated across trials. The results revealed that performing two concurrent tracking tasks, whether they were in the same or different modalities, affected tracking performance as compared to performing each task alone (concurrence costs). Moreover, increasing task difficulty also led to increased costs in both the single-task and dual-task conditions (load-dependent costs). The comparison of concurrence costs between visual-visual and visual-auditory dual-task performance revealed slightly greater interference when two visual tracking tasks were paired. Interestingly, however, increasing task difficulty led to equivalent costs for visual-visual and visual-auditory pairings. We concluded that visual and auditory tracking draw largely, though not exclusively, on common central attentional resources.

Parsing the Behavioral and Brain Mechanisms of Third-Party Punishment.

UNLABELLED: The evolved capacity for third-party punishment is considered crucial to the emergence and maintenance of elaborate human social organization and is central to the modern provision of fairness and justice within society. Although it is well established that the mental state of the offender and the severity of the harm he caused are the two primary predictors of punishment decisions, the precise cognitive and brain mechanisms by which these distinct components are evaluated and integrated into a punishment decision are poorly understood. Using fMRI, here we implement a novel experimental design to functionally dissociate the mechanisms underlying evaluation, integration, and decision that were conflated in previous studies of third-party punishment. Behaviorally, the punishment decision is primarily defined by a superadditive interaction between harm and mental state, with subjects weighing the interaction factor more than the single factors of harm and mental state. On a neural level, evaluation of harms engaged brain areas associated with affective and somatosensory processing, whereas mental state evaluation primarily recruited circuitry involved in mentalization. Harm and mental state evaluations are integrated in medial prefrontal and posterior cingulate structures, with the amygdala acting as a pivotal hub of the interaction between harm and mental state. This integrated information is used by the right dorsolateral prefrontal cortex at the time of the decision to assign an appropriate punishment through a distributed coding system. Together, these findings provide a blueprint of the brain mechanisms by which neutral third parties render punishment decisions. SIGNIFICANCE STATEMENT: Punishment undergirds large-scale cooperation and helps dispense criminal justice. Yet it is currently unknown precisely how people assess the mental states of offenders, evaluate the harms they caused, and integrate those two components into a single punishment decision. Using a new design, we isolated these three processes, identifying the distinct brain systems and activities that enable each. Additional findings suggest that the amygdala plays a crucial role in mediating the interaction of mental state and harm information, whereas the dorsolateral prefrontal cortex plays a crucial, final-stage role, both in integrating mental state and harm information and in selecting a suitable punishment amount. These findings deepen our understanding of how punishment decisions are made, which may someday help to improve them.

Central attention is serial, but midlevel and peripheral attention are parallel-A hypothesis.

In this brief review, we argue that attention operates along a hierarchy from peripheral through central mechanisms. We further argue that these mechanisms are distinguished not just by their functional roles in cognition, but also by a distinction between serial mechanisms (associated with central attention) and parallel mechanisms (associated with midlevel and peripheral attention). In particular, we suggest that peripheral attentional deployments in distinct representational systems may be maintained simultaneously with little or no interference, but that the serial nature of central attention means that even tasks that largely rely on distinct representational systems will come into conflict when central attention is demanded. We go on to review both the behavioral and neural evidence for this prediction. We conclude that even though the existing evidence mostly favors our account of serial central and parallel noncentral attention, we know of no experiment that has conclusively borne out these claims. As such, this article offers a framework of attentional mechanisms that will aid in guiding future research on this topic.

From Blame to Punishment: Disrupting Prefrontal Cortex Activity Reveals Norm Enforcement Mechanisms.

The social welfare provided by cooperation depends on the enforcement of social norms. Determining blameworthiness and assigning a deserved punishment are two cognitive cornerstones of norm enforcement. Although prior work has implicated the dorsolateral prefrontal cortex (DLPFC) in norm-based judgments, the relative contribution of this region to blameworthiness and punishment decisions remains poorly understood. Here, we used repetitive transcranial magnetic stimulation (rTMS) and fMRI to determine the specific role of DLPFC function in norm-enforcement behavior. DLPFC rTMS reduced punishment for wrongful acts without affecting blameworthiness ratings, and fMRI revealed punishment-selective DLPFC recruitment, suggesting that these two facets of norm-based decision making are neurobiologically dissociable. Finally, we show that DLPFC rTMS affects punishment decision making by altering the integration of information about culpability and harm. Together, these findings reveal a selective, causal role for DLPFC in norm enforcement: representational integration of the distinct information streams used to make punishment decisions.

Crowding in Visual Working Memory Reveals Its Spatial Resolution and the Nature of Its Representations.

Spatial resolution fundamentally limits any image representation. Although this limit has been extensively investigated for perceptual representations by assessing how neighboring flankers degrade the perception of a peripheral target with visual crowding, the corresponding limit for representations held in visual working memory (VWM) is unknown. In the present study, we evoked crowding in VWM and directly compared resolution in VWM and perception. Remarkably, the spatial resolution of VWM proved to be no worse than that of perception. However, mixture modeling of errors caused by crowding revealed the qualitatively distinct nature of these representations. Perceptual crowding errors arose from both increased imprecision in target representations and substitution of flankers for targets. By contrast, VWM crowding errors arose exclusively from substitutions, which suggests that VWM transforms analog perceptual representations into discrete items. Thus, although perception and VWM share a common resolution limit, exceeding this limit reveals distinct mechanisms for perceiving images and holding them in mind.

Breakdown of the brain's functional network modularity with awareness.

Neurobiological theories of awareness propose divergent accounts of the spatial extent of brain changes that support conscious perception. Whereas focal theories posit mostly local regional changes, global theories propose that awareness emerges from the propagation of neural signals across a broad extent of sensory and association cortex. Here we tested the scalar extent of brain changes associated with awareness using graph theoretical analysis applied to functional connectivity data acquired at ultra-high field while subjects performed a simple masked target detection task. We found that awareness of a visual target is associated with a degradation of the modularity of the brain's functional networks brought about by an increase in intermodular functional connectivity. These results provide compelling evidence that awareness is associated with truly global changes in the brain's functional connectivity.

Working memory storage is intrinsically domain specific.

A longstanding debate in working memory (WM) is whether information is maintained in a central, capacity-limited storage system or whether there are domain-specific stores for different modalities. This question is typically addressed by determining whether concurrent storage of 2 different memory arrays produces interference. Prior studies using this approach have shown at least some cost to maintaining 2 memory arrays that differed in perceptual modalities. However, it is not clear whether these WM costs resulted from competition for a central, capacity-limited store or from other potential sources of dual-task interference, such as task preparation and coordination, overlap in representational content (e.g., object vs. space based), or cognitive strategies (e.g., verbalization, chunking of the stimulus material in a higher order structure). In the present study we assess dual-task costs during the concurrent performance of a visuospatial WM task and an auditory object WM task when such sources of interference are minimized. The results show that performance of these 2 WM tasks are independent from each another, even at high WM load. Only when we introduced a common representational format (spatial information) to both WM tasks did dual-task performance begin to suffer. These results are inconsistent with the notion of a domain-independent storage system, and suggest instead that WM is constrained by multiple domain-specific stores and central executive processes. Evidently, there is nothing intrinsic about the functional architecture of the human mind that prevents it from storing 2 distinct representations in WM, as long as these representations do not overlap in any functional domain.

Corticolimbic gating of emotion-driven punishment.

Determining the appropriate punishment for a norm violation requires consideration of both the perpetrator's state of mind (for example, purposeful or blameless) and the strong emotions elicited by the harm caused by their actions. It has been hypothesized that such affective responses serve as a heuristic that determines appropriate punishment. However, an actor's mental state often trumps the effect of emotions, as unintended harms may go unpunished, regardless of their magnitude. Using fMRI, we found that emotionally graphic descriptions of harmful acts amplify punishment severity, boost amygdala activity and strengthen amygdala connectivity with lateral prefrontal regions involved in punishment decision-making. However, this was only observed when the actor's harm was intentional; when harm was unintended, a temporoparietal-medial-prefrontal circuit suppressed amygdala activity and the effect of graphic descriptions on punishment was abolished. These results reveal the brain mechanisms by which evaluation of a transgressor's mental state gates our emotional urges to punish.

The effects of stimulus-driven competition and task set on involuntary attention.

It is well established that involuntary attention­the exogenous capture of attention by salient but task-irrelevant stimuli­can strongly modulate target detection and discrimination performance. There is an ongoing debate, however, about how involuntary attention affects target performance. Some studies suggest that it results from enhanced perception of the target, whereas others indicate instead that it affects decisional stages of information processing. From a review of these studies, we hypothesized that the presence of distractors and task sets are key factors in determining the effect of involuntary attention on target perception. Consistent with this hypothesis, here we found that noninformative cues summoning involuntary attention affected perceptual identification of a target when distractors were present. This cuing effect could not be attributed to reduced target location uncertainty or decision bias. The only condition under which involuntary attention improved target perception in the absence of distractors occurred when observers did not adopt a task set to focus attention on the target location. We conclude that the perceptual effects of involuntary attention depend on distractor interference and the adoption of a task set to resolve such stimulus competition.

Functional fractionation of the stimulus-driven attention network.

A novel, salient event in the environment powerfully captures attention. This stimulus-driven attentional capture not only includes orienting of attention toward the event, but also an evaluative process to determine the behavioral significance and appropriate response to the event. Whereas a network of human brain regions composed of prefrontal and temporoparietal regions have been associated with stimulus-driven attention, the neural substrates of orienting have never been teased apart from those of evaluative processes. Here we used fMRI to measure the human brain's response to the temporally extended presentations of salient, task-irrelevant stimuli, and found a clear functional dissociation in the stimulus-driven attention network; the anterior insula and cingulate cortex showed transient orienting responses to the onsets and offsets of the stimuli, whereas the temporoparietal cortex exhibited sustained activity throughout event evaluation. The lateral prefrontal cortex was implicated in both attentional and evaluative processes, pointing to its central, integrative role in stimulus-driven attention.

The attentional blink reveals the probabilistic nature of discrete conscious perception.

Attention and awareness are two tightly coupled processes that have been the subject of the same enduring debate: Are they allocated in a discrete or in a graded fashion? Using the attentional blink paradigm and mixture-modeling analysis, we show that awareness arises at central stages of information processing in an all-or-none manner. Manipulating the temporal delay between two targets affected subjects' likelihood of consciously perceiving the second target, but did not affect the precision of its representation. Furthermore, these results held across stimulus categories and paradigms, and they were dependent on attention having been allocated to the first target. The findings distinguish the fundamental contributions of attention and awareness at central stages of visual cognition: Conscious perception emerges in a quantal manner, with attention serving to modulate the probability that representations reach awareness.

Disrupting prefrontal cortex prevents performance gains from sensory-motor training.

Humans show large and reliable performance impairments when required to make more than one simple decision simultaneously. Such multitasking costs are thought to largely reflect capacity limits in response selection (Welford, 1952; Pashler, 1984, 1994), the information processing stage at which sensory input is mapped to a motor response. Neuroimaging has implicated the left posterior lateral prefrontal cortex (pLPFC) as a key neural substrate of response selection (Dux et al., 2006, 2009; Ivanoff et al., 2009). For example, activity in left pLPFC tracks improvements in response selection efficiency typically observed following training (Dux et al., 2009). To date, however, there has been no causal evidence that pLPFC contributes directly to sensory-motor training effects, or the operations through which training occurs. Moreover, the left hemisphere lateralization of this operation remains controversial (Jiang and Kanwisher, 2003; Sigman and Dehaene, 2008; Verbruggen et al., 2010). We used anodal (excitatory), cathodal (inhibitory), and sham transcranial direct current stimulation (tDCS) to left and right pLPFC and measured participants' performance on high and low response selection load tasks after different amounts of training. Both anodal and cathodal stimulation of the left pLPFC disrupted training effects for the high load condition relative to sham. No disruption was found for the low load and right pLPFC stimulation conditions. The findings implicate the left pLPFC in both response selection and training effects. They also suggest that training improves response selection efficiency by fine-tuning activity in pLPFC relating to sensory-motor translations.