Learning attentional templates for value-based decision-making.

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring.

Cognitive flexibility, the ability to alter strategy according to changing stimulus-response-reward relationships, is critical for updating learned behavior. Attentional set-shifting, a test of cognitive flexibility, depends on the activity of prefrontal cortex (PFC). It remains unclear, however, what role PFC neurons play to support set-shifting. Using optogenetics and two-photon calcium imaging, we demonstrate that medial PFC activity does not bias sensorimotor responses during set-shifting, but rather enables set-shifting by encoding trial feedback information, a role it has been known to play in other contexts. Unexpectedly, the functional properties of PFC cells did not vary with their efferent projection targets. Instead, representations of trial feedback formed a topological gradient, with cells more strongly selective for feedback information located further from the pial surface, where afferent input from the anterior cingulate cortex was denser. These findings identify a critical role for deep PFC projection neurons in enabling set-shifting through behavioral feedback monitoring.

Hyperactivity with Disrupted Attention by Activation of an Astrocyte Synaptogenic Cue.

Hyperactivity and disturbances of attention are common behavioral disorders whose underlying cellular and neural circuit causes are not understood. We report the discovery that striatal astrocytes drive such phenotypes through a hitherto unknown synaptic mechanism. We found that striatal medium spiny neurons (MSNs) triggered astrocyte signaling via γ-aminobutyric acid B (GABAB) receptors. Selective chemogenetic activation of this pathway in striatal astrocytes in vivo resulted in acute behavioral hyperactivity and disrupted attention. Such responses also resulted in upregulation of the synaptogenic cue thrombospondin-1 (TSP1) in astrocytes, increased excitatory synapses, enhanced corticostriatal synaptic transmission, and increased MSN action potential firing in vivo. All of these changes were reversed by blocking TSP1 effects. Our data identify a form of bidirectional neuron-astrocyte communication and demonstrate that acute reactivation of a single latent astrocyte synaptogenic cue alters striatal circuits controlling behavior, revealing astrocytes and the TSP1 pathway as therapeutic targets in hyperactivity, attention deficit, and related psychiatric disorders.

The Visual Systems of Zebrafish.

The zebrafish visual system has become a paradigmatic preparation for behavioral and systems neuroscience. Around 40 types of retinal ganglion cells (RGCs) serve as matched filters for stimulus features, including light, optic flow, prey, and objects on a collision course. RGCs distribute their signals via axon collaterals to 12 retinorecipient areas in forebrain and midbrain. The major visuomotor hub, the optic tectum, harbors nine RGC input layers that combine information on multiple features. The retinotopic map in the tectum is locally adapted to visual scene statistics and visual subfield-specific behavioral demands. Tectal projections to premotor centers are topographically organized according to behavioral commands. The known connectivity in more than 20 processing streams allows us to dissect the cellular basis of elementary perceptual and cognitive functions. Visually evoked responses, such as prey capture or loom avoidance, are controlled by dedicated multistation pathways that-at least in the larva-resemble labeled lines. This architecture serves the neuronal code's purpose of driving adaptive behavior.

Cortical-subcortical interactions in goal-directed behavior.

Flexibly selecting appropriate actions in response to complex, ever-changing environments requires both cortical and subcortical regions, which are typically described as participating in a strict hierarchy. In this traditional view, highly specialized subcortical circuits allow for efficient responses to salient stimuli, at the cost of adaptability and context specificity, which are attributed to the neocortex. Their interactions are often described as the cortex providing top-down command signals for subcortical structures to implement; however, as available technologies develop, studies increasingly demonstrate that behavior is represented by brainwide activity and that even subcortical structures contain early signals of choice, suggesting that behavioral functions emerge as a result of different regions interacting as truly collaborative networks. In this review, we discuss the field's evolving understanding of how cortical and subcortical regions in placental mammals interact cooperatively, not only via top-down cortical-subcortical inputs but through bottom-up interactions, especially via the thalamus. We describe our current understanding of the circuitry of both the cortex and two exemplar subcortical structures, the superior colliculus and striatum, to identify which information is prioritized by which regions. We then describe the functional circuits these regions form with one another, and the thalamus, to create parallel loops and complex networks for brainwide information flow. Finally, we challenge the classic view that functional modules are contained within specific brain regions; instead, we propose that certain regions prioritize specific types of information over others, but the subnetworks they form, defined by their anatomical connections and functional dynamics, are the basis of true specialization.

Perceptual Inference, Learning, and Attention in a Multisensory World.

Adaptive behavior in a complex, dynamic, and multisensory world poses some of the most fundamental computational challenges for the brain, notably inference, decision-making, learning, binding, and attention. We first discuss how the brain integrates sensory signals from the same source to support perceptual inference and decision-making by weighting them according to their momentary sensory uncertainties. We then show how observers solve the binding or causal inference problem-deciding whether signals come from common causes and should hence be integrated or else be treated independently. Next, we describe the multifarious interplay between multisensory processing and attention. We argue that attentional mechanisms are crucial to compute approximate solutions to the binding problem in naturalistic environments when complex time-varying signals arise from myriad causes. Finally, we review how the brain dynamically adapts multisensory processing to a changing world across multiple timescales.

Human Representation Learning.

The central theme of this review is the dynamic interaction between information selection and learning. We pose a fundamental question about this interaction: How do we learn what features of our experiences are worth learning about? In humans, this process depends on attention and memory, two cognitive functions that together constrain representations of the world to features that are relevant for goal attainment. Recent evidence suggests that the representations shaped by attention and memory are themselves inferred from experience with each task. We review this evidence and place it in the context of work that has explicitly characterized representation learning as statistical inference. We discuss how inference can be scaled to real-world decisions by approximating beliefs based on a small number of experiences. Finally, we highlight some implications of this inference process for human decision-making in social environments.

The Neural Basis of Escape Behavior in Vertebrates.

Escape is one of the most studied animal behaviors, and there is a rich normative theory that links threat properties to evasive actions and their timing. The behavioral principles of escape are evolutionarily conserved and rely on elementary computational steps such as classifying sensory stimuli and executing appropriate movements. These are common building blocks of general adaptive behaviors. Here we consider the computational challenges required for escape behaviors to be implemented, discuss possible algorithmic solutions, and review some of the underlying neural circuits and mechanisms. We outline shared neural principles that can be implemented by evolutionarily ancient neural systems to generate escape behavior, to which cortical encephalization has been added to allow for increased sophistication and flexibility in responding to threat.

The Anatomy and Physiology of Claustrum-Cortex Interactions.

The claustrum is one of the most widely connected regions of the forebrain, yet its function has remained obscure, largely due to the experimentally challenging nature of targeting this small, thin, and elongated brain area. However, recent advances in molecular techniques have enabled the anatomy and physiology of the claustrum to be studied with the spatiotemporal and cell type-specific precision required to eventually converge on what this area does. Here we review early anatomical and electrophysiological results from cats and primates, as well as recent work in the rodent, identifying the connectivity, cell types, and physiological circuit mechanisms underlying the communication between the claustrum and the cortex. The emerging picture is one in which the rodent claustrum is closely tied to frontal/limbic regions and plays a role in processes, such as attention, that are associated with these areas.

Spillover effects of reminder nudges in complex environments.

We present an experiment on the immediate and lasting effects of reminder nudges in a complex environment. In the study, 1,542 subjects face a setting where, within a brief time frame, they have to pay attention to and perform multiple actions in a computer game. The experiment investigates i) the effect of reminders on the reminded actions and their spillovers on nonreminded actions; ii) spillovers between multiple nudges when the number of reminded actions is increased; and iii) intertemporal spillovers from having been exposed to reminders on actions after reminders are withdrawn. Our findings reveal, first, that reminders have a positive effect on the overall number of actions performed. It results from the positive direct effect on the reminded actions dominating the negative spillovers on nonreminded actions. These negative effects are notable in our setting, where reminders could potentially have positive spillovers by freezing attention or by indirectly prompting actions similar to the reminded ones. Second, we observe that reminder nudges are scalable. Increasing the number of reminded actions leads subjects to take more actions overall, albeit with diminishing returns and more pronounced negative spillover effects. Third, after reminders are withdrawn, the positive effect on reminded actions diminishes, while negative spillovers on nonreminded actions persist, thus rendering reminders ineffective in increasing the overall number of actions performed.

Demonstratives as attention tools: Evidence of mentalistic representations within language.

Linguistic communication is an intrinsically social activity that enables us to share thoughts across minds. Many complex social uses of language can be captured by domain-general representations of other minds (i.e., mentalistic representations) that externally modulate linguistic meaning through Gricean reasoning. However, here we show that representations of others' attention are embedded within language itself. Across ten languages, we show that demonstratives-basic grammatical words (e.g., "this"/"that") which are evolutionarily ancient, learned early in life, and documented in all known languages-are intrinsic attention tools. Beyond their spatial meanings, demonstratives encode both joint attention and the direction in which the listener must turn to establish it. Crucially, the frequency of the spatial and attentional uses of demonstratives varies across languages, suggesting that both spatial and mentalistic representations are part of their conventional meaning. Using computational modeling, we show that mentalistic representations of others' attention are internally encoded in demonstratives, with their effect further boosted by Gricean reasoning. Yet, speakers are largely unaware of this, incorrectly reporting that they primarily capture spatial representations. Our findings show that representations of other people's cognitive states (namely, their attention) are embedded in language and suggest that the most basic building blocks of the linguistic system crucially rely on social cognition.

The time course of feature-selective attention inside and outside the focus of spatial attention.

Research on attentional selection of stimulus features has yielded seemingly contradictory results. On the one hand, many experiments in humans and animals have observed a "global" facilitation of attended features across the entire visual field, even when spatial attention is focused on a single location. On the other hand, several event-related potential studies in humans reported that attended features are enhanced at the attended location only. The present experiment demonstrates that these conflicting results can be explained by differences in the timing of attentional allocation inside and outside the spatial focus of attention. Participants attended to fields of either red or blue randomly moving dots on either the left or right side of fixation with the task of detecting brief coherent motion targets. Recordings of steady-state visual evoked potentials elicited by the flickering stimuli allowed concurrent measurement of the time course of feature-selective attention in visual cortex on both the attended and the unattended sides. The onset of feature-selective attentional modulation on the attended side occurred around 150 ms earlier than on the unattended side. This finding that feature-selective attention is not spatially global from the outset but extends to unattended locations after a temporal delay resolves previous contradictions between studies finding global versus hierarchical selection of features and provides insight into the fundamental relationship between feature-based and location-based (spatial) attention mechanisms.

Selective social interactions and speed-induced leadership in schooling fish.

Animals moving together in groups are believed to interact among each other with effective social forces, such as attraction, repulsion, and alignment. Such forces can be inferred using "force maps," i.e., by analyzing the dependency of the acceleration of a focal individual on relevant variables. Here, we introduce a force map technique suitable for the analysis of the alignment forces experienced by individuals. After validating it using an agent-based model, we apply the force map to experimental data of schooling fish. We observe signatures of an effective alignment force with faster neighbors and an unexpected antialignment with slower neighbors. Instead of an explicit antialignment behavior, we suggest that the observed pattern is the result of a selective attention mechanism, where fish pay less attention to slower neighbors. This mechanism implies the existence of temporal leadership interactions based on relative speeds between neighbors. We present support for this hypothesis both from agent-based modeling as well as from exploring leader-follower relationships in the experimental data.

Pathway-selective optogenetics reveals the functional anatomy of top-down attentional modulation in the macaque visual cortex.

Spatial attention represents a powerful top-down influence on sensory responses in primate visual cortical areas. The frontal eye field (FEF) has emerged as a key candidate area for the source of this modulation. However, it is unclear whether the FEF exerts its effects via its direct axonal projections to visual areas or indirectly through other brain areas and whether the FEF affects both the enhancement of attended and the suppression of unattended sensory responses. We used pathway-selective optogenetics in rhesus macaques performing a spatial attention task to inhibit the direct input from the FEF to area MT, an area along the dorsal visual pathway specialized for the processing of visual motion information. Our results show that the optogenetic inhibition of the FEF input specifically reduces attentional modulation in MT by about a third without affecting the neurons' sensory response component. We find that the direct FEF-to-MT pathway contributes to both the enhanced processing of target stimuli and the suppression of distractors. The FEF, thus, selectively modulates firing rates in visual area MT, and it does so via its direct axonal projections.

Correlated variability and its attentional modulation depend on anatomical connectivity.

Visual cortical neurons show variability in their responses to repeated presentations of a stimulus and a portion of this variability is shared across neurons. Attention may enhance visual perception by reducing shared spiking variability. However, shared variability and its attentional modulation are not consistent within or across cortical areas, and depend on additional factors such as neuronal type. A critical factor that has not been tested is actual anatomical connectivity. We measured spike count correlations among pairs of simultaneously recorded neurons in the primary visual cortex (V1) for which anatomical connectivity was inferred from spiking cross-correlations. Neurons were recorded in monkeys performing a contrast-change discrimination task requiring covert shifts in visual spatial attention. Accordingly, spike count correlations were compared across trials in which attention was directed toward or away from the visual stimulus overlapping recorded neuronal receptive fields. Consistent with prior findings, attention did not significantly alter spike count correlations among random pairings of unconnected V1 neurons. However, V1 neurons connected via excitatory synapses showed a significant reduction in spike count correlations with attention. Interestingly, V1 neurons connected via inhibitory synapses demonstrated high spike count correlations overall that were not modulated by attention. Correlated variability in excitatory circuits also depended upon neuronal tuning for contrast, the task-relevant stimulus feature. These results indicate that shared variability depends on the type of connectivity in neuronal circuits. Also, attention significantly reduces shared variability in excitatory circuits, even when attention effects on randomly sampled neurons within the same area are weak.

Onscreen presence of instructors in video lectures affects learners' neural synchrony and visual attention during multimedia learning.

COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit-cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.

Accurate estimation of biological age and its application in disease prediction using a multimodal image Transformer system.

Aging in an individual refers to the temporal change, mostly decline, in the body's ability to meet physiological demands. Biological age (BA) is a biomarker of chronological aging and can be used to stratify populations to predict certain age-related chronic diseases. BA can be predicted from biomedical features such as brain MRI, retinal, or facial images, but the inherent heterogeneity in the aging process limits the usefulness of BA predicted from individual body systems. In this paper, we developed a multimodal Transformer-based architecture with cross-attention which was able to combine facial, tongue, and retinal images to estimate BA. We trained our model using facial, tongue, and retinal images from 11,223 healthy subjects and demonstrated that using a fusion of the three image modalities achieved the most accurate BA predictions. We validated our approach on a test population of 2,840 individuals with six chronic diseases and obtained significant difference between chronological age and BA (AgeDiff) than that of healthy subjects. We showed that AgeDiff has the potential to be utilized as a standalone biomarker or conjunctively alongside other known factors for risk stratification and progression prediction of chronic diseases. Our results therefore highlight the feasibility of using multimodal images to estimate and interrogate the aging process.

Cognition as a Window into Neuronal Population Space.

Understanding how cognitive processes affect the responses of sensory neurons may clarify the relationship between neuronal population activity and behavior. However, tools for analyzing neuronal activity have not kept up with technological advances in recording from large neuronal populations. Here, we describe prevalent hypotheses of how cognitive processes affect sensory neurons, driven largely by a model based on the activity of single neurons or pools of neurons as the units of computation. We then use simple simulations to expand this model to a new conceptual framework that focuses on subspaces of population activity as the relevant units of computation, uses comparisons between brain areas or to behavior to guide analyses of these subspaces, and suggests that population activity is optimized to decode the large variety of stimuli and tasks that animals encounter in natural behavior. This framework provides new ways of understanding the ever-growing quantity of recorded population activity data.

Attention mechanism models for precision medicine.

The development of deep learning models plays a crucial role in advancing precision medicine. These models enable personalized medical treatments and interventions based on the unique genetic, environmental and lifestyle factors of individual patients, and the promotion of precision medicine is achieved mainly through genomic data analysis, variant annotation and interpretation, pharmacogenomics research, biomarker discovery, disease typing, clinical decision support and disease mechanism interpretation. Extensive research has been conducted to address precision medicine challenges using attention mechanism models such as SAN, GAT and transformers. Especially, the recent popularity of ChatGPT has significantly propelled the application of this model type to a new height. Therefore, I propose a Special Issue for Briefings in Bioinformatics about the topic 'Attention Mechanism Models for Precision Medicine'. This Special Issue aims to provide a comprehensive overview and presentation of innovative researches on the application of graph attention mechanism models in precision medicine.

Deletion variants calling in third-generation sequencing data based on a dual-attention mechanism.

Deletion is a crucial type of genomic structural variation and is associated with numerous genetic diseases. The advent of third-generation sequencing technology has facilitated the analysis of complex genomic structures and the elucidation of the mechanisms underlying phenotypic changes and disease onset due to genomic variants. Importantly, it has introduced innovative perspectives for deletion variants calling. Here we propose a method named Dual Attention Structural Variation (DASV) to analyze deletion structural variations in sequencing data. DASV converts gene alignment information into images and integrates them with genomic sequencing data through a dual attention mechanism. Subsequently, it employs a multi-scale network to precisely identify deletion regions. Compared with four widely used genome structural variation calling tools: cuteSV, SVIM, Sniffles and PBSV, the results demonstrate that DASV consistently achieves a balance between precision and recall, enhancing the F1 score across various datasets. The source code is available at https://github.com/deconvolution-w/DASV.