Implicit model of other people's visual attention as an invisible, force-carrying beam projecting from the eyes.

As a part of social cognition, people automatically construct rich models of other people's vision. Here we show that when people judge the mechanical forces acting on an object, their judgments are biased by another person gazing at the object. The bias is consistent with an implicit perception that gaze adds a gentle force, pushing on the object. The bias was present even though the participants were not explicitly aware of it and claimed that they did not believe in an extramission view of vision (a common folk view of vision in which the eyes emit an invisible energy). A similar result was not obtained on control trials when participants saw a blindfolded face turned toward the object, or a face with open eyes turned away from the object. The findings suggest that people automatically and implicitly generate a model of other people's vision that uses the simplifying construct of beams coming out of the eyes. This implicit model of active gaze may be a hidden, yet fundamental, part of the rich process of social cognition, contributing to how we perceive visual agency. It may also help explain the extraordinary cultural persistence of the extramission myth of vision.

Projecting one's own spatial bias onto others during a theory-of-mind task.

Many people show a left-right bias in visual processing. We measured spatial bias in neurotypical participants using a variant of the line bisection task. In the same participants, we measured performance in a social cognition task. This theory-of-mind task measured whether each participant had a processing-speed bias toward the right of, or left of, a cartoon agent about which the participant was thinking. Crucially, the cartoon was rotated such that what was left and right with respect to the cartoon was up and down with respect to the participant. Thus, a person's own left-right bias could not align directly onto left and right with respect to the cartoon head. Performance on the two tasks was significantly correlated. People who had a natural bias toward processing their own left side of space were quicker to process how the cartoon might think about objects to the left side of its face, and likewise for a rightward bias. One possible interpretation of these results is that the act of processing one's own personal space shares some of the same underlying mechanisms as the social cognitive act of reconstructing someone else's processing of their space.

Cortical networks involved in visual awareness independent of visual attention.

It is now well established that visual attention, as measured with standard spatial attention tasks, and visual awareness, as measured by report, can be dissociated. It is possible to attend to a stimulus with no reported awareness of the stimulus. We used a behavioral paradigm in which people were aware of a stimulus in one condition and unaware of it in another condition, but the stimulus drew a similar amount of spatial attention in both conditions. The paradigm allowed us to test for brain regions active in association with awareness independent of level of attention. Participants performed the task in an MRI scanner. We looked for brain regions that were more active in the aware than the unaware trials. The largest cluster of activity was obtained in the temporoparietal junction (TPJ) bilaterally. Local independent component analysis (ICA) revealed that this activity contained three distinct, but overlapping, components: a bilateral, anterior component; a left dorsal component; and a right dorsal component. These components had brain-wide functional connectivity that partially overlapped the ventral attention network and the frontoparietal control network. In contrast, no significant activity in association with awareness was found in the banks of the intraparietal sulcus, a region connected to the dorsal attention network and traditionally associated with attention control. These results show the importance of separating awareness and attention when testing for cortical substrates. They are also consistent with a recent proposal that awareness is associated with ventral attention areas, especially in the TPJ.

Functional Connectivity Between the Temporoparietal Cortex and Cerebellum in Autism Spectrum Disorder.

The neural basis of autism spectrum disorder (ASD) is not yet understood. ASD is marked by social deficits and is strongly associated with cerebellar abnormalities. We studied the organization and cerebellar connectivity of the temporoparietal junction (TPJ), an area that plays a crucial role in social cognition. We applied localized independent component analysis to resting-state fMRI data from autistic and neurotypical adolescents to yield an unbiased parcellation of the bilateral TPJ into 11 independent components (ICs). A comparison between neurotypical and autistic adolescents showed that the organization of the TPJ was not significantly altered in ASD. Second, we used the time courses of the TPJ ICs as spatially unbiased "seeds" for a functional connectivity analysis applied to voxels within the cerebellum. We found that the cerebellum contained a fine-grained, lateralized map of the TPJ. The connectivity of the TPJ subdivisions with cerebellar zones showed one striking difference in ASD. The right dorsal TPJ showed markedly less connectivity with the left Crus II. Disturbed cerebellar input to this key region for cognition and multimodal integration may contribute to social deficits in ASD. The findings might also suggest that the right TPJ and/or left Crus II are potential targets for noninvasive brain stimulation therapies.

Attributing awareness to oneself and to others.

This study tested the possible relationship between reported visual awareness ("I see a visual stimulus in front of me") and the social attribution of awareness to someone else ("That person is aware of an object next to him"). Subjects were tested in two steps. First, in an fMRI experiment, subjects were asked to attribute states of awareness to a cartoon face. Activity associated with this task was found bilaterally within the temporoparietal junction (TPJ) among other areas. Second, the TPJ was transiently disrupted using single-pulse transcranial magnetic stimulation (TMS). When the TMS was targeted to the same cortical sites that had become active during the social attribution task, the subjects showed symptoms of visual neglect in that their detection of visual stimuli was significantly affected. In control trials, when TMS was targeted to nearby cortical sites that had not become active during the social attribution task, no significant effect on visual detection was found. These results suggest that there may be at least some partial overlap in brain mechanisms that participate in the social attribution of sensory awareness to other people and in attributing sensory awareness to oneself.

Neural Processes in the Human Temporoparietal Cortex Separated by Localized Independent Component Analysis.

The human temporoparietal junction (TPJ) is a topic of intense research. Imaging studies have identified TPJ activation in association with many higher-order functions such as theory-of-mind, episodic memory, and attention, causing debate about the distribution of different processes. One major challenge is the lack of consensus about the anatomical location and extent of the TPJ. Here, we address this problem using data-driven analysis to test the hypothesis that the bilateral TPJ can be parcellated into subregions. We applied independent component analysis (ICA) to task-free fMRI data within a local region around the bilateral TPJ, iterating the ICA at multiple model orders and in several datasets. The localized analysis allowed finer separation of processes and the use of multiple dimensionalities provided qualitative information about lateralization. We identified four subdivisions that were bilaterally symmetrical and one that was right biased. To test whether the independent components (ICs) reflected true subdivisions, we performed functional connectivity analysis using the IC coordinates as seeds. This confirmed that the subdivisions belonged to distinct networks. The right-biased IC was connected with a network often associated with attentional processing. One bilateral subdivision was connected to sensorimotor regions and another was connected to auditory regions. One subdivision that presented as distinct left- and right-biased ICs was connected to frontoparietal regions. Another subdivision that also had left- and right-biased ICs was connected to social or default mode networks. Our results show that the TPJ in both hemispheres hosts multiple neural processes with connectivity patterns consistent with well developed specialization and lateralization.

Effects of Awareness on the Control of Attention.

Previous studies show that it is possible to attend to a stimulus without awareness of it. Whether attention and awareness are independent or have a specific relationship, however, remains debated. Here, we tested three aspects of visual attention with and without awareness of the visual stimulus. Metacontrast masking rendered participants either subjectively aware or not aware of the stimulus. Attention drawn to the stimulus was measured by using the stimulus as a cue in a spatial attention task. We found that attention was drawn to the stimulus regardless of whether or not people were aware of it. However, attention changed significantly in the absence of awareness in at least three ways. First, attention to a task-relevant stimulus was less stable over time. Second, inhibition of return, the automatic suppression of attention to a task-irrelevant stimulus, was reduced. Third, attention was more driven by the luminance contrast of the stimulus. These findings add to the growing information on the behavior of attention with and without awareness. The findings are also consistent with our recently proposed account of the relationship between attention and awareness. In the attention schema theory, awareness is the internal model of attention. Just as the brain contains a body schema that models the body and helps control the body, so it contains an attention schema that helps control attention. In that theory, in the absence of awareness, the control of attention should suffer in basic ways predictable from dynamical systems theory. The present results confirm some of those predictions.

Determinantal point process attention over grid cell code supports out of distribution generalization.

Deep neural networks have made tremendous gains in emulating human-like intelligence, and have been used increasingly as ways of understanding how the brain may solve the complex computational problems on which this relies. However, these still fall short of, and therefore fail to provide insight into how the brain supports strong forms of generalization of which humans are capable. One such case is out-of-distribution (OOD) generalization - successful performance on test examples that lie outside the distribution of the training set. Here, we identify properties of processing in the brain that may contribute to this ability. We describe a two-part algorithm that draws on specific features of neural computation to achieve OOD generalization, and provide a proof of concept by evaluating performance on two challenging cognitive tasks. First we draw on the fact that the mammalian brain represents metric spaces using grid cell code (e.g., in the entorhinal cortex): abstract representations of relational structure, organized in recurring motifs that cover the representational space. Second, we propose an attentional mechanism that operates over the grid cell code using determinantal point process (DPP), that we call DPP attention (DPP-A) - a transformation that ensures maximum sparseness in the coverage of that space. We show that a loss function that combines standard task-optimized error with DPP-A can exploit the recurring motifs in the grid cell code, and can be integrated with common architectures to achieve strong OOD generalization performance on analogy and arithmetic tasks. This provides both an interpretation of how the grid cell code in the mammalian brain may contribute to generalization performance, and at the same time a potential means for improving such capabilities in artificial neural networks.

The relational bottleneck as an inductive bias for efficient abstraction.

A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.

Natural statistics support a rational account of confidence biases.

Previous work has sought to understand decision confidence as a prediction of the probability that a decision will be correct, leading to debate over whether these predictions are optimal, and whether they rely on the same decision variable as decisions themselves. This work has generally relied on idealized, low-dimensional models, necessitating strong assumptions about the representations over which confidence is computed. To address this, we used deep neural networks to develop a model of decision confidence that operates directly over high-dimensional, naturalistic stimuli. The model accounts for a number of puzzling dissociations between decisions and confidence, reveals a rational explanation of these dissociations in terms of optimization for the statistics of sensory inputs, and makes the surprising prediction that, despite these dissociations, decisions and confidence depend on a common decision variable.

Attention control and the attention schema theory of consciousness.

In the attention schema theory (AST), the brain constructs a schematic, simplified model of attention. The model is associated with three cognitive processes: a model of one's own attention contributes to the endogenous control of attention, a model of the attention of others contributes to theory of mind, and the contents of these models leads to the common human claim that we contain a non-physical consciousness or awareness inside us. Because AST is a control-engineering style theory, it can make specific predictions in complex situations. Here, over six experiments, we examined interactions between attention and awareness to test predictions of AST. Participants performed a visual task in which a cue stimulus affected their attention, as measured by their reactions to a subsequent target stimulus. The task measured both exogenous attention drawn to the cue and endogenous attention directed to a target location predicted by the cue. When participants were not aware that the cue predicted the target, both exogenous and endogenous attention effects remained. In contrast, when participants were not visually aware of the cue itself, the exogenous attention effect remained and the endogenous effect was impaired. In an additional two experiments, when participants learned an implicit shift of attention, the learning generalized from trained spatial locations to adjacent, untrained locations. Each of these findings matched predictions of AST. The results support the interpretation that attention control relies partly on an internal model that is responsible for claims of awareness.

Topographical Organization of Attentional, Social, and Memory Processes in the Human Temporoparietal Cortex.

The temporoparietal junction (TPJ) is activated in association with a large range of functions, including social cognition, episodic memory retrieval, and attentional reorienting. An ongoing debate is whether the TPJ performs an overarching, domain-general computation, or whether functions reside in domain-specific subdivisions. We scanned subjects with fMRI during five tasks known to activate the TPJ, probing social, attentional, and memory functions, and used data-driven parcellation (independent component analysis) to isolate task-related functional processes in the bilateral TPJ. We found that one dorsal component in the right TPJ, which was connected with the frontoparietal control network, was activated in all of the tasks. Other TPJ subregions were specific for attentional reorienting, oddball target detection, or social attribution of belief. The TPJ components that participated in attentional reorienting and oddball target detection appeared spatially separated, but both were connected with the ventral attention network. The TPJ component that participated in the theory-of-mind task was part of the default-mode network. Further, we found that the BOLD response in the domain-general dorsal component had a longer latency than responses in the domain-specific components, suggesting an involvement in distinct, perhaps postperceptual, computations. These findings suggest that the TPJ performs both domain-general and domain-specific computations that reside within spatially distinct functional components.

The attention schema theory: a mechanistic account of subjective awareness.

We recently proposed the attention schema theory, a novel way to explain the brain basis of subjective awareness in a mechanistic and scientifically testable manner. The theory begins with attention, the process by which signals compete for the brain's limited computing resources. This internal signal competition is partly under a bottom-up influence and partly under top-down control. We propose that the top-down control of attention is improved when the brain has access to a simplified model of attention itself. The brain therefore constructs a schematic model of the process of attention, the 'attention schema,' in much the same way that it constructs a schematic model of the body, the 'body schema.' The content of this internal model leads a brain to conclude that it has a subjective experience. One advantage of this theory is that it explains how awareness and attention can sometimes become dissociated; the brain's internal models are never perfect, and sometimes a model becomes dissociated from the object being modeled. A second advantage of this theory is that it explains how we can be aware of both internal and external events. The brain can apply attention to many types of information including external sensory information and internal information about emotions and cognitive states. If awareness is a model of attention, then this model should pertain to the same domains of information to which attention pertains. A third advantage of this theory is that it provides testable predictions. If awareness is the internal model of attention, used to help control attention, then without awareness, attention should still be possible but should suffer deficits in control. In this article, we review the existing literature on the relationship between attention and awareness, and suggest that at least some of the predictions of the theory are borne out by the evidence.