Using the Veil of Ignorance to align AI systems with principles of justice.

The philosopher John Rawls proposed the Veil of Ignorance (VoI) as a thought experiment to identify fair principles for governing a society. Here, we apply the VoI to an important governance domain: artificial intelligence (AI). In five incentive-compatible studies (N = 2, 508), including two preregistered protocols, participants choose principles to govern an Artificial Intelligence (AI) assistant from behind the veil: that is, without knowledge of their own relative position in the group. Compared to participants who have this information, we find a consistent preference for a principle that instructs the AI assistant to prioritize the worst-off. Neither risk attitudes nor political preferences adequately explain these choices. Instead, they appear to be driven by elevated concerns about fairness: Without prompting, participants who reason behind the VoI more frequently explain their choice in terms of fairness, compared to those in the Control condition. Moreover, we find initial support for the ability of the VoI to elicit more robust preferences: In the studies presented here, the VoI increases the likelihood of participants continuing to endorse their initial choice in a subsequent round where they know how they will be affected by the AI intervention and have a self-interested motivation to change their mind. These results emerge in both a descriptive and an immersive game. Our findings suggest that the VoI may be a suitable mechanism for selecting distributive principles to govern AI.

What can classic Atari video games tell us about the human brain?

In this issue of Neuron, Cross et. al (2021) use a deep reinforcement learning algorithm to understand human neural activation evoked by playing different video games. The results shed light on the neural principles underlying reward-guided decisions in naturalistic domains.

Big-Loop Recurrence within the Hippocampal System Supports Integration of Information across Episodes.

Recent evidence challenges the widely held view that the hippocampus is specialized for episodic memory, by demonstrating that it also underpins the integration of information across experiences. Contemporary computational theories propose that these two contrasting functions can be accomplished by big-loop recurrence, whereby the output of the system is recirculated back into the hippocampus. We use ultra-high-resolution fMRI to provide support for this hypothesis, by showing that retrieved information is presented as a new input on the superficial entorhinal cortex-driven by functional connectivity between the deep and superficial entorhinal layers. Further, the magnitude of this laminar connectivity correlated with inferential performance, demonstrating its importance for behavior. Our findings offer a novel perspective on information processing within the hippocampus and support a unifying framework in which the hippocampus captures higher-order structure across experiences, by creating a dynamic memory space from separate episodic codes for individual experiences.

Nonmonotonic recruitment of ventromedial prefrontal cortex during remote memory recall.

Systems-level consolidation refers to the time-dependent reorganisation of memory traces in the neocortex, a process in which the ventromedial prefrontal cortex (vmPFC) has been implicated. Capturing the precise temporal evolution of this crucial process in humans has long proved elusive. Here, we used multivariate methods and a longitudinal functional magnetic resonance imaging (fMRI) design to detect, with high granularity, the extent to which autobiographical memories of different ages were represented in vmPFC and how this changed over time. We observed an unexpected time course of vmPFC recruitment during retrieval, rising and falling around an initial peak of 8-12 months, before reengaging for older 2- and 5-year-old memories. This pattern was replicated in 2 independent sets of memories. Moreover, it was further replicated in a follow-up study 8 months later with the same participants and memories, for which the individual memory representations had undergone their hypothesised strengthening or weakening over time. We conclude that the temporal engagement of vmPFC in memory retrieval seems to be nonmonotonic, revealing a complex relationship between systems-level consolidation and prefrontal cortex recruitment that is unaccounted for by current theories.

Vector-based navigation using grid-like representations in artificial agents.

Deep neural networks have achieved impressive successes in fields ranging from object recognition to complex games such as Go1,2. Navigation, however, remains a substantial challenge for artificial agents, with deep neural networks trained by reinforcement learning3-5 failing to rival the proficiency of mammalian spatial behaviour, which is underpinned by grid cells in the entorhinal cortex 6 . Grid cells are thought to provide a multi-scale periodic representation that functions as a metric for coding space7,8 and is critical for integrating self-motion (path integration)6,7,9 and planning direct trajectories to goals (vector-based navigation)7,10,11. Here we set out to leverage the computational functions of grid cells to develop a deep reinforcement learning agent with mammal-like navigational abilities. We first trained a recurrent network to perform path integration, leading to the emergence of representations resembling grid cells, as well as other entorhinal cell types 12 . We then showed that this representation provided an effective basis for an agent to locate goals in challenging, unfamiliar, and changeable environments-optimizing the primary objective of navigation through deep reinforcement learning. The performance of agents endowed with grid-like representations surpassed that of an expert human and comparison agents, with the metric quantities necessary for vector-based navigation derived from grid-like units within the network. Furthermore, grid-like representations enabled agents to conduct shortcut behaviours reminiscent of those performed by mammals. Our findings show that emergent grid-like representations furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for vector-based navigation7,10,11, demonstrating that the latter can be combined with path-based strategies to support navigation in challenging environments.

Hippocampal damage and memory impairment in congenital cyanotic heart disease.

Neonatal hypoxia can lead to hippocampal atrophy, which can lead, in turn, to memory impairment. To test the generalizability of this causal sequence, we examined a cohort of 41 children aged 8-16, who, having received the arterial switch operation to correct for transposition of the great arteries, had sustained significant neonatal cyanosis but were otherwise neurodevelopmentally normal. As predicted, the cohort had significant bilateral reduction of hippocampal volumes relative to the volumes of 64 normal controls. They also had significant, yet selective, impairment of episodic memory as measured by standard tests of memory, despite relatively normal levels of intelligence, academic attainment, and verbal fluency. Across the cohort, degree of memory impairment was correlated with degree of hippocampal atrophy suggesting that even as early as neonatal life no other structure can fully compensate for hippocampal injury and its special role in serving episodic long term memory. © 2017 Wiley Periodicals, Inc.

Semantic representations in the temporal pole predict false memories.

Recent advances in neuroscience have given us unprecedented insight into the neural mechanisms of false memory, showing that artificial memories can be inserted into the memory cells of the hippocampus in a way that is indistinguishable from true memories. However, this alone is not enough to explain how false memories can arise naturally in the course of our daily lives. Cognitive psychology has demonstrated that many instances of false memory, both in the laboratory and the real world, can be attributed to semantic interference. Whereas previous studies have found that a diverse set of regions show some involvement in semantic false memory, none have revealed the nature of the semantic representations underpinning the phenomenon. Here we use fMRI with representational similarity analysis to search for a neural code consistent with semantic false memory. We find clear evidence that false memories emerge from a similarity-based neural code in the temporal pole, a region that has been called the "semantic hub" of the brain. We further show that each individual has a partially unique semantic code within the temporal pole, and this unique code can predict idiosyncratic patterns of memory errors. Finally, we show that the same neural code can also predict variation in true-memory performance, consistent with an adaptive perspective on false memory. Taken together, our findings reveal the underlying structure of neural representations of semantic knowledge, and how this semantic structure can both enhance and distort our memories.

Sexual Dimorphism in White Matter Developmental Trajectories Using Tract-Based Spatial Statistics.

Increasing evidence is emerging for sexual dimorphism in the trajectory of white matter development in children assessed using volumetric magnetic resonance imaging (MRI) and more recently diffusion MRI. Recent studies using diffusion MRI have examined cohorts with a wide age range (typically between 5 and 30 years) showing focal regions of differential diffusivity and fractional anisotropy (FA) and have implicated puberty as a possible contributory factor. To further investigate possible dimorphic trajectories in a young cohort, presumably closer to the expected onset of puberty, we used tract-based spatial statistics to investigate diffusion metrics. The cohort consisted of 23 males and 30 females between the ages of 8 and 16 years. Differences in diffusion metrics were corrected for age, total brain volume, and full scale IQ. In contrast to previous studies showing focal differences between males and females, widespread sexually dimorphic trajectories in structural white matter development were observed. These differences were characterized by more advanced development in females compared to males indicated by lower mean diffusivity, radial and axial diffusivity, and higher FA in females. This difference appeared to be larger at lower ages (8-9 years) with diffusion measures from males and females tending to converge between 10 and 14 years of age. Males showed a steeper slope for age-diffusion metric correlations compared to females, who either did not correlate with age or correlated in fewer regions. Further studies are now warranted to determine the role of hormones on the observed differences, particularly in 8-9-year-old children.

Chronologically organized structure in autobiographical memory search.

Each of us has a rich set of autobiographical memories that provides us with a coherent story of our lives. These memories are known to be highly structured both thematically and temporally. However, it is not known how we naturally tend to explore the mental timeline of our memories. Here we developed a novel cued retrieval paradigm in order to investigate the temporal element of memory search. We found that, when asked to search for memories in the days immediately surrounding a salient cued event, participants displayed a marked set of temporal biases in their search patterns. Specifically, participants first tended to jump back in time and retrieve memories from the day prior to the cued event. Following this they then transitioned forward in time, and retrieved memories from the day after the cued event. This pattern of results replicated in a second experiment with a much larger group of participants, and a different method of cueing the memories. We argue that this set of temporal biases is consistent with memory search conforming to a temporally ordered narrative structure.

A goal direction signal in the human entorhinal/subicular region.

Navigating to a safe place, such as a home or nest, is a fundamental behavior for all complex animals. Determining the direction to such goals is a crucial first step in navigation. Surprisingly, little is known about how or where in the brain this "goal direction signal" is represented. In mammals, "head-direction cells" are thought to support this process, but despite 30 years of research, no evidence for a goal direction representation has been reported. Here, we used fMRI to record neural activity while participants made goal direction judgments based on a previously learned virtual environment. We applied multivoxel pattern analysis to these data and found that the human entorhinal/subicular region contains a neural representation of intended goal direction. Furthermore, the neural pattern expressed for a given goal direction matched the pattern expressed when simply facing that same direction. This suggests the existence of a shared neural representation of both goal and facing direction. We argue that this reflects a mechanism based on head-direction populations that simulate future goal directions during route planning. Our data further revealed that the strength of direction information predicts performance. Finally, we found a dissociation between this geocentric information in the entorhinal/subicular region and egocentric direction information in the precuneus.

Optic radiation structure and anatomy in the normally developing brain determined using diffusion MRI and tractography.

The optic radiation (OR) is a component of the visual system known to be myelin mature very early in life. Diffusion tensor imaging (DTI) and its unique ability to reconstruct the OR in vivo were used to study structural maturation through analysis of DTI metrics in a cohort of 90 children aged 5-18 years. As the OR is at risk of damage during epilepsy surgery, we measured its position relative to characteristic anatomical landmarks. Anatomical distances, DTI metrics and volume of the OR were investigated for age, gender and hemisphere effects. We observed changes in DTI metrics with age comparable to known trajectories in other white matter tracts. Left lateralization of DTI metrics was observed that showed a gender effect in lateralization. Sexual dimorphism of DTI metrics in the right hemisphere was also found. With respect to OR dimensions, volume was shown to be right lateralised and sexual dimorphism demonstrated for the extent of the left OR. The anatomical results presented for the OR have potentially important applications for neurosurgical planning.

Sparse encoding of automatic visual association in hippocampal networks.

Intelligent action entails exploiting predictions about associations between elements of ones environment. The hippocampus and mediotemporal cortex are endowed with the network topology, physiology, and neurochemistry to automatically and sparsely code sensori-cognitive associations that can be reconstructed from single or partial inputs. Whilst acquiring fMRI data and performing an attentional task, participants were incidentally presented with a sequence of cartoon images. By assigning subjects a post-scan free-association task on the same images we assayed the density of associations triggered by these stimuli. Using multivariate Bayesian decoding, we show that human hippocampal and temporal neocortical structures host sparse associative representations that are automatically triggered by visual input. Furthermore, as predicted theoretically, there was a significant increase in sparsity in the Cornu Ammonis subfields, relative to the entorhinal cortex. Remarkably, the sparsity of CA encoding correlated significantly with associative memory performance over subjects; elsewhere within the temporal lobe, entorhinal, parahippocampal, perirhinal and fusiform cortices showed the highest model evidence for the sparse encoding of associative density. In the absence of reportability or attentional confounds, this charts a distribution of visual associative representations within hippocampal populations and their temporal lobe afferent fields, and demonstrates the viability of retrospective associative sampling techniques for assessing the form of reflexive associative encoding.

CA3 size predicts the precision of memory recall.

There is enduring interest in why some of us have clearer memories than others, given the substantial individual variation that exists in retrieval ability and the precision with which we can differentiate past experiences. Here we report novel evidence showing that variation in the size of human hippocampal subfield CA3 predicted the amount of neural interference between episodic memories within CA3, which in turn predicted how much retrieval confusion occurred between past memories. This effect was not apparent in other hippocampal subfields. This shows that subtle individual differences in subjective mnemonic experience can be accurately gauged from measurable variations in the anatomy and neural coding of hippocampal region CA3. Moreover, this mechanism may be relevant for understanding memory muddles in aging and pathological states.

Representations of recent and remote autobiographical memories in hippocampal subfields.

The hippocampus has long been implicated in supporting autobiographical memories, but little is known about how they are instantiated in hippocampal subfields. Using high-resolution functional magnetic resonance imaging (fMRI) combined with multivoxel pattern analysis we found that it was possible to detect representations of specific autobiographical memories in individual hippocampal subfields. Moreover, while subfields in the anterior hippocampus contained information about both recent (2 weeks old) and remote (10 years old) autobiographical memories, posterior CA3 and DG only contained information about the remote memories. Thus, the hippocampal subfields are differentially involved in the representation of recent and remote autobiographical memories during vivid recall.

Assessing hippocampal functional reserve in temporal lobe epilepsy: a multi-voxel pattern analysis of fMRI data.

Assessing the functional reserve of key memory structures in the medial temporal lobes (MTL) of pre-surgical patients with intractable temporal lobe epilepsy (TLE) remains a challenge. Conventional functional MRI (fMRI) memory paradigms have yet to fully convince of their ability to confidently assess the risk of a post-surgical amnesia. An alternative fMRI analysis method, multi-voxel pattern analysis (MVPA), focuses on the patterns of activity across voxels in specific brain regions that are associated with individual memory traces. This method makes it possible to investigate whether the hippocampus and related structures contralateral to any proposed surgery are capable of laying down and representing specific memories. Here we used MVPA-fMRI to assess the functional integrity of the hippocampi and MTL in patients with long-standing medically refractory TLE associated with unilateral hippocampal sclerosis (HS). Patients were exposed to movie clips of everyday events prior to scanning, which they subsequently recalled during high-resolution fMRI. MTL structures were delineated and pattern classifiers were trained to learn the patterns of brain activity across voxels associated with each memory. Predictable patterns of activity across voxels associated with specific memories could be detected in MTL structures, including the hippocampus, on the side contralateral to the HS, indicating their functional viability. By contrast, no discernible memory representations were apparent in the sclerotic hippocampus, but adjacent MTL regions contained detectable information about the memories. These findings suggest that MVPA in fMRI memory studies of TLE can indicate hippocampal functional reserve and may be useful to predict the effects of hippocampal resection in individual patients.

The hippocampus extrapolates beyond the view in scenes: an fMRI study of boundary extension.

Boundary extension (BE) is a pervasive phenomenon whereby people remember seeing more of a scene than was present in the physical input, because they extrapolate beyond the borders of the original stimulus. This automatic embedding of a scene into a wider context supports our experience of a continuous and coherent world, and is therefore highly adaptive. BE, whilst occurring rapidly, is nevertheless thought to comprise two stages. The first involves the active extrapolation of the scene beyond its physical boundaries, and is constructive in nature. The second phase occurs at retrieval, where the initial extrapolation beyond the original scene borders is revealed by a subsequent memory error. The brain regions associated with the initial, and crucial, extrapolation of a scene beyond the view have never been investigated. Here, using functional MRI (fMRI) and a classic BE paradigm, we found that this extrapolation of scenes occurred rapidly around the time a scene was first viewed, and was associated with engagement of the hippocampus (HC) and parahippocampal cortex (PHC). Using connectivity analyses we determined that the HC in particular seemed to drive the BE effect, exerting top-down influence on PHC and indeed as far back down the processing stream as early visual cortex (VC). These cortical regions subsequently displayed activity profiles that tracked the trial-by-trial subjective perception of the scenes, rather than physical reality, thereby reflecting the behavioural expression of the BE error. Together our results show that the HC is involved in the active extrapolation of scenes beyond their physical borders. This information is then automatically and rapidly channelled through the scene processing hierarchy as far back as early VC. This suggests that the anticipation and construction of scenes is a pervasive and important aspect of our online perception, with the HC playing a central role.

Detecting representations of recent and remote autobiographical memories in vmPFC and hippocampus.

How autobiographical memories are represented in the human brain and whether this changes with time are questions central to memory neuroscience. Two regions in particular have been consistently implicated, the ventromedial prefrontal cortex (vmPFC) and the hippocampus, although their precise contributions are still contested. The key question in this debate, when reduced to its simplest form, concerns where information about specific autobiographical memories is located. Here, we availed ourselves of the opportunity afforded by multivoxel pattern analysis to provide an alternative to conventional neuropsychological and fMRI approaches, by detecting representations of individual autobiographical memories in patterns of fMRI activity. We examined whether information about specific recent (two weeks old) and remote (10 years old) autobiographical memories was represented in vmPFC and hippocampus, and other medial temporal and neocortical regions. vmPFC contained information about recent and remote autobiographical memories, although remote memories were more readily detected there, indicating that consolidation or a change of some kind had occurred. Information about both types of memory was also present in the hippocampus, suggesting it plays a role in the retrieval of vivid autobiographical memories regardless of remoteness. Interestingly, we also found that while recent and remote memories were both represented within anterior and posterior hippocampus, the latter nevertheless contained more information about remote memories. Thus, like vmPFC, the hippocampus too respected the distinction between recent and remote memories. Overall, these findings clarify and extend our view of vmPFC and hippocampus while also informing systems-level consolidation and providing clear targets for future studies.

Multi-voxel pattern analysis in human hippocampal subfields.

A complete understanding of the hippocampus depends on elucidating the representations and computations that exist in its anatomically distinct subfields. High-resolution structural and functional MRI scanning is starting to permit insights into hippocampal subfields in humans. In parallel, such scanning has facilitated the use of multi-voxel pattern analysis (MVPA) to examine information present in the distributed pattern of activity across voxels. The aim of this study was to combine these two relatively new innovations and deploy MVPA in the hippocampal subfields. Delineating subregions of the human hippocampus, a prerequisite for our study, remains a significant challenge, with extant methods often only examining part of the hippocampus, or being unable to differentiate CA3 and dentate gyrus (DG). We therefore devised a new high-resolution anatomical scanning and subfield segmentation protocol that allowed us to overcome these issues, and separately identify CA1, CA3, DG, and subiculum (SUB) across the whole hippocampus using a standard 3T MRI scanner. We then used MVPA to examine fMRI data associated with a decision-making paradigm involving highly similar scenes that had relevance for the computations that occur in hippocampal subfields. Intra- and inter-rater scores for subfield identification using our procedure confirmed its reliability. Moreover, we found that decoding of information within hippocampal subfields was possible using MVPA, with findings that included differential effects for CA3 and DG. We suggest that MVPA in human hippocampal subfields may open up new opportunities to examine how different types of information are represented and processed at this fundamental level.

Decoding information in the human hippocampus: a user's guide.

Multi-voxel pattern analysis (MVPA), or 'decoding', of fMRI activity has gained popularity in the neuroimaging community in recent years. MVPA differs from standard fMRI analyses by focusing on whether information relating to specific stimuli is encoded in patterns of activity across multiple voxels. If a stimulus can be predicted, or decoded, solely from the pattern of fMRI activity, it must mean there is information about that stimulus represented in the brain region where the pattern across voxels was identified. This ability to examine the representation of information relating to specific stimuli (e.g., memories) in particular brain areas makes MVPA an especially suitable method for investigating memory representations in brain structures such as the hippocampus. This approach could open up new opportunities to examine hippocampal representations in terms of their content, and how they might change over time, with aging, and pathology. Here we consider published MVPA studies that specifically focused on the hippocampus, and use them to illustrate the kinds of novel questions that can be addressed using MVPA. We then discuss some of the conceptual and methodological challenges that can arise when implementing MVPA in this context. Overall, we hope to highlight the potential utility of MVPA, when appropriately deployed, and provide some initial guidance to those considering MVPA as a means to investigate the hippocampus.