A self-supervised deep neural network for image completion resembles early visual cortex fMRI activity patterns for occluded scenes.

The promise of artificial intelligence in understanding biological vision relies on the comparison of computational models with brain data with the goal of capturing functional principles of visual information processing. Convolutional neural networks (CNN) have successfully matched the transformations in hierarchical processing occurring along the brain's feedforward visual pathway, extending into ventral temporal cortex. However, we are still to learn if CNNs can successfully describe feedback processes in early visual cortex. Here, we investigated similarities between human early visual cortex and a CNN with encoder/decoder architecture, trained with self-supervised learning to fill occlusions and reconstruct an unseen image. Using representational similarity analysis (RSA), we compared 3T functional magnetic resonance imaging (fMRI) data from a nonstimulated patch of early visual cortex in human participants viewing partially occluded images, with the different CNN layer activations from the same images. Results show that our self-supervised image-completion network outperforms a classical object-recognition supervised network (VGG16) in terms of similarity to fMRI data. This work provides additional evidence that optimal models of the visual system might come from less feedforward architectures trained with less supervision. We also find that CNN decoder pathway activations are more similar to brain processing compared to encoder activations, suggesting an integration of mid- and low/middle-level features in early visual cortex. Challenging an artificial intelligence model to learn natural image representations via self-supervised learning and comparing them with brain data can help us to constrain our understanding of information processing, such as neuronal predictive coding.

Scene Representations Conveyed by Cortical Feedback to Early Visual Cortex Can Be Described by Line Drawings.

Human behavior is dependent on the ability of neuronal circuits to predict the outside world. Neuronal circuits in early visual areas make these predictions based on internal models that are delivered via non-feedforward connections. Despite our extensive knowledge of the feedforward sensory features that drive cortical neurons, we have a limited grasp on the structure of the brain's internal models. Progress in neuroscience therefore depends on our ability to replicate the models that the brain creates internally. Here we record human fMRI data while presenting partially occluded visual scenes. Visual occlusion allows us to experimentally control sensory input to subregions of visual cortex while internal models continue to influence activity in these regions. Because the observed activity is dependent on internal models, but not on sensory input, we have the opportunity to map visual features conveyed by the brain's internal models. Our results show that activity related to internal models in early visual cortex are more related to scene-specific features than to categorical or depth features. We further demonstrate that behavioral line drawings provide a good description of internal model structure representing scene-specific features. These findings extend our understanding of internal models, showing that line drawings provide a window into our brains' internal models of vision.SIGNIFICANCE STATEMENT We find that fMRI activity patterns corresponding to occluded visual information in early visual cortex fill in scene-specific features. Line drawings of the missing scene information correlate with our recorded activity patterns, and thus to internal models. Despite our extensive knowledge of the sensory features that drive cortical neurons, we have a limited grasp on the structure of our brains' internal models. These results therefore constitute an advance to the field of neuroscience by extending our knowledge about the models that our brains construct to efficiently represent and predict the world. Moreover, they link a behavioral measure to these internal models, which play an active role in many components of human behavior, including visual predictions, action planning, and decision making.

Multi-site harmonization of 7 tesla MRI neuroimaging protocols.

Increasing numbers of 7 T (7 T) magnetic resonance imaging (MRI) scanners are in research and clinical use. 7 T MRI can increase the scanning speed, spatial resolution and contrast-to-noise-ratio of many neuroimaging protocols, but technical challenges in implementation have been addressed in a variety of ways across sites. In order to facilitate multi-centre studies and ensure consistency of findings across sites, it is desirable that 7 T MRI sites implement common high-quality neuroimaging protocols that can accommodate different scanner models and software versions. With the installation of several new 7 T MRI scanners in the United Kingdom, the UK7T Network was established with an aim to create a set of harmonized structural and functional neuroimaging sequences and protocols. The Network currently includes five sites, which use three different scanner platforms, provided by two different vendors. Here we describe the harmonization of functional and anatomical imaging protocols across the three different scanner models, detailing the necessary changes to pulse sequences and reconstruction methods. The harmonized sequences are fully described, along with implementation details. Example datasets acquired from the same subject on all Network scanners are made available. Based on these data, an evaluation of the harmonization is provided. In addition, the implementation and validation of a common system calibration process is described.

Multi-centre, multi-vendor reproducibility of 7T QSM and R2* in the human brain: Results from the UK7T study.

INTRODUCTION: We present the reliability of ultra-high field T2* MRI at 7T, as part of the UK7T Network's "Travelling Heads" study. T2*-weighted MRI images can be processed to produce quantitative susceptibility maps (QSM) and R2* maps. These reflect iron and myelin concentrations, which are altered in many pathophysiological processes. The relaxation parameters of human brain tissue are such that R2* mapping and QSM show particularly strong gains in contrast-to-noise ratio at ultra-high field (7T) vs clinical field strengths (1.5-3T). We aimed to determine the inter-subject and inter-site reproducibility of QSM and R2* mapping at 7T, in readiness for future multi-site clinical studies. METHODS: Ten healthy volunteers were scanned with harmonised single- and multi-echo T2*-weighted gradient echo pulse sequences. Participants were scanned five times at each "home" site and once at each of four other sites. The five sites had 1× Philips, 2× Siemens Magnetom, and 2× Siemens Terra scanners. QSM and R2* maps were computed with the Multi-Scale Dipole Inversion (MSDI) algorithm (https://github.com/fil-physics/Publication-Code). Results were assessed in relevant subcortical and cortical regions of interest (ROIs) defined manually or by the MNI152 standard space. RESULTS AND DISCUSSION: Mean susceptibility (χ) and R2* values agreed broadly with literature values in all ROIs. The inter-site within-subject standard deviation was 0.001-0.005 ppm (χ) and 0.0005-0.001 ms-1 (R2*). For χ this is 2.1-4.8 fold better than 3T reports, and 1.1-3.4 fold better for R2*. The median ICC from within- and cross-site R2* data was 0.98 and 0.91, respectively. Multi-echo QSM had greater variability vs single-echo QSM especially in areas with large B0 inhomogeneity such as the inferior frontal cortex. Across sites, R2* values were more consistent than QSM in subcortical structures due to differences in B0-shimming. On a between-subject level, our measured χ and R2* cross-site variance is comparable to within-site variance in the literature, suggesting that it is reasonable to pool data across sites using our harmonised protocol. CONCLUSION: The harmonized UK7T protocol and pipeline delivers on average a 3-fold improvement in the coefficient of reproducibility for QSM and R2* at 7T compared to previous reports of multi-site reproducibility at 3T. These protocols are ready for use in multi-site clinical studies at 7T.

Risk-taking behavior: dopamine D2/D3 receptors, feedback, and frontolimbic activity.

Decision-making involves frontolimbic and dopaminergic brain regions, but how prior choice outcomes, dopamine neurotransmission, and frontostriatal activity are integrated to affect choices is unclear. We tested 60 healthy volunteers using the Balloon Analogue Risk Task (BART) during functional magnetic resonance imaging. In the BART, participants can pump virtual balloons to increase potential monetary reward or cash out to receive accumulated reward; each pump presents greater risk and potential reward (represented by the pump number). In a separate session, we measured striatal D2/D3 dopamine receptor binding potential (BPND) with positron emission tomography in 13 of the participants. Losses were followed by fewer risky choices than wins; and during risk-taking after loss, amygdala and hippocampal activation exhibited greater modulation by pump number than after a cash-out event. Striatal D2/D3 BPND was positively related to the modulation of ventral striatal activation when participants decided to cash out and negatively to the number of pumps in the subsequent trial; but negatively related to the modulation of prefrontal cortical activation by pump number when participants took risk, and to overall earnings. These findings provide in vivo evidence for a potential mechanism by which dopaminergic neurotransmission may modulate risk-taking behavior through an interactive system of frontal and striatal activity.

Striatal dopamine D2/D3 receptors mediate response inhibition and related activity in frontostriatal neural circuitry in humans.

Impulsive behavior is thought to reflect a traitlike characteristic that can have broad consequences for an individual's success and well-being, but its neurobiological basis remains elusive. Although striatal dopamine D2-like receptors have been linked with impulsive behavior and behavioral inhibition in rodents, a role for D2-like receptor function in frontostriatal circuits mediating inhibitory control in humans has not been shown. We investigated this role in a study of healthy research participants who underwent positron emission tomography with the D2/D3 dopamine receptor ligand [¹8F]fallypride and BOLD fMRI while they performed the Stop-signal Task, a test of response inhibition. Striatal dopamine D2/D3 receptor availability was negatively correlated with speed of response inhibition (stop-signal reaction time) and positively correlated with inhibition-related fMRI activation in frontostriatal neural circuitry. Correlations involving D2/D3 receptor availability were strongest in the dorsal regions (caudate and putamen) of the striatum, consistent with findings of animal studies relating dopamine receptors and response inhibition. The results suggest that striatal D2-like receptor function in humans plays a major role in the neural circuitry that mediates behavioral control, an ability that is essential for adaptive responding and is compromised in a variety of common neuropsychiatric disorders.

The simplified reference tissue model with 18F-fallypride positron emission tomography: choice of reference region.

The development of high-affinity radiotracers for positron emission tomography (PET) has allowed for quantification of dopamine receptors in extrastriatal and striatal regions of the brain. As these new radiotracers have distinctly different kinetic properties than their predecessors, it is important to examine the suitability of kinetic models to represent their uptake, distribution, and in vivo washout. Using the simplified reference tissue model, we investigated the influence of reference region choice on the striatal binding potential of 18F-fallypride, a high-affinity dopamine D2/D3 receptor ligand. We compared the use of the visual cortex and a white matter region (superior longitudinal fasciculus) to the cerebellum, a commonly used reference tissue, in a PET-fallypride study of healthy and methamphetamine-dependent subjects. Compared to the cerebellum, use of the visual cortex produced significantly greater sample variance in binding potential relative to nondisplaceable uptake (BP(ND)). Use of the white matter region was associated with BP(ND) values and sample variance similar to those obtained with the cerebellum and a larger effect size for the group differences in striatal BP(ND) between healthy and methamphetamine-dependent subjects. Our results do not support the use of the visual cortex as a reference region in 18F-fallypride studies and suggest that white matter may be a reasonable alternative to the cerebellum as it displays similar statistical and kinetic properties.