Modular-integrative modeling: a new framework for building brain models that blend biological realism and functional performance.

This Perspective presents the Modular-Integrative Modeling approach, a novel framework in neuroscience for developing brain models that blend biological realism with functional performance to provide a holistic view on brain function in interaction with the body and environment.

Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling.

Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.

An integrative, multiscale view on neural theories of consciousness.

How is conscious experience related to material brain processes? A variety of theories aiming to answer this age-old question have emerged from the recent surge in consciousness research, and some are now hotly debated. Although most researchers have so far focused on the development and validation of their preferred theory in relative isolation, this article, written by a group of scientists representing different theories, takes an alternative approach. Noting that various theories often try to explain different aspects or mechanistic levels of consciousness, we argue that the theories do not necessarily contradict each other. Instead, several of them may converge on fundamental neuronal mechanisms and be partly compatible and complementary, so that multiple theories can simultaneously contribute to our understanding. Here, we consider unifying, integration-oriented approaches that have so far been largely neglected, seeking to combine valuable elements from various theories.

Characterisation of laminar and vascular spatiotemporal dynamics of CBV and BOLD signals using VASO and ME-GRE at 7T in humans.

Interpretation of cortical laminar functional magnetic resonance imaging (fMRI) activity requires detailed knowledge of the spatiotemporal haemodynamic response across vascular compartments due to the well-known vascular biases (e.g. the draining veins). Further complications arise from the spatiotemporal hemodynamic response that differs depending on the duration of stimulation. This information is crucial for future studies using depth-dependent cerebral blood volume (CBV) measurements, which promise higher specificity for the cortical microvasculature than the blood oxygenation level dependent (BOLD) contrast. To date, direct information about CBV dynamics with respect to stimulus duration, cortical depth and vasculature is missing in humans. Therefore, we characterized the cortical depth-dependent CBV-haemodynamic responses across a wide set of stimulus durations with 0.9 mm isotropic spatial and 0.785 seconds effective temporal resolution in humans using slice-selective slab-inversion vascular space occupancy (SS-SI VASO). Additionally, we investigated signal contributions from macrovascular compartments using fine-scale vascular information from multi-echo gradient-echo (ME-GRE) data at 0.35 mm isotropic resolution. In total, this resulted in >7.5h of scanning per participant (n=5). We have three major findings: (I) While we could demonstrate that 1 second stimulation is viable using VASO, more than 12 seconds stimulation provides better CBV responses in terms of specificity to microvasculature, but durations beyond 24 seconds of stimulation may be wasteful for certain applications. (II) We observe that CBV responses show dilation patterns across the cortex. (III) While we found increasingly strong BOLD signal responses in vessel-dominated voxels with longer stimulation durations, we found increasingly strong CBV signal responses in vessel-dominated voxels only until 4 second stimulation durations. After 4 seconds, only the signal from non-vessel dominated voxels kept increasing. This might explain why CBV responses are more specific to the underlying neuronal activity for long stimulus durations.

Early visual experience refines the retinotopic organization within and across visual cortical regions.

Early visual areas are retinotopically organized in human and non-human primates. Population receptive field (pRF) size increases with eccentricity and from lower- to higher-level visual areas. Furthermore, the cortical magnification factor (CMF), a measure of how much cortical space is devoted to each degree of visual angle, is typically larger for foveal as opposed to peripheral regions of the visual field. Whether this fine-scale organization within and across visual areas depends on early visual experience has yet been unknown. Here, we employed 7T functional magnetic resonance imaging pRF mapping to assess the retinotopic organization of early visual regions (i.e., V1, V2, and V3) in eight sight recovery individuals with a history of congenital blindness until a maximum of 4 years of age. Compared with sighted controls, foveal pRF sizes in these individuals were larger, and pRF sizes did not show the typical increase with eccentricity and down the visual cortical processing stream (V1-V2-V3). Cortical magnification was overall diminished and decreased less from foveal to parafoveal visual field locations. Furthermore, cortical magnification correlated with visual acuity in sight recovery individuals. The results of this study suggest that early visual experience is essential for refining a presumably innate prototypical retinotopic organization in humans within and across visual areas, which seems to be crucial for acquiring full visual capabilities.

No threat: Emotion regulation neurofeedback for police special forces recruits.

Police officers of the Special Forces are confronted with highly demanding situations in terms of stress, high tension and threats to their lives. Their tasks are specifically high-risk operations, such as arrests of armed suspects and anti-terror interventions. Improving the emotion regulation skills of police officers might be a vital investment, supporting them to stay calm and focused. A promising approach is training emotion regulation by using real-time (rt-) fMRI neurofeedback. Specifically, downregulating activity in key areas of the fronto-limbic emotion regulation network in the presence of threatening stimuli. Thirteen recruits of the Dutch police special forces underwent six weekly rt-fMRI sessions, receiving neurofeedback from individualized regions of their emotion regulation network. Their task was to reduce the image size of threatening images, wherein the image size represented their brain activity. A reduction in image size represented successful downregulation. Participants were free to use their preferred regulation strategy. A control group of fifteen recruits received no neurofeedback. Both groups completed behavioural tests (image rating on evoked valence and arousal, questionnaire) before and after the neurofeedback training. We hypothesized that the neurofeedback group would improve in downregulation and would score better than the control group on the behavioural tests after the neurofeedback training. Neurofeedback training resulted in a significant decrease in image size (t(12) = 2.82, p = .015) and a trend towards decreased activation in the target regions (t(10) = 1.82, p = .099) from the first to the last session. Notably, subjects achieved downregulation below the pre-stimulus baseline in the last two sessions. No relevant differences between groups were found in the behavioural tasks. Through the training of rt-fMRI neurofeedback, participants learned to downregulate the activity in individualized areas of the emotion regulation network, by using their own preferred strategies. The lack of behavioural between-group differences may be explained by floor effects. Tasks that are close to real-life situations may be needed to uncover behavioural correlates of this emotion regulation training.

Layer-fMRI VASO with short stimuli and event-related designs at 7 T.

Layers and columns are the dominant processing units in the human (neo)cortex at the mesoscopic scale. While the blood oxygenation dependent (BOLD) signal has a high detection sensitivity, it is biased towards unwanted signals from large draining veins at the cortical surface. The additional fMRI contrast of vascular space occupancy (VASO) has the potential to augment the neuroscientific interpretability of layer-fMRI results by means of capturing complementary information of locally specific changes in cerebral blood volume (CBV). Specifically, VASO is not subject to unwanted sensitivity amplifications of large draining veins. Because of constrained sampling efficiency, it has been mainly applied in combination with efficient block task designs and long trial durations. However, to study cognitive processes in neuroscientific contexts, or probe vascular reactivity, short stimulation periods are often necessary. Here, we developed a VASO acquisition procedure with a short acquisition period and sub-millimeter resolution. During visual event-related stimulation, we show reliable responses in visual cortices within a reasonable number of trials (∼20). Furthermore, the short TR and high spatial specificity of our VASO implementation enabled us to show differences in laminar reactivity and onset times. Finally, we explore the generalizability to a different stimulus modality (somatosensation). With this, we showed that CBV-sensitive VASO provides the means to capture layer-specific haemodynamic responses with high spatio-temporal resolution and is able to be used with event-related paradigms.

Imaging the columnar functional organization of human area MT+ to axis-of-motion stimuli using VASO at 7 Tesla.

Cortical columns of direction-selective neurons in the motion sensitive area (MT) have been successfully established as a microscopic feature of the neocortex in animals. The same property has been investigated at mesoscale (<1 mm) in the homologous brain area (hMT+, V5) in living humans by using ultra-high field functional magnetic resonance imaging (fMRI). Despite the reproducibility of the selective response to axis-of-motion stimuli, clear quantitative evidence for the columnar organization of hMT+ is still lacking. Using cerebral blood volume (CBV)-sensitive fMRI at 7 Tesla with submillimeter resolution and high spatial specificity to microvasculature, we investigate the columnar functional organization of hMT+ in 5 participants perceiving axis-of-motion stimuli for both blood oxygenation level dependent (BOLD) and vascular space occupancy (VASO) contrast mechanisms provided by the used slice-selective slab-inversion (SS-SI)-VASO sequence. With the development of a new searchlight algorithm for column detection, we provide the first quantitative columnarity map that characterizes the entire 3D hMT+ volume. Using voxel-wise measures of sensitivity and specificity, we demonstrate the advantage of using CBV-sensitive fMRI to detect mesoscopic cortical features by revealing higher specificity of axis-of-motion cortical columns for VASO as compared to BOLD contrast. These voxel-wise metrics also provide further insights on how to mitigate the highly debated draining veins effect. We conclude that using CBV-VASO fMRI together with voxel-wise measurements of sensitivity, specificity and columnarity offers a promising avenue to quantify the mesoscopic organization of hMT+ with respect to axis-of-motion stimuli. Furthermore, our approach and methodological developments are generalizable and applicable to other human brain areas where similar mesoscopic research questions are addressed.

Talking with hands and feet: Selective somatosensory attention and fMRI enable robust and convenient brain-based communication.

In brain-based communication, voluntarily modulated brain signals (instead of motor output) are utilized to interact with the outside world. The possibility to circumvent the motor system constitutes an important alternative option for severely paralyzed. Most communication brain-computer interface (BCI) paradigms require intact visual capabilities and impose a high cognitive load, but for some patients, these requirements are not given. In these situations, a better-suited, less cognitively demanding information-encoding approach may exploit auditorily-cued selective somatosensory attention to vibrotactile stimulation. Here, we propose, validate and optimize a novel communication-BCI paradigm using differential fMRI activation patterns evoked by selective somatosensory attention to tactile stimulation of the right hand or left foot. Using cytoarchitectonic probability maps and multi-voxel pattern analysis (MVPA), we show that the locus of selective somatosensory attention can be decoded from fMRI-signal patterns in (especially primary) somatosensory cortex with high accuracy and reliability, with the highest classification accuracy (85.93%) achieved when using Brodmann area 2 (SI-BA2) at a probability level of 0.2. Based on this outcome, we developed and validated a novel somatosensory attention-based yes/no communication procedure and demonstrated its high effectiveness even when using only a limited amount of (MVPA) training data. For the BCI user, the paradigm is straightforward, eye-independent, and requires only limited cognitive functioning. In addition, it is BCI-operator friendly given its objective and expertise-independent procedure. For these reasons, our novel communication paradigm has high potential for clinical applications.

Ultra-High Field Imaging of Human Visual Cognition.

Functional magnetic resonance imaging (fMRI), the key methodology for mapping the functions of the human brain in a noninvasive manner, is limited by low temporal and spatial resolution. Recent advances in ultra-high field (UHF) fMRI provide a mesoscopic (i.e., submillimeter resolution) tool that allows us to probe laminar and columnar circuits, distinguish bottom-up versus top-down pathways, and map small subcortical areas. We review recent work demonstrating that UHF fMRI provides a robust methodology for imaging the brain across cortical depths and columns that provides insights into the brain's organization and functions at unprecedented spatial resolution, advancing our understanding of the fine-scale computations and interareal communication that support visual cognition.

A vision of 14 T MR for fundamental and clinical science.

OBJECTIVE: We outline our vision for a 14 Tesla MR system. This comprises a novel whole-body magnet design utilizing high temperature superconductor; a console and associated electronic equipment; an optimized radiofrequency coil setup for proton measurement in the brain, which also has a local shim capability; and a high-performance gradient set. RESEARCH FIELDS: The 14 Tesla system can be considered a 'mesocope': a device capable of measuring on biologically relevant scales. In neuroscience the increased spatial resolution will anatomically resolve all layers of the cortex, cerebellum, subcortical structures, and inner nuclei. Spectroscopic imaging will simultaneously measure excitatory and inhibitory activity, characterizing the excitation/inhibition balance of neural circuits. In medical research (including brain disorders) we will visualize fine-grained patterns of structural abnormalities and relate these changes to functional and molecular changes. The significantly increased spectral resolution will make it possible to detect (dynamic changes in) individual metabolites associated with pathological pathways including molecular interactions and dynamic disease processes. CONCLUSIONS: The 14 Tesla system will offer new perspectives in neuroscience and fundamental research. We anticipate that this initiative will usher in a new era of ultra-high-field MR.

Magnetic resonance imaging at 9.4 T: the Maastricht journey.

The 9.4 T scanner in Maastricht is a whole-body magnet with head gradients and parallel RF transmit capability. At the time of the design, it was conceptualized to be one of the best fMRI scanners in the world, but it has also been used for anatomical and diffusion imaging. 9.4 T offers increases in sensitivity and contrast, but the technical ultra-high field (UHF) challenges, such as field inhomogeneities and constraints set by RF power deposition, are exacerbated compared to 7 T. This article reviews some of the 9.4 T work done in Maastricht. Functional imaging experiments included blood oxygenation level-dependent (BOLD) and blood-volume weighted (VASO) fMRI using different readouts. BOLD benefits from shorter T2* at 9.4 T while VASO from longer T1. We show examples of both ex vivo and in vivo anatomical imaging. For many applications, pTx and optimized coils are essential to harness the full potential of 9.4 T. Our experience shows that, while considerable effort was required compared to our 7 T scanner, we could obtain high-quality anatomical and functional data, which illustrates the potential of MR acquisitions at even higher field strengths. The practical challenges of working with a relatively unique system are also discussed.

AngoraPy: A Python toolkit for modeling anthropomorphic goal-driven sensorimotor systems.

Goal-driven deep learning increasingly supplements classical modeling approaches in computational neuroscience. The strength of deep neural networks as models of the brain lies in their ability to autonomously learn the connectivity required to solve complex and ecologically valid tasks, obviating the need for hand-engineered or hypothesis-driven connectivity patterns. Consequently, goal-driven models can generate hypotheses about the neurocomputations underlying cortical processing that are grounded in macro- and mesoscopic anatomical properties of the network's biological counterpart. Whereas, goal-driven modeling is already becoming prevalent in the neuroscience of perception, its application to the sensorimotor domain is currently hampered by the complexity of the methods required to train models comprising the closed sensation-action loop. This paper describes AngoraPy, a Python library that mitigates this obstacle by providing researchers with the tools necessary to train complex recurrent convolutional neural networks that model the human sensorimotor system. To make the technical details of this toolkit more approachable, an illustrative example that trains a recurrent toy model on in-hand object manipulation accompanies the theoretical remarks. An extensive benchmark on various classical, 3D robotic, and anthropomorphic control tasks demonstrates AngoraPy's general applicability to a wide range of tasks. Together with its ability to adaptively handle custom architectures, the flexibility of this toolkit demonstrates its power for goal-driven sensorimotor modeling.

Mesoscopic in vivo human T2* dataset acquired using quantitative MRI at 7 Tesla.

Mesoscopic (0.1-0.5 mm) interrogation of the living human brain is critical for advancing neuroscience and bridging the resolution gap with animal models. Despite the variety of MRI contrasts measured in recent years at the mesoscopic scale, in vivo quantitative imaging of T2* has not been performed. Here we provide a dataset containing empirical T2* measurements acquired at 0.35 × 0.35 × 0.35 mm3 voxel resolution using 7 Tesla MRI. To demonstrate unique features and high quality of this dataset, we generate flat map visualizations that reveal fine-scale cortical substructures such as layers and vessels, and we report quantitative depth-dependent T2* (as well as R2*) values in primary visual cortex and auditory cortex that are highly consistent across subjects. This dataset is freely available at https://doi.org/10.17605/OSF.IO/N5BJ7, and may prove useful for anatomical investigations of the human brain, as well as for improving our understanding of the basis of the T2*-weighted (f)MRI signal.

Self-modulation of motor cortex activity after stroke: a randomized controlled trial.

Real-time functional MRI neurofeedback allows individuals to self-modulate their ongoing brain activity. This may be a useful tool in clinical disorders that are associated with altered brain activity patterns. Motor impairment after stroke has previously been associated with decreased laterality of motor cortex activity. Here we examined whether chronic stroke survivors were able to use real-time fMRI neurofeedback to increase laterality of motor cortex activity and assessed effects on motor performance and on brain structure and function. We carried out a randomized, double-blind, sham-controlled trial (ClinicalTrials.gov: NCT03775915) in which 24 chronic stroke survivors with mild to moderate upper limb impairment experienced three training days of either Real (n = 12) or Sham (n = 12) neurofeedback. Assessments of brain structure, brain function and measures of upper-limb function were carried out before and 1 week after neurofeedback training. Additionally, measures of upper-limb function were repeated 1 month after neurofeedback training. Primary outcome measures were (i) changes in lateralization of motor cortex activity during movements of the stroke-affected hand throughout neurofeedback training days; and (ii) changes in motor performance of the affected limb on the Jebsen Taylor Test (JTT). Stroke survivors were able to use Real neurofeedback to increase laterality of motor cortex activity within (P = 0.019), but not across, training days. There was no group effect on the primary behavioural outcome measure, which was average JTT performance across all subtasks (P = 0.116). Secondary analysis found improvements in the performance of the gross motor subtasks of the JTT in the Real neurofeedback group compared to Sham (P = 0.010). However, there were no improvements on the Action Research Arm Test or the Upper Extremity Fugl-Meyer score (both P > 0.5). Additionally, decreased white-matter asymmetry of the corticospinal tracts was detected 1 week after neurofeedback training (P = 0.008), indicating that the tracts become more similar with Real neurofeedback. Changes in the affected corticospinal tract were positively correlated with participants neurofeedback performance (P = 0.002). Therefore, here we demonstrate that chronic stroke survivors are able to use functional MRI neurofeedback to self-modulate motor cortex activity in comparison to a Sham control, and that training is associated with improvements in gross hand motor performance and with white matter structural changes.

Improved correspondence of fMRI visual field localizer data after cortex-based macroanatomical alignment.

Studying the visual system with fMRI often requires using localizer paradigms to define regions of interest (ROIs). However, the considerable interindividual variability of the cerebral cortex represents a crucial confound for group-level analyses. Cortex-based alignment (CBA) techniques reliably reduce interindividual macroanatomical variability. Yet, their utility has not been assessed for visual field localizer paradigms, which map specific parts of the visual field within retinotopically organized visual areas. We evaluated CBA for an attention-enhanced visual field localizer, mapping homologous parts of each visual quadrant in 50 participants. We compared CBA with volume-based alignment and a surface-based analysis, which did not include macroanatomical alignment. CBA led to the strongest increase in the probability of activation overlap (up to 86%). At the group level, CBA led to the most consistent increase in ROI size while preserving vertical ROI symmetry. Overall, our results indicate that in addition to the increased signal-to-noise ratio of a surface-based analysis, macroanatomical alignment considerably improves statistical power. These findings confirm and extend the utility of CBA for the study of the visual system in the context of group analyses. CBA should be particularly relevant when studying neuropsychiatric disorders with abnormally increased interindividual macroanatomical variability.

Semantic fMRI neurofeedback: a multi-subject study at 3 tesla.

Objective.Real-time functional magnetic resonance imaging neurofeedback (rt-fMRI-NF) is a non-invasive procedure allowing the self-regulation of brain functions via enhanced self-control of fMRI based neural activation. In semantic rt-fMRI-NF, an estimated relation between multivariate fMRI activation patterns and abstract mental states is exploited for a multi-dimensional feedback stimulus via real-time representational similarity analysis (rt-RSA). Here, we assessed the performances of this framework in a multi-subject multi-session study on a 3 T MRI clinical scanner.Approach.Eighteen healthy volunteers underwent two semantic rt-fMRI-NF sessions on two different days. In each session, participants were first requested to engage in specific mental states while local fMRI patterns of brain activity were recorded during stimulated mental imagery of concrete objects (pattern generation). The obtained neural representations were to be replicated and modulated by the participants in subsequent runs of the same session under the guidance of a rt-RSA generated visual feedback (pattern modulation). Performance indicators were derived from the rt-RSA output to assess individual abilities in replicating (and maintaining over time) a target pattern. Simulations were carried out to assess the impact of the geometric distortions implied by the low-dimensional representation of patterns' dissimilarities in the visual feedback.Main results.Sixteen subjects successfully completed both semantic rt-fMRI-NF sessions. Considering some performance indicators, a significant improvement between the first and the second runs, and within run increasing modulation performances were observed, whereas no improvements were found between sessions. Simulations confirmed that in a small percentage of cases visual feedback could be affected by metric distortions due to dimensionality reduction implicit to the rt-RSA approach.Significance.Our results proved the feasibility of the semantic rt-fMRI-NF at 3 T, showing that subjects can successfully modulate and maintain a target mental state when guided by rt-RSA derived feedback. Further development is needed to encourage future clinical applications.

Linking Brain Structure, Activity, and Cognitive Function through Computation.

Understanding the human brain is a "Grand Challenge" for 21st century research. Computational approaches enable large and complex datasets to be addressed efficiently, supported by artificial neural networks, modeling and simulation. Dynamic generative multiscale models, which enable the investigation of causation across scales and are guided by principles and theories of brain function, are instrumental for linking brain structure and function. An example of a resource enabling such an integrated approach to neuroscientific discovery is the BigBrain, which spatially anchors tissue models and data across different scales and ensures that multiscale models are supported by the data, making the bridge to both basic neuroscience and medicine. Research at the intersection of neuroscience, computing and robotics has the potential to advance neuro-inspired technologies by taking advantage of a growing body of insights into perception, plasticity and learning. To render data, tools and methods, theories, basic principles and concepts interoperable, the Human Brain Project (HBP) has launched EBRAINS, a digital neuroscience research infrastructure, which brings together a transdisciplinary community of researchers united by the quest to understand the brain, with fascinating insights and perspectives for societal benefits.

See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions.

Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.