The multimodality cell segmentation challenge: toward universal solutions.

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multimodality cell segmentation benchmark, comprising more than 1,500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.

Effect of in-plane and out-of-plane bifurcated microfluidic channels on the flow of aggregating red blood cells.

The blood flow through our microvascular system is a renowned difficult process to understand because the complex flow behavior of blood is intertwined with the complex geometry it has to flow through. Conventional 2D microfluidics has provided important insights, but progress is hampered by the limitation of 2-D confinement. Here we use selective laser-induced etching to excavate non-planar 3-D microfluidic channels in glass that consist of two generations of bifurcations, heading towards more physiological geometries. We identify a cross-talk between the first and second bifurcation only when both bifurcations are in the same plane, as observed in 2D microfluidics. Contrarily, the flow in the branch where the second bifurcation is perpendicular to the first is hardly affected by the initial distortion. This difference in flow behavior is only observed when red blood cells are aggregated, due to the presence of dextran, and disappears by increasing the distance between both generations of bifurcations. Thus, 3-D structures scramble in-plane flow distortions, exemplifying the importance of experimenting with truly 3D microfluidic designs in order to understand complex physiological flow behavior.

A framework to improve the alignment of individual cytoarchitectonic maps of the Julich-Brain atlas using cortical folding landmarks.

The segregation of the cortical mantle into cytoarchitectonic areas provides a structural basis for the specialization of different brain regions. In vivo neuroimaging experiments can be linked to this postmortem cytoarchitectonic parcellation via Julich-Brain. This atlas embeds probabilistic maps that account for inter-individual variability in the localization of cytoarchitectonic areas in the reference spaces targeted by spatial normalization. We built a framework to improve the alignment of architectural areas across brains using cortical folding landmarks. This framework, initially designed for in vivo imaging, was adapted to postmortem histological data. We applied this to the first 14 brains used to establish the Julich-Brain atlas to infer a refined atlas with more focal probabilistic maps. The improvement achieved is significant in the primary regions and some of the associative areas. This framework also provides a tool for exploring the relationship between cortical folding patterns and cytoarchitectonic areas in different cortical regions to establish new landmarks in the remainder of the cortex.

AtOM, an ontology model to standardize use of brain atlases in tools, workflows, and data infrastructures.

Brain atlases are important reference resources for accurate anatomical description of neuroscience data. Open access, three-dimensional atlases serve as spatial frameworks for integrating experimental data and defining regions-of-interest in analytic workflows. However, naming conventions, parcellation criteria, area definitions, and underlying mapping methodologies differ considerably between atlases and across atlas versions. This lack of standardized description impedes use of atlases in analytic tools and registration of data to different atlases. To establish a machine-readable standard for representing brain atlases, we identified four fundamental atlas elements, defined their relations, and created an ontology model. Here we present our Atlas Ontology Model (AtOM) and exemplify its use by applying it to mouse, rat, and human brain atlases. We discuss how AtOM can facilitate atlas interoperability and data integration, thereby increasing compliance with the FAIR guiding principles. AtOM provides a standardized framework for communication and use of brain atlases to create, use, and refer to specific atlas elements and versions. We argue that AtOM will accelerate analysis, sharing, and reuse of neuroscience data.

Cytoarchitectonic mapping of the human frontal operculum-New correlates for a variety of brain functions.

The human frontal operculum (FOp) is a brain region that covers parts of the ventral frontal cortex next to the insula. Functional imaging studies showed activations in this region in tasks related to language, somatosensory, and cognitive functions. While the precise cytoarchitectonic areas that correlate to these processes have not yet been revealed, earlier receptorarchitectonic analysis resulted in a detailed parcellation of the FOp. We complemented this analysis by a cytoarchitectonic study of a sample of ten postmortem brains and mapped the posterior FOp in serial, cell-body stained histological sections using image analysis and multivariate statistics. Three new areas were identified: Op5 represents the most posterior area, followed by Op6 and the most anterior region Op7. Areas Op5-Op7 approach the insula, up to the circular sulcus. Area 44 of Broca's region, the most ventral part of premotor area 6, and parts of the parietal operculum are dorso-laterally adjacent to Op5-Op7. The areas did not show any interhemispheric or sex differences. Three-dimensional probability maps and a maximum probability map were generated in stereotaxic space, and then used, in a first proof-of-concept-study, for functional decoding and analysis of structural and functional connectivity. Functional decoding revealed different profiles of cytoarchitectonically identified Op5-Op7. While left Op6 was active in music cognition, right Op5 was involved in chewing/swallowing and sexual processing. Both areas showed activation during the exercise of isometric force in muscles. An involvement in the coordination of flexion/extension could be shown for the right Op6. Meta-analytic connectivity modeling revealed various functional connections of the FOp areas within motor and somatosensory networks, with the most evident connection with the music/language network for Op6 left. The new cytoarchitectonic maps are part of Julich-Brain, and publicly available to serve as a basis for future analyses of structural-functional relationships in this region.

Characterization of regional differences in resting-state fMRI with a data-driven network model of brain dynamics.

Model-based data analysis of whole-brain dynamics links the observed data to model parameters in a network of neural masses. Recently, studies focused on the role of regional variance of model parameters. Such analyses however necessarily depend on the properties of preselected neural mass model. We introduce a method to infer from the functional data both the neural mass model representing the regional dynamics and the region- and subject-specific parameters while respecting the known network structure. We apply the method to human resting-state fMRI. We find that the underlying dynamics can be described as noisy fluctuations around a single fixed point. The method reliably discovers three regional parameters with clear and distinct role in the dynamics, one of which is strongly correlated with the first principal component of the gene expression spatial map. The present approach opens a novel way to the analysis of resting-state fMRI with possible applications for understanding the brain dynamics during aging or neurodegeneration.

Mapping Cytoarchitectonics and Receptor Architectonics to Understand Brain Function and Connectivity.

This review focuses on cytoarchitectonics and receptor architectonics as biological correlates of function and connectivity. It introduces the 3-dimensional cytoarchitectonic probabilistic maps of cortical areas and nuclei of the Julich-Brain Atlas, available at EBRAINS, to study structure-function relationships. The maps are linked to the BigBrain as microanatomical reference model and template space. The siibra software tool suite enables programmatic access to the maps and to receptor architectonic data that are anchored to brain areas. Such cellular and molecular data are tools for studying magnetic resonance connectivity including modeling and simulation. At the end, we highlight perspectives of the Julich-Brain as well as methodological considerations. Thus, microstructural maps as part of a multimodal atlas help elucidate the biological correlates of large-scale networks and brain function with a high level of anatomical detail, which provides a basis to study brains of patients with psychiatric disorders.

Cytoarchitecture, probability maps and segregation of the human insula.

The human insular cortex supports multifunctional integration including interoceptive, sensorimotor, cognitive and social-emotional processing. Different concepts of the underlying microstructure have been proposed over more than a century. However, a 3D map of the cytoarchitectonic segregation of the insula in standard reference space, that could be directly linked to neuroimaging experiments addressing different cognitive tasks, is not yet available. Here we analyzed the middle posterior and dorsal anterior insula with image analysis and a statistical mapping procedure to delineate cytoarchitectonic areas in ten human postmortem brains. 3D-probability maps of seven new areas with granular (Ig3, posterior), agranular (Ia1, posterior) and dysgranular (Id2-Id6, middle to dorsal anterior) cytoarchitecture have been calculated to represent the new areas in stereotaxic space. A hierarchical cluster analysis based on cytoarchitecture resulted in three distinct clusters in the superior posterior, inferior posterior and dorsal anterior insula, providing deeper insights into the structural organization of the insula. The maps are openly available to support future studies addressing relations between structure and function in the human insula.

Contour proposal networks for biomedical instance segmentation.

We present a conceptually simple framework for object instance segmentation, called Contour Proposal Network (CPN), which detects possibly overlapping objects in an image while simultaneously fitting closed object contours using a fixed-size representation based on Fourier Descriptors. The CPN can incorporate state-of-the-art object detection architectures as backbone networks into a single-stage instance segmentation model that can be trained end-to-end. We construct CPN models with different backbone networks and apply them to instance segmentation of cells in datasets from different modalities. In our experiments, CPNs outperform U-Net, Mask R-CNN and StarDist in instance segmentation accuracy. We present variants with execution times suitable for real-time applications. The trained models generalize well across different domains of cell types. Since the main assumption of the framework is closed object contours, it is applicable to a wide range of detection problems also beyond the biomedical domain. An implementation of the model architecture in PyTorch is freely available.

Brain simulation as a cloud service: The Virtual Brain on EBRAINS.

The Virtual Brain (TVB) is now available as open-source services on the cloud research platform EBRAINS (ebrains.eu). It offers software for constructing, simulating and analysing brain network models including the TVB simulator; magnetic resonance imaging (MRI) processing pipelines to extract structural and functional brain networks; combined simulation of large-scale brain networks with small-scale spiking networks; automatic conversion of user-specified model equations into fast simulation code; simulation-ready brain models of patients and healthy volunteers; Bayesian parameter optimization in epilepsy patient models; data and software for mouse brain simulation; and extensive educational material. TVB cloud services facilitate reproducible online collaboration and discovery of data assets, models, and software embedded in scalable and secure workflows, a precondition for research on large cohort data sets, better generalizability, and clinical translation.

The BigBrainWarp toolbox for integration of BigBrain 3D histology with multimodal neuroimaging.

Neuroimaging stands to benefit from emerging ultrahigh-resolution 3D histological atlases of the human brain; the first of which is 'BigBrain'. Here, we review recent methodological advances for the integration of BigBrain with multi-modal neuroimaging and introduce a toolbox, 'BigBrainWarp', that combines these developments. The aim of BigBrainWarp is to simplify workflows and support the adoption of best practices. This is accomplished with a simple wrapper function that allows users to easily map data between BigBrain and standard MRI spaces. The function automatically pulls specialised transformation procedures, based on ongoing research from a wide collaborative network of researchers. Additionally, the toolbox improves accessibility of histological information through dissemination of ready-to-use cytoarchitectural features. Finally, we demonstrate the utility of BigBrainWarp with three tutorials and discuss the potential of the toolbox to support multi-scale investigations of brain organisation.

Convolutional neural networks for cytoarchitectonic brain mapping at large scale.

Human brain atlases provide spatial reference systems for data characterizing brain organization at different levels, coming from different brains. Cytoarchitecture is a basic principle of the microstructural organization of the brain, as regional differences in the arrangement and composition of neuronal cells are indicators of changes in connectivity and function. Automated scanning procedures and observer-independent methods are prerequisites to reliably identify cytoarchitectonic areas, and to achieve reproducible models of brain segregation. Time becomes a key factor when moving from the analysis of single regions of interest towards high-throughput scanning of large series of whole-brain sections. Here we present a new workflow for mapping cytoarchitectonic areas in large series of cell-body stained histological sections of human postmortem brains. It is based on a Deep Convolutional Neural Network (CNN), which is trained on a pair of section images with annotations, with a large number of un-annotated sections in between. The model learns to create all missing annotations in between with high accuracy, and faster than our previous workflow based on observer-independent mapping. The new workflow does not require preceding 3D-reconstruction of sections, and is robust against histological artefacts. It processes large data sets with sizes in the order of multiple Terabytes efficiently. The workflow was integrated into a web interface, to allow access without expertise in deep learning and batch computing. Applying deep neural networks for cytoarchitectonic mapping opens new perspectives to enable high-resolution models of brain areas, introducing CNNs to identify borders of brain areas.

Deep learning networks reflect cytoarchitectonic features used in brain mapping.

The distribution of neurons in the cortex (cytoarchitecture) differs between cortical areas and constitutes the basis for structural maps of the human brain. Deep learning approaches provide a promising alternative to overcome throughput limitations of currently used cytoarchitectonic mapping methods, but typically lack insight as to what extent they follow cytoarchitectonic principles. We therefore investigated in how far the internal structure of deep convolutional neural networks trained for cytoarchitectonic brain mapping reflect traditional cytoarchitectonic features, and compared them to features of the current grey level index (GLI) profile approach. The networks consisted of a 10-block deep convolutional architecture trained to segment the primary and secondary visual cortex. Filter activations of the networks served to analyse resemblances to traditional cytoarchitectonic features and comparisons to the GLI profile approach. Our analysis revealed resemblances to cellular, laminar- as well as cortical area related cytoarchitectonic features. The networks learned filter activations that reflect the distinct cytoarchitecture of the segmented cortical areas with special regard to their laminar organization and compared well to statistical criteria of the GLI profile approach. These results confirm an incorporation of relevant cytoarchitectonic features in the deep convolutional neural networks and mark them as a valid support for high-throughput cytoarchitectonic mapping workflows.

BigBrain 3D atlas of cortical layers: Cortical and laminar thickness gradients diverge in sensory and motor cortices.

Histological atlases of the cerebral cortex, such as those made famous by Brodmann and von Economo, are invaluable for understanding human brain microstructure and its relationship with functional organization in the brain. However, these existing atlases are limited to small numbers of manually annotated samples from a single cerebral hemisphere, measured from 2D histological sections. We present the first whole-brain quantitative 3D laminar atlas of the human cerebral cortex. It was derived from a 3D histological atlas of the human brain at 20-micrometer isotropic resolution (BigBrain), using a convolutional neural network to segment, automatically, the cortical layers in both hemispheres. Our approach overcomes many of the historical challenges with measurement of histological thickness in 2D, and the resultant laminar atlas provides an unprecedented level of precision and detail. We utilized this BigBrain cortical atlas to test whether previously reported thickness gradients, as measured by MRI in sensory and motor processing cortices, were present in a histological atlas of cortical thickness and which cortical layers were contributing to these gradients. Cortical thickness increased across sensory processing hierarchies, primarily driven by layers III, V, and VI. In contrast, motor-frontal cortices showed the opposite pattern, with decreases in total and pyramidal layer thickness from motor to frontal association cortices. These findings illustrate how this laminar atlas will provide a link between single-neuron morphology, mesoscale cortical layering, macroscopic cortical thickness, and, ultimately, functional neuroanatomy.

BigBrain: an ultrahigh-resolution 3D human brain model.

Reference brains are indispensable tools in human brain mapping, enabling integration of multimodal data into an anatomically realistic standard space. Available reference brains, however, are restricted to the macroscopic scale and do not provide information on the functionally important microscopic dimension. We created an ultrahigh-resolution three-dimensional (3D) model of a human brain at nearly cellular resolution of 20 micrometers, based on the reconstruction of 7404 histological sections. "BigBrain" is a free, publicly available tool that provides considerable neuroanatomical insight into the human brain, thereby allowing the extraction of microscopic data for modeling and simulation. BigBrain enables testing of hypotheses on optimal path lengths between interconnected cortical regions or on spatial organization of genetic patterning, redefining the traditional neuroanatomy maps such as those of Brodmann and von Economo.

Automatic identification of gray and white matter components in polarized light imaging.

Polarized light imaging (PLI) enables the visualization of fiber tracts with high spatial resolution in microtome sections of postmortem brains. Vectors of the fiber orientation defined by inclination and direction angles can directly be derived from the optical signals employed by PLI analysis. The polarization state of light propagating through a rotating polarimeter is varied in such a way that the detected signal of each spatial unit describes a sinusoidal signal. Noise, light scatter and filter inhomogeneities, however, interfere with the original sinusoidal PLI signals, which in turn have direct impact on the accuracy of subsequent fiber tracking. Recently we showed that the primary sinusoidal signals can effectively be restored after noise and artifact rejection utilizing independent component analysis (ICA). In particular, regions with weak intensities are greatly enhanced after ICA based artifact rejection and signal restoration. Here, we propose a user independent way of identifying the components of interest after decomposition; i.e., components that are related to gray and white matter. Depending on the size of the postmortem brain and the section thickness, the number of independent component maps can easily be in the range of a few ten thousand components for one brain. Therefore, we developed an automatic and, more importantly, user independent way of extracting the signal of interest. The automatic identification of gray and white matter components is based on the evaluation of the statistical properties of the so-called feature vectors of each individual component map, which, in the ideal case, shows a sinusoidal waveform. Our method enables large-scale analysis (i.e., the analysis of thousands of whole brain sections) of nerve fiber orientations in the human brain using polarized light imaging.

Classification of ambiguous nerve fiber orientations in 3D polarized light imaging.

3D polarized light imaging (3D-PLI) has been shown to measure the orientation of nerve fibers in post mortem human brains at ultra high resolution. The 3D orientation in each voxel is obtained as a pair of angles, the direction angle and the inclination angle with unknown sign. The sign ambiguity is a major problem for the correct interpretation of fiber orientation. Measurements from a tiltable specimen stage, that are highly sensitive to noise, extract information, which allows drawing conclusions about the true inclination sign. In order to reduce noise, we propose a global classification of the inclination sign, which combines measurements with spatial coherence constraints. The problem is formulated as a second order Markov random field and solved efficiently with graph cuts. We evaluate our approach on synthetic and human brain data. The results of global optimization are compared to independent pixel classification with subsequent edge-preserving smoothing.

High-resolution fiber tract reconstruction in the human brain by means of three-dimensional polarized light imaging.

Functional interactions between different brain regions require connecting fiber tracts, the structural basis of the human connectome. To assemble a comprehensive structural understanding of neural network elements from the microscopic to the macroscopic dimensions, a multimodal and multiscale approach has to be envisaged. However, the integration of results from complementary neuroimaging techniques poses a particular challenge. In this paper, we describe a steadily evolving neuroimaging technique referred to as three-dimensional polarized light imaging (3D-PLI). It is based on the birefringence of the myelin sheaths surrounding axons, and enables the high-resolution analysis of myelinated axons constituting the fiber tracts. 3D-PLI provides the mapping of spatial fiber architecture in the postmortem human brain at a sub-millimeter resolution, i.e., at the mesoscale. The fundamental data structure gained by 3D-PLI is a comprehensive 3D vector field description of fibers and fiber tract orientations - the basis for subsequent tractography. To demonstrate how 3D-PLI can contribute to unravel and assemble the human connectome, a multiscale approach with the same technology was pursued. Two complementary state-of-the-art polarimeters providing different sampling grids (pixel sizes of 100 and 1.6 µm) were used. To exemplarily highlight the potential of this approach, fiber orientation maps and 3D fiber models were reconstructed in selected regions of the brain (e.g., Corpus callosum, Internal capsule, Pons). The results demonstrate that 3D-PLI is an ideal tool to serve as an interface between the microscopic and macroscopic levels of organization of the human connectome.