A new family of instance-level loss functions for improving instance-level segmentation and detection of white matter hyperintensities in routine clinical brain MRI.

In this study, we introduce "instance loss functions", a new family of loss functions designed to enhance the training of neural networks in the instance-level segmentation and detection of objects in biomedical image data, particularly those of varied numbers and sizes. Intended to be utilized conjointly with traditional loss functions, these proposed functions, prioritize object instances over pixel-by-pixel comparisons. The specific functions, the instance segmentation loss (Linstance), the instance center loss (Lcenter), the false instance rate loss (Lfalse), and the instance proximity loss (Lproximity), serve distinct purposes. Specifically, Linstance improves instance-wise segmentation quality, Lcenter enhances segmentation quality of small instances, Lfalse minimizes the rate of false and missed detections across varied instance sizes, and Lproximity improves detection quality by pulling predicted instances towards the ground truth instances. Through the task of segmenting white matter hyperintensities (WMH) in brain MRI, we benchmarked our proposed instance loss functions, both individually and in combination via an ensemble inference models approach, against traditional pixel-level loss functions. Data were sourced from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and the WMH Segmentation Challenge datasets, which exhibit significant variation in WMH instance sizes. Empirical evaluations demonstrate that combining two instance-level loss functions through ensemble inference models outperforms models using other loss function on both the ADNI and WMH Segmentation Challenge datasets for the segmentation and detection of WMH instances. Further, applying these functions to the segmentation of nuclei in histopathology images demonstrated their effectiveness and generalizability beyond WMH, improving performance even in contexts with less severe instance imbalance.

Study of Dendrite Differentiation Using Drosophila Dendritic Arborization Neurons.

Neurons receive, process, and integrate inputs. These operations are organized by dendrite arbor morphology, and the dendritic arborization (da) neurons of the Drosophila peripheral sensory nervous system are an excellent experimental model for examining the differentiation processes that build and shape the dendrite arbor. Studies in da neurons are enabled by a wealth of fly genetic tools that allow targeted neuron manipulation and labeling of the neuron's cytoskeletal or organellar components. Moreover, as da neuron dendrite arbors cover the body wall, they are highly accessible for live imaging analysis of arbor patterning. Here, we outline the structure and function of different da neuron types and give examples of how they are used to elucidate central mechanisms of dendritic arbor formation.

Use of DeTerm for Automated Drosophila Dendrite Arbor Terminal Counts.

Neurons have a complex dendritic architecture that governs information flow through a circuit. Manual quantification of dendritic arbor morphometrics is time-consuming and can be inaccurate. Automated quantification systems such as DeTerm help to overcome these limitations. DeTerm is a software tool that automatically recognizes dendrite branch terminals with high precision. It uses an artificial neural network to label the terminals, count them, and provide each terminal's positional data. DeTerm can recognize the dendritic terminals of Drosophila dendritic arborization (da) neurons, and it can also examine other types of neurons, including mouse Purkinje cells. It is freely available and works on Mac, Windows, and Linux. Here, we describe the use of DeTerm.

Exploring the performance of implicit neural representations for brain image registration.

Pairwise image registration is a necessary prerequisite for brain image comparison and data integration in neuroscience and radiology. In this work, we explore the efficacy of implicit neural representations (INRs) in improving the performance of brain image registration in magnetic resonance imaging. In this setting, INRs serve as a continuous and coordinate based approximation of the deformation field obtained through a multi-layer perceptron. Previous research has demonstrated that sinusoidal representation networks (SIRENs) surpass ReLU models in performance. In this study, we first broaden the range of activation functions to further investigate the registration performance of implicit networks equipped with activation functions that exhibit diverse oscillatory properties. Specifically, in addition to the SIRENs and ReLU, we evaluate activation functions based on snake, sine+, chirp and Morlet wavelet functions. Second, we conduct experiments to relate the hyper-parameters of the models to registration performance. Third, we propose and assess various techniques, including cycle consistency loss, ensembles and cascades of implicit networks, as well as a combined image fusion and registration objective, to enhance the performance of implicit registration networks beyond the standard approach. The investigated implicit methods are compared to the VoxelMorph convolutional neural network and to the symmetric image normalization (SyN) registration algorithm from the Advanced Normalization Tools (ANTs). Our findings not only highlight the remarkable capabilities of implicit networks in addressing pairwise image registration challenges, but also showcase their potential as a powerful and versatile off-the-shelf tool in the fields of neuroscience and radiology.

Distinctiveness and continuity in transcriptome and connectivity in the anterior-posterior axis of the paraventricular nucleus of the thalamus.

The paraventricular nucleus of the thalamus (PVT) projects axons to multiple areas, mediates a wide range of behaviors, and exhibits regional heterogeneity in both functions and axonal projections. Still, questions regarding the cell types present in the PVT and the extent of their differences remain inadequately addressed. We applied single-cell RNA sequencing to depict the transcriptomic characteristics of mouse PVT neurons. We found that one of the most significant variances in the PVT transcriptome corresponded to the anterior-posterior axis. While the single-cell transcriptome classified PVT neurons into five types, our transcriptomic and histological analyses showed continuity among the cell types. We discovered that anterior and posterior subpopulations had nearly non-overlapping projection patterns, while another population showed intermediate patterns. In addition, these subpopulations responded differently to appetite-related neuropeptides, with their activation showing opposing effects on food consumption. Our studies unveiled the contrasts and the continuity of PVT neurons that underpin their function.

The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain.

The primate brain has unique anatomical characteristics, which translate into advanced cognitive, sensory, and motor abilities. Thus, it is important that we gain insight on its structure to provide a solid basis for models that will clarify function. Here, we report on the implementation and features of the Brain/MINDS Marmoset Connectivity Resource (BMCR), a new open-access platform that provides access to high-resolution anterograde neuronal tracer data in the marmoset brain, integrated to retrograde tracer and tractography data. Unlike other existing image explorers, the BMCR allows visualization of data from different individuals and modalities in a common reference space. This feature, allied to an unprecedented high resolution, enables analyses of features such as reciprocity, directionality, and spatial segregation of connections. The present release of the BMCR focuses on the prefrontal cortex (PFC), a uniquely developed region of the primate brain that is linked to advanced cognition, including the results of 52 anterograde and 164 retrograde tracer injections in the cortex of the marmoset. Moreover, the inclusion of tractography data from diffusion MRI allows systematic analyses of this noninvasive modality against gold-standard cellular connectivity data, enabling detection of false positives and negatives, which provide a basis for future development of tractography. This paper introduces the BMCR image preprocessing pipeline and resources, which include new tools for exploring and reviewing the data.

Local and long-distance organization of prefrontal cortex circuits in the marmoset brain.

The prefrontal cortex (PFC) has dramatically expanded in primates, but its organization and interactions with other brain regions are only partially understood. We performed high-resolution connectomic mapping of the marmoset PFC and found two contrasting corticocortical and corticostriatal projection patterns: "patchy" projections that formed many columns of submillimeter scale in nearby and distant regions and "diffuse" projections that spread widely across the cortex and striatum. Parcellation-free analyses revealed representations of PFC gradients in these projections' local and global distribution patterns. We also demonstrated column-scale precision of reciprocal corticocortical connectivity, suggesting that PFC contains a mosaic of discrete columns. Diffuse projections showed considerable diversity in the laminar patterns of axonal spread. Altogether, these fine-grained analyses reveal important principles of local and long-distance PFC circuits in marmosets and provide insights into the functional organization of the primate brain.

Multi-modal brain magnetic resonance imaging database covering marmosets with a wide age range.

Magnetic resonance imaging (MRI) is a non-invasive neuroimaging technique that is useful for identifying normal developmental and aging processes and for data sharing. Marmosets have a relatively shorter life expectancy than other primates, including humans, because they grow and age faster. Therefore, the common marmoset model is effective in aging research. The current study investigated the aging process of the marmoset brain and provided an open MRI database of marmosets across a wide age range. The Brain/MINDS Marmoset Brain MRI Dataset contains brain MRI information from 216 marmosets ranging in age from 1 and 10 years. At the time of its release, it is the largest public dataset in the world. It also includes multi-contrast MRI images. In addition, 91 of 216 animals have corresponding high-resolution ex vivo MRI datasets. Our MRI database, available at the Brain/MINDS Data Portal, might help to understand the effects of various factors, such as age, sex, body size, and fixation, on the brain. It can also contribute to and accelerate brain science studies worldwide.

Tomographic tract tracing and data driven approaches to unravel complex 3D fiber anatomy of DBS relevant prefrontal projections to the diencephalic-mesencephalic junction in the marmoset.

BACKGROUND: Understanding prefrontal cortex projections to diencephalic-mesencephalic junction (DMJ), especially to subthalamic nucleus (STN) and ventral mesencephalic tegmentum (VMT) helps our comprehension of Deep Brain Stimulation (DBS) in major depression (MD) and obsessive-compulsive disorder (OCD). Fiber routes are complex and tract tracing studies in non-human primate species (NHP) have yielded conflicting results. The superolateral medial forebrain bundle (slMFB) is a promising target for DBS in MD and OCD. It has become a focus of criticism owing to its name and its diffusion weighted-imaging based primary description. OBJECTIVE: To investigate DMJ connectivity in NHP with a special focus on slMFB and the limbic hyperdirect pathway utilizing three-dimensional and data driven techniques. METHODS: We performed left prefrontal adeno-associated virus - tracer based injections in the common marmoset monkey (n = 52). Histology and two-photon microscopy were integrated into a common space. Manual and data driven cluster analyses of DMJ, subthalamic nucleus and VMT together, followed by anterior tract tracing streamline (ATTS) tractography were deployed. RESULTS: Typical pre- and supplementary motor hyperdirect connectivity was confirmed. The advanced tract tracing unraveled the complex connectivity to the DMJ. Limbic prefrontal territories directly projected to the VMT but not STN. DISCUSSION: Intricate results of tract tracing studies warrant the application of advanced three-dimensional analyses to understand complex fiber-anatomical routes. The applied three-dimensional techniques can enhance anatomical understanding also in other regions with complex fiber anatomy. CONCLUSION: Our work confirms slMFB anatomy and enfeebles previous misconceptions. The rigorous NHP approach strengthens the role of the slMFB as a target structure for DBS predominantly in psychiatric indications like MD and OCD.

A dataset of rodent cerebrovasculature from in vivo multiphoton fluorescence microscopy imaging.

We present MiniVess, the first annotated dataset of rodent cerebrovasculature, acquired using two-photon fluorescence microscopy. MiniVess consists of 70 3D image volumes with segmented ground truths. Segmentations were created using traditional image processing operations, a U-Net, and manual proofreading. Code for image preprocessing steps and the U-Net are provided. Supervised machine learning methods have been widely used for automated image processing of biomedical images. While much emphasis has been placed on the development of new network architectures and loss functions, there has been an increased emphasis on the need for publicly available annotated, or segmented, datasets. Annotated datasets are necessary during model training and validation. In particular, datasets that are collected from different labs are necessary to test the generalizability of models. We hope this dataset will be helpful in testing the reliability of machine learning tools for analyzing biomedical images.

Where Position Matters-Deep-Learning-Driven Normalization and Coregistration of Computed Tomography in the Postoperative Analysis of Deep Brain Stimulation.

INTRODUCTION: Recent developments in the postoperative evaluation of deep brain stimulation surgery on the group level warrant the detection of achieved electrode positions based on postoperative imaging. Computed tomography (CT) is a frequently used imaging modality, but because of its idiosyncrasies (high spatial accuracy at low soft tissue resolution), it has not been sufficient for the parallel determination of electrode position and details of the surrounding brain anatomy (nuclei). The common solution is rigid fusion of CT images and magnetic resonance (MR) images, which have much better soft tissue contrast and allow accurate normalization into template spaces. Here, we explored a deep-learning approach to directly relate positions (usually the lead position) in postoperative CT images to the native anatomy of the midbrain and group space. MATERIALS AND METHODS: Deep learning is used to create derived tissue contrasts (white matter, gray matter, cerebrospinal fluid, brainstem nuclei) based on the CT image; that is, a convolution neural network (CNN) takes solely the raw CT image as input and outputs several tissue probability maps. The ground truth is based on coregistrations with MR contrasts. The tissue probability maps are then used to either rigidly coregister or normalize the CT image in a deformable way to group space. The CNN was trained in 220 patients and tested in a set of 80 patients. RESULTS: Rigorous validation of such an approach is difficult because of the lack of ground truth. We examined the agreements between the classical and proposed approaches and considered the spread of implantation locations across a group of identically implanted subjects, which serves as an indicator of the accuracy of the lead localization procedure. The proposed procedure agrees well with current magnetic resonance imaging-based techniques, and the spread is comparable or even lower. CONCLUSIONS: Postoperative CT imaging alone is sufficient for accurate localization of the midbrain nuclei and normalization to the group space. In the context of group analysis, it seems sufficient to have a single postoperative CT image of good quality for inclusion. The proposed approach will allow researchers and clinicians to include cases that were not previously suitable for analysis.

A Spiking Neural Network Builder for Systematic Data-to-Model Workflow.

In building biological neural network models, it is crucial to efficiently convert diverse anatomical and physiological data into parameters of neurons and synapses and to systematically estimate unknown parameters in reference to experimental observations. Web-based tools for systematic model building can improve the transparency and reproducibility of computational models and can facilitate collaborative model building, validation, and evolution. Here, we present a framework to support collaborative data-driven development of spiking neural network (SNN) models based on the Entity-Relationship (ER) data description commonly used in large-scale business software development. We organize all data attributes, including species, brain regions, neuron types, projections, neuron models, and references as tables and relations within a database management system (DBMS) and provide GUI interfaces for data registration and visualization. This allows a robust "business-oriented" data representation that supports collaborative model building and traceability of source information for every detail of a model. We tested this data-to-model framework in cortical and striatal network models by successfully combining data from papers with existing neuron and synapse models and by generating NEST simulation codes for various network sizes. Our framework also helps to check data integrity and consistency and data comparisons across species. The framework enables the modeling of any region of the brain and is being deployed to support the integration of anatomical and physiological datasets from the brain/MINDS project for systematic SNN modeling of the marmoset brain.

Cellular-resolution gene expression profiling in the neonatal marmoset brain reveals dynamic species- and region-specific differences.

Precise spatiotemporal control of gene expression in the developing brain is critical for neural circuit formation, and comprehensive expression mapping in the developing primate brain is crucial to understand brain function in health and disease. Here, we developed an unbiased, automated, large-scale, cellular-resolution in situ hybridization (ISH)-based gene expression profiling system (GePS) and companion analysis to reveal gene expression patterns in the neonatal New World marmoset cortex, thalamus, and striatum that are distinct from those in mice. Gene-ontology analysis of marmoset-specific genes revealed associations with catalytic activity in the visual cortex and neuropsychiatric disorders in the thalamus. Cortically expressed genes with clear area boundaries were used in a three-dimensional cortical surface mapping algorithm to delineate higher-order cortical areas not evident in two-dimensional ISH data. GePS provides a powerful platform to elucidate the molecular mechanisms underlying primate neurobiology and developmental psychiatric and neurological disorders.

Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference.

Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can reveal the brain's global network architecture and also abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result. Here we propose a general data-driven framework to optimize and validate parameters of dMRI-based fiber tracking algorithms using neural tracer data as a reference. Japan's Brain/MINDS Project provides invaluable datasets containing both dMRI and neural tracer data from the same primates. A fundamental difference when comparing dMRI-based tractography and neural tracer data is that the former cannot specify the direction of connectivity; therefore, evaluating the fitting of dMRI-based tractography becomes challenging. The framework implements multi-objective optimization based on the non-dominated sorting genetic algorithm II. Its performance is examined in two experiments using data from ten subjects for optimization and six for testing generalization. The first uses a seed-based tracking algorithm, iFOD2, and objectives for sensitivity and specificity of region-level connectivity. The second uses a global tracking algorithm and a more refined set of objectives: distance-weighted coverage, true/false positive ratio, projection coincidence, and commissural passage. In both experiments, with optimized parameters compared to default parameters, fiber tracking performance was significantly improved in coverage and fiber length. Improvements were more prominent using global tracking with refined objectives, achieving an average fiber length from 10 to 17 mm, voxel-wise coverage of axonal tracts from 0.9 to 15%, and the correlation of target areas from 40 to 68%, while minimizing false positives and impossible cross-hemisphere connections. Optimized parameters showed good generalization capability for test brain samples in both experiments, demonstrating the flexible applicability of our framework to different tracking algorithms and objectives. These results indicate the importance of data-driven adjustment of fiber tracking algorithms and support the validity of dMRI-based tractography, if appropriate adjustments are employed.

The NanoZoomer artificial intelligence connectomics pipeline for tracer injection studies of the marmoset brain.

We describe our connectomics pipeline for processing anterograde tracer injection data for the brain of the common marmoset (Callithrix jacchus). Brain sections were imaged using a batch slide scanner (NanoZoomer 2.0-HT) and we used artificial intelligence to precisely segment the tracer signal from the background in the fluorescence images. The shape of each brain was reconstructed by reference to a block-face and all data were mapped into a common 3D brain space with atlas and 2D cortical flat map. To overcome the effect of using a single template atlas to specify cortical boundaries, brains were cyto- and myelo-architectonically annotated to create individual 3D atlases. Registration between the individual and common brain cortical boundaries in the flat map space was done to absorb the variation of each brain and precisely map all tracer injection data into one cortical brain space. We describe the methodology of our pipeline and analyze the accuracy of our tracer segmentation and brain registration approaches. Results show our pipeline can successfully process and normalize tracer injection experiments into a common space, making it suitable for large-scale connectomics studies with a focus on the cerebral cortex.

DeTerm: Software for automatic detection of neuronal dendritic branch terminals via an artificial neural network.

Dendrites of neurons receive and process synaptic or sensory inputs. The Drosophila class IV dendritic arborization (da) neuron is an established model system to explore molecular mechanisms of dendrite morphogenesis. The total number of dendritic branch terminals is one of the frequently employed parameters to characterize dendritic arborization complexity of class IV neurons. This parameter gives a useful phenotypic readout of arborization during neurogenesis, and it is typically determined by laborious manual analyses of numerous images. Ideally, an automated analysis would greatly reduce the workload; however, it is challenging to automatically discriminate dendritic branch terminals from signals of surrounding tissues in whole-mount live larvae. Here, we describe our newly developed software, called DeTerm, which automatically recognizes and quantifies dendrite branch terminals via an artificial neural network. Once we input an image file of a neuronal dendritic arbor and its region of interest information, DeTerm is capable of labeling terminals of larval class IV neurons with high precision, and it also provides positional data of individual terminals. We further show that DeTerm is applicable to other types of neurons, including mouse cerebellar Purkinje cells. DeTerm is freely available on the web and was successfully tested on Mac, Windows and Linux.

PAT-Probabilistic Axon Tracking for Densely Labeled Neurons in Large 3-D Micrographs.

A major goal of contemporary neuroscience research is to map the structural connectivity of mammalian brain using microscopy imaging data. In this context, the reconstruction of densely labeled axons from two-photon microscopy images is a challenging and important task. The visually overlapping, crossing, and often strongly distorted images of the axons allow many ambiguous interpretations to be made. We address the problem of tracking axons in densely labeled samples of neurons in large image data sets acquired from marmoset brains. Our high-resolution images were acquired using two-photon microscopy and they provided whole brain coverage, occupying terabytes of memory. Both the image distortions and the large data set size frequently make it impractical to apply present-day neuron tracing algorithms to such data due to the optimization of such algorithms to the precise tracing of either single or sparse sets of neurons. Thus, new tracking techniques are needed. We propose a probabilistic axon tracking algorithm (PAT). PAT tackles the tracking of axons in two steps: locally (L-PAT) and globally (G-PAT). L-PAT is a probabilistic tracking algorithm that can tackle distorted, cluttered images of densely labeled axons. L-PAT divides a large micrograph into smaller image stacks. It then processes each image stack independently before mapping the axons in each image to a sparse model of axon trajectories. G-PAT merges the sparse L-PAT models into a single global model of axon trajectories by minimizing a global objective function using a probabilistic optimization method. We demonstrate the superior performance of PAT over standard approaches on synthetic data. Furthermore, we successfully apply PAT to densely labeled axons in large images acquired from marmoset brains.

Modelfree global tractography.

Tractography based on diffusion-weighted MRI investigates the large scale arrangement of the neurite fibers in brain white matter. It is usually assumed that the signal is a convolution of a fiber specific response function (FRF) with a fiber orientation distribution (FOD). The FOD is the focus of tractography. While in the past the FRF was estimated beforehand and was usually assumed to be fix, more recent approaches estimate the response function during tractography. This work proposes a novel objective function independent of the FRF, just aiming for FOD reconstruction. The objective is integrated into global tractography showing promising results.

Efficient Monte Carlo image analysis for the location of vascular entity.

Tubular shaped networks appear not only in medical images like X-ray-, time-of-flight MRI- or CT-angiograms but also in microscopic images of neuronal networks. We present EMILOVE (Efficient Monte-carlo Image-analysis for the Location Of Vascular Entity), a novel modeling algorithm for tubular networks in biomedical images. The model is constructed using tablet shaped particles and edges connecting them. The particles encode the intrinsic information of tubular structure, including position, scale and orientation. The edges connecting the particles determine the topology of the networks. For simulated data, EMILOVE was able to accurately extract the tubular network. EMILOVE showed high performance in real data as well; it successfully modeled vascular networks in real cerebral X-ray and time-of-flight MRI angiograms. We also show some promising, preliminary results on microscopic images of neurons.

Rotation covariant image processing for biomedical applications.

With the advent of novel biomedical 3D image acquisition techniques, the efficient and reliable analysis of volumetric images has become more and more important. The amount of data is enormous and demands an automated processing. The applications are manifold, ranging from image enhancement, image reconstruction, and image description to object/feature detection and high-level contextual feature extraction. In most scenarios, it is expected that geometric transformations alter the output in a mathematically well-defined manner. In this paper we emphasis on 3D translations and rotations. Many algorithms rely on intensity or low-order tensorial-like descriptions to fulfill this demand. This paper proposes a general mathematical framework based on mathematical concepts and theories transferred from mathematical physics and harmonic analysis into the domain of image analysis and pattern recognition. Based on two basic operations, spherical tensor differentiation and spherical tensor multiplication, we show how to design a variety of 3D image processing methods in an efficient way. The framework has already been applied to several biomedical applications ranging from feature and object detection tasks to image enhancement and image restoration techniques. In this paper, the proposed methods are applied on a variety of different 3D data modalities stemming from medical and biological sciences.