Improving Image Segmentation with Contextual and Structural Similarity.

Deep learning models for medical image segmentation are usually trained with voxel-wise losses, e.g., cross-entropy loss, focusing on unary supervision without considering inter-voxel relationships. This oversight potentially leads to semantically inconsistent predictions. Here, we propose a contextual similarity loss (CSL) and a structural similarity loss (SSL) to explicitly and efficiently incorporate inter-voxel relationships for improved performance. The CSL promotes consistency in predicted object categories for each image sub-region compared to ground truth. The SSL enforces compatibility between the predictions of voxel pairs by computing pair-wise distances between them, ensuring that voxels of the same class are close together whereas those from different classes are separated by a wide margin in the distribution space. The effectiveness of the CSL and SSL is evaluated using a clinical cone-beam computed tomography (CBCT) dataset of patients with various craniomaxillofacial (CMF) deformities and a public pancreas dataset. Experimental results show that the CSL and SSL outperform state-of-the-art regional loss functions in preserving segmentation semantics.

Optimal shrinkage denoising breaks the noise floor in high-resolution diffusion MRI.

The spatial resolution attainable in diffusion magnetic resonance (MR) imaging is inherently limited by noise. The weaker signal associated with a smaller voxel size, especially at a high level of diffusion sensitization, is often buried under the noise floor owing to the non-Gaussian nature of the MR magnitude signal. Here, we show how the noise floor can be suppressed remarkably via optimal shrinkage of singular values associated with noise in complex-valued k-space data from multiple receiver channels. We explore and compare different low-rank signal matrix recovery strategies to utilize the inherently redundant information from multiple channels. In combination with background phase removal, the optimal strategy reduces the noise floor by 11 times. Our framework enables imaging with substantially improved resolution for precise characterization of tissue microstructure and white matter pathways without relying on expensive hardware upgrades and time-consuming acquisition repetitions, outperforming other related denoising methods.

Federated learning for medical image analysis: A survey.

Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.

Source-free unsupervised domain adaptation: A survey.

Unsupervised domain adaptation (UDA) via deep learning has attracted appealing attention for tackling domain-shift problems caused by distribution discrepancy across different domains. Existing UDA approaches highly depend on the accessibility of source domain data, which is usually limited in practical scenarios due to privacy protection, data storage and transmission cost, and computation burden. To tackle this issue, many source-free unsupervised domain adaptation (SFUDA) methods have been proposed recently, which perform knowledge transfer from a pre-trained source model to the unlabeled target domain with source data inaccessible. A comprehensive review of these works on SFUDA is of great significance. In this paper, we provide a timely and systematic literature review of existing SFUDA approaches from a technical perspective. Specifically, we categorize current SFUDA studies into two groups, i.e., white-box SFUDA and black-box SFUDA, and further divide them into finer subcategories based on different learning strategies they use. We also investigate the challenges of methods in each subcategory, discuss the advantages/disadvantages of white-box and black-box SFUDA methods, conclude the commonly used benchmark datasets, and summarize the popular techniques for improved generalizability of models learned without using source data. We finally discuss several promising future directions in this field.

A multimodal submillimeter MRI atlas of the human cerebellum.

The human cerebellum is engaged in a broad array of tasks related to motor coordination, cognition, language, attention, memory, and emotional regulation. A detailed cerebellar atlas can facilitate the investigation of the structural and functional organization of the cerebellum. However, existing cerebellar atlases are typically limited to a single imaging modality with insufficient characterization of tissue properties. Here, we introduce a multifaceted cerebellar atlas based on high-resolution multimodal MRI, facilitating the understanding of the neurodevelopment and neurodegeneration of the cerebellum based on cortical morphology, tissue microstructure, and intra-cerebellar and cerebello-cerebral connectivity.

Learning multi-site harmonization of magnetic resonance images without traveling human phantoms.

Harmonization improves Magn. Reson. Imaging (MRI) data consistency and is central to effective integration of diverse imaging data acquired across multiple sites. Recent deep learning techniques for harmonization are predominantly supervised in nature and hence require imaging data of the same human subjects to be acquired at multiple sites. Data collection as such requires the human subjects to travel across sites and is hence challenging, costly, and impractical, more so when sufficient sample size is needed for reliable network training. Here we show how harmonization can be achieved with a deep neural network that does not rely on traveling human phantom data. Our method disentangles site-specific appearance information and site-invariant anatomical information from images acquired at multiple sites and then employs the disentangled information to generate the image of each subject for any target site. We demonstrate with more than 6,000 multi-site T1- and T2-weighted images that our method is remarkably effective in generating images with realistic site-specific appearances without altering anatomical details. Our method allows retrospective harmonization of data in a wide range of existing modern large-scale imaging studies, conducted via different scanners and protocols, without additional data collection.

Aberrant dynamic functional network connectivity in type 2 diabetes mellitus individuals.

An increasing number of recent brain imaging studies are dedicated to understanding the neuro mechanism of cognitive impairment in type 2 diabetes mellitus (T2DM) individuals. In contrast to efforts to date that are limited to static functional connectivity, here we investigate abnormal connectivity in T2DM individuals by characterizing the time-varying properties of brain functional networks. Using group independent component analysis (GICA), sliding-window analysis, and k-means clustering, we extracted thirty-one intrinsic connectivity networks (ICNs) and estimated four recurring brain states. We observed significant group differences in fraction time (FT) and mean dwell time (MDT), and significant negative correlation between the Montreal Cognitive Assessment (MoCA) scores and FT/MDT. We found that in the T2DM group the inter- and intra-network connectivity decreases and increases respectively for the default mode network (DMN) and task-positive network (TPN). We also found alteration in the precuneus network (PCUN) and enhanced connectivity between the salience network (SN) and the TPN. Our study provides evidence of alterations of large-scale resting networks in T2DM individuals and shed light on the fundamental mechanisms of neurocognitive deficits in T2DM.

Organ-aware CBCT enhancement via dual path learning for prostate cancer treatment.

BACKGROUND: Cone-beam computed tomography (CBCT) plays a crucial role in the intensity modulated radiotherapy (IMRT) of prostate cancer. However, poor image contrast and fuzzy organ boundaries pose challenges to precise targeting for dose delivery and plan reoptimization for adaptive therapy. PURPOSE: In this work, we aim to enhance pelvic CBCT images by translating them to high-quality CT images with a particular focus on the anatomical structures important for radiotherapy. METHODS: We develop a novel dual-path learning framework, covering both global and local information, for organ-aware enhancement of the prostate, bladder and rectum. The global path learns coarse inter-modality translation at the image level. The local path learns organ-aware translation at the regional level. This dual-path learning architecture can serve as a plug-and-play module adaptable to other medical image-to-image translation frameworks. RESULTS: We evaluated the performance of the proposed method both quantitatively and qualitatively. The training dataset consists of unpaired 40 CBCT and 40 CT scans, the validation dataset consists of 5 paired CBCT-CT scans, and the testing dataset consists of 10 paired CBCT-CT scans. The peak signal-to-noise ratio (PSNR) between enhanced CBCT and reference CT images is 27.22 ± 1.79, and the structural similarity (SSIM) between enhanced CBCT and the reference CT images is 0.71 ± 0.03. We also compared our method with state-of-the-art image-to-image translation methods, where our method achieves the best performance. Moreover, the statistical analysis confirms that the improvements achieved by our method are statistically significant. CONCLUSIONS: The proposed method demonstrates its superiority in enhancing pelvic CBCT images, especially at the organ level, compared to relevant methods.

Longitudinal Prediction of Postnatal Brain Magnetic Resonance Images via a Metamorphic Generative Adversarial Network.

Missing scans are inevitable in longitudinal studies due to either subject dropouts or failed scans. In this paper, we propose a deep learning framework to predict missing scans from acquired scans, catering to longitudinal infant studies. Prediction of infant brain MRI is challenging owing to the rapid contrast and structural changes particularly during the first year of life. We introduce a trustworthy metamorphic generative adversarial network (MGAN) for translating infant brain MRI from one time-point to another. MGAN has three key features: (i) Image translation leveraging spatial and frequency information for detail-preserving mapping; (ii) Quality-guided learning strategy that focuses attention on challenging regions. (iii) Multi-scale hybrid loss function that improves translation of image contents. Experimental results indicate that MGAN outperforms existing GANs by accurately predicting both tissue contrasts and anatomical details.

Mapping the evolution of regional brain network efficiency and its association with cognitive abilities during the first twenty-eight months of life.

Human brain undergoes rapid growth during the first few years of life. While previous research has employed graph theory to study early brain development, it has mostly focused on the topological attributes of the whole brain. However, examining regional graph-theory features may provide unique insights into the development of cognitive abilities. Utilizing a large and longitudinal rsfMRI dataset from the UNC/UMN Baby Connectome Project, we investigated the developmental trajectories of regional efficiency and evaluated the relationships between these changes and cognitive abilities using Mullen Scales of Early Learning during the first twenty-eight months of life. Our results revealed a complex and spatiotemporally heterogeneous development pattern of regional global and local efficiency during this age period. Furthermore, we found that the trajectories of the regional global efficiency at the left temporal occipital fusiform and bilateral occipital fusiform gyri were positively associated with cognitive abilities, including visual reception, expressive language, receptive language, and early learning composite scores (P < 0.05, FDR corrected). However, these associations were weakened with age. These findings offered new insights into the regional developmental features of brain topologies and their associations with cognition and provided evidence of ongoing optimization of brain networks at both whole-brain and regional levels.

A paired dataset of T1- and T2-weighted MRI at 3 Tesla and 7 Tesla.

Brain magnetic resonance imaging (MRI) provides detailed soft tissue contrasts that are critical for disease diagnosis and neuroscience research. Higher MRI resolution typically comes at the cost of signal-to-noise ratio (SNR) and tissue contrast, particularly for more common 3 Tesla (3T) MRI scanners. At ultra-high magnetic field strength, 7 Tesla (7T) MRI allows for higher resolution with greater tissue contrast and SNR. However, the prohibitively high costs of 7T MRI scanners deter their widespread adoption in clinical and research centers. To obtain higher-quality images without 7T MRI scanners, algorithms that can synthesize 7T MR images from 3T MR images are under active development. Here, we make available a dataset of paired T1-weighted and T2-weighted MR images at 3T and 7T of 10 healthy subjects to facilitate the development and evaluation of 3T-to-7T MR image synthesis models. The quality of the dataset is assessed using image quality metrics implemented in MRIQC.

Attention-Guided Autoencoder for Automated Progression Prediction of Subjective Cognitive Decline With Structural MRI.

Subjective cognitive decline (SCD) is the preclinical stage of Alzheimer's disease (AD) which happens even earlier than mild cognitive impairment (MCI). Progressive SCD will convert to MCI with the potential of further evolving to AD. Therefore, early identification of progressive SCD with neuroimaging techniques (e.g., structural MRI) is of great clinical value for early intervention of AD. However, existing MRI-based machine/deep learning methods usually suffer the small-sample-size problem and lack interpretability. To this end, we propose an interpretable autoencoder model with domain transfer learning (IADT) for progression prediction of SCD. Firstly, the proposed model can leverage MRIs from both the target domain (i.e., SCD) and auxiliary domains (e.g., AD and NC) for progressive SCD identification. Besides, it can automatically locate the disease-related brain regions of interest (defined in brain atlases) through an attention mechanism, which shows good interpretability. In addition, the IADT model is straightforward to train and test with only 5  âˆ¼ 10 seconds on CPUs and is suitable for medical tasks with small datasets. Extensive experiments on the publicly available ADNI dataset and a private CLAS dataset have demonstrated the effectiveness of the proposed method.

Detecting type 2 diabetes mellitus cognitive impairment using whole-brain functional connectivity.

Type 2 diabetes mellitus (T2DM) is closely linked to cognitive decline and alterations in brain structure and function. Resting-state functional magnetic resonance imaging (rs-fMRI) is used to diagnose neurodegenerative diseases, such as cognitive impairment (CI), Alzheimer's disease (AD), and vascular dementia (VaD). However, whether the functional connectivity (FC) of patients with T2DM and mild cognitive impairment (T2DM-MCI) is conducive to early diagnosis remains unclear. To answer this question, we analyzed the rs-fMRI data of 37 patients with T2DM and mild cognitive impairment (T2DM-MCI), 93 patients with T2DM but no cognitive impairment (T2DM-NCI), and 69 normal controls (NC). We achieved an accuracy of 87.91% in T2DM-MCI versus T2DM-NCI classification and 80% in T2DM-NCI versus NC classification using the XGBoost model. The thalamus, angular, caudate nucleus, and paracentral lobule contributed most to the classification outcome. Our findings provide valuable knowledge to classify and predict T2DM-related CI, can help with early clinical diagnosis of T2DM-MCI, and provide a basis for future studies.

Accumulation of network redundancy marks the early stage of Alzheimer's disease.

Brain wiring redundancy counteracts aging-related cognitive decline by reserving additional communication channels as a neuroprotective mechanism. Such a mechanism plays a potentially important role in maintaining cognitive function during the early stages of neurodegenerative disorders such as Alzheimer's disease (AD). AD is characterized by severe cognitive decline and involves a long prodromal stage of mild cognitive impairment (MCI). Since MCI subjects are at high risk of converting to AD, identifying MCI individuals is essential for early intervention. To delineate the redundancy profile during AD progression and enable better MCI diagnosis, we define a metric that reflects redundant disjoint connections between brain regions and extract redundancy features in three high-order brain networks-medial frontal, frontoparietal, and default mode networks-based on dynamic functional connectivity (dFC) captured by resting-state functional magnetic resonance imaging (rs-fMRI). We show that redundancy increases significantly from normal control (NC) to MCI individuals and decreases slightly from MCI to AD individuals. We further demonstrate that statistical features of redundancy are highly discriminative and yield state-of-the-art accuracy of up to 96.8 ± 1.0% in support vector machine (SVM) classification between NC and MCI individuals. This study provides evidence supporting the notion that redundancy serves as a crucial neuroprotective mechanism in MCI.

Feasibility of free-breathing 19 F MRI image acquisition to characterize ventilation defects in CF and healthy volunteers at wash-in.

PURPOSE: To explore the feasibility of measuring ventilation defect percentage (VDP) using 19 F MRI during free-breathing wash-in of fluorinated gas mixture with postacquisition denoising and to compare these results with those obtained through traditional Cartesian breath-hold acquisitions. METHODS: Eight adults with cystic fibrosis and 5 healthy volunteers completed a single MR session on a Siemens 3T Prisma. 1 H Ultrashort-TE MRI sequences were used for registration and masking, and ventilation images with 19 F MRI were obtained while the subjects breathed a normoxic mixture of 79% perfluoropropane and 21% oxygen (O2 ). 19 F MRI was performed during breath holds and while free breathing with one overlapping spiral scan at breath hold for VDP value comparison. The 19 F spiral data were denoised using a low-rank matrix recovery approach. RESULTS: VDP measured using 19 F VIBE and 19 F spiral images were highly correlated (r = 0.84) at 10 wash-in breaths. Second-breath VDPs were also highly correlated (r = 0.88). Denoising greatly increased SNR (pre-denoising spiral SNR, 2.46 ± 0.21; post-denoising spiral SNR, 33.91 ± 6.12; and breath-hold SNR, 17.52 ± 2.08). CONCLUSION: Free-breathing 19 F lung MRI VDP analysis was feasible and highly correlated with breath-hold measurements. Free-breathing methods are expected to increase patient comfort and extend ventilation MRI use to patients who are unable to perform breath holds, including younger subjects and those with more severe lung disease.

Deep learning prediction of diffusion MRI data with microstructure-sensitive loss functions.

Deep learning prediction of diffusion MRI (DMRI) data relies on the utilization of effective loss functions. Existing losses typically measure the signal-wise differences between the predicted and target DMRI data without considering the quality of derived diffusion scalars that are eventually utilized for quantification of tissue microstructure. Here, we propose two novel loss functions, called microstructural loss and spherical variance loss, to explicitly consider the quality of both the predicted DMRI data and derived diffusion scalars. We apply these loss functions to the prediction of multi-shell data and enhancement of angular resolution. Evaluation based on infant and adult DMRI data indicates that both microstructural loss and spherical variance loss improve the quality of derived diffusion scalars.

Multifaceted atlases of the human brain in its infancy.

Brain atlases are spatial references for integrating, processing, and analyzing brain features gathered from different individuals, sources, and scales. Here we introduce a collection of joint surface-volume atlases that chart postnatal development of the human brain in a spatiotemporally dense manner from two weeks to two years of age. Our month-specific atlases chart normative patterns and capture key traits of early brain development and are therefore conducive to identifying aberrations from normal developmental trajectories. These atlases will enhance our understanding of early structural and functional development by facilitating the mapping of diverse features of the infant brain to a common reference frame for precise multifaceted quantification of cortical and subcortical changes.

Bidirectional prediction of facial and bony shapes for orthognathic surgical planning.

This paper proposes a deep learning framework to encode subject-specific transformations between facial and bony shapes for orthognathic surgical planning. Our framework involves a bidirectional point-to-point convolutional network (P2P-Conv) to predict the transformations between facial and bony shapes. P2P-Conv is an extension of the state-of-the-art P2P-Net and leverages dynamic point-wise convolution (i.e., PointConv) to capture local-to-global spatial information. Data augmentation is carried out in the training of P2P-Conv with multiple point subsets from the facial and bony shapes. During inference, network outputs generated for multiple point subsets are combined into a dense transformation. Finally, non-rigid registration using the coherent point drift (CPD) algorithm is applied to generate surface meshes based on the predicted point sets. Experimental results on real-subject data demonstrate that our method substantially improves the prediction of facial and bony shapes over state-of-the-art methods.

Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

High-Resolution 3D Magnetic Resonance Fingerprinting With a Graph Convolutional Network.

Magnetic resonance fingerprinting (MRF) is a novel quantitative imaging framework for rapid and simultaneous quantification of multiple tissue properties. 3D MRF allows higher through-plane resolution, but the acquisition process is slow when whole-brain coverage is needed. Existing methods for acceleration mainly rely on GRAPPA for k-space interpolation in the partition-encoding direction, limiting the acceleration factor to 2 or 3. In this work, we replace GRAPPA with a deep learning approach for accurate tissue quantification with greater acceleration. Specifically, a graph convolution network (GCN) is developed to cater to the non-Cartesian spiral sampling trajectories typical in MRF acquisition. The GCN maintains high quantification accuracy with up to 6-fold acceleration and allows 1mm isotropic resolution whole-brain 3D MRF data to be acquired in 3min and submillimeter 3D MRF (0.8mm) in 5min, greatly improving the feasibility of MRF in clinical settings.