Empirical assessment of the assumptions of ComBat with diffusion tensor imaging.

Purpose: Diffusion tensor imaging (DTI) is a magnetic resonance imaging technique that provides unique information about white matter microstructure in the brain but is susceptible to confounding effects introduced by scanner or acquisition differences. ComBat is a leading approach for addressing these site biases. However, despite its frequent use for harmonization, ComBat's robustness toward site dissimilarities and overall cohort size have not yet been evaluated in terms of DTI. Approach: As a baseline, we match N=358 participants from two sites to create a "silver standard" that simulates a cohort for multi-site harmonization. Across sites, we harmonize mean fractional anisotropy and mean diffusivity, calculated using participant DTI data, for the regions of interest defined by the JHU EVE-Type III atlas. We bootstrap 10 iterations at 19 levels of total sample size, 10 levels of sample size imbalance between sites, and 6 levels of mean age difference between sites to quantify (i) ßAGE, the linear regression coefficient of the relationship between FA and age; (ii) Î³/f*, the ComBat-estimated site-shift; and (iii) Î´/f*, the ComBat-estimated site-scaling. We characterize the reliability of ComBat by evaluating the root mean squared error in these three metrics and examine if there is a correlation between the reliability of ComBat and a violation of assumptions. Results: ComBat remains well behaved for ßAGE when N>162 and when the mean age difference is less than 4 years. The assumptions of the ComBat model regarding the normality of residual distributions are not violated as the model becomes unstable. Conclusion: Prior to harmonization of DTI data with ComBat, the input cohort should be examined for size and covariate distributions of each site. Direct assessment of residual distributions is less informative on stability than bootstrap analysis. We caution use ComBat of in situations that do not conform to the above thresholds.

Sex and age effects on gray matter volume trajectories in young children with prenatal alcohol exposure.

Prenatal alcohol exposure (PAE) occurs in ~11% of North American pregnancies and is the most common known cause of neurodevelopmental disabilities such as fetal alcohol spectrum disorder (FASD; ~2-5% prevalence). PAE has been consistently associated with smaller gray matter volumes in children, adolescents, and adults. A small number of longitudinal studies show altered gray matter development trajectories in late childhood/early adolescence, but patterns in early childhood and potential sex differences have not been characterized in young children. Using longitudinal T1-weighted MRI, the present study characterized gray matter volume development in young children with PAE (N = 42, 84 scans, ages 3-8 years) compared to unexposed children (N = 127, 450 scans, ages 2-8.5 years). Overall, we observed altered global and regional gray matter development trajectories in the PAE group, wherein they had attenuated age-related increases and more volume decreases relative to unexposed children. Moreover, we found more pronounced sex differences in children with PAE; females with PAE having the smallest gray matter volumes and the least age-related changes of all groups. This pattern of altered development may indicate reduced brain plasticity and/or accelerated maturation and may underlie the cognitive/behavioral difficulties often experienced by children with PAE. In conjunction with previous research on older children, adolescents, and adults with PAE, our results suggest that gray matter volume differences associated with PAE vary by age and may become more apparent in older children.

Deep conditional generative model for longitudinal single-slice abdominal computed tomography harmonization.

Purpose: Two-dimensional single-slice abdominal computed tomography (CT) provides a detailed tissue map with high resolution allowing quantitative characterization of relationships between health conditions and aging. However, longitudinal analysis of body composition changes using these scans is difficult due to positional variation between slices acquired in different years, which leads to different organs/tissues being captured. Approach: To address this issue, we propose C-SliceGen, which takes an arbitrary axial slice in the abdominal region as a condition and generates a pre-defined vertebral level slice by estimating structural changes in the latent space. Results: Our experiments on 2608 volumetric CT data from two in-house datasets and 50 subjects from the 2015 Multi-Atlas Abdomen Labeling Challenge Beyond the Cranial Vault (BTCV) dataset demonstrate that our model can generate high-quality images that are realistic and similar. We further evaluate our method's capability to harmonize longitudinal positional variation on 1033 subjects from the Baltimore longitudinal study of aging dataset, which contains longitudinal single abdominal slices, and confirmed that our method can harmonize the slice positional variance in terms of visceral fat area. Conclusion: This approach provides a promising direction for mapping slices from different vertebral levels to a target slice and reducing positional variance for single-slice longitudinal analysis. The source code is available at: https://github.com/MASILab/C-SliceGen.

Curating retrospective multimodal and longitudinal data for community cohorts at risk for lung cancer.

BACKGROUND: Large community cohorts are useful for lung cancer research, allowing for the analysis of risk factors and development of predictive models. OBJECTIVE: A robust methodology for (1) identifying lung cancer and pulmonary nodules diagnoses as well as (2) associating multimodal longitudinal data with these events from electronic health record (EHRs) is needed to optimally curate cohorts at scale. METHODS: In this study, we leveraged (1) SNOMED concepts to develop ICD-based decision rules for building a cohort that captured lung cancer and pulmonary nodules and (2) clinical knowledge to define time windows for collecting longitudinal imaging and clinical concepts. We curated three cohorts with clinical data and repeated imaging for subjects with pulmonary nodules from our Vanderbilt University Medical Center. RESULTS: Our approach achieved an estimated sensitivity 0.930 (95% CI: [0.879, 0.969]), specificity of 0.996 (95% CI: [0.989, 1.00]), positive predictive value of 0.979 (95% CI: [0.959, 1.000]), and negative predictive value of 0.987 (95% CI: [0.976, 0.994]) for distinguishing lung cancer from subjects with SPNs. CONCLUSION: This work represents a general strategy for high-throughput curation of multi-modal longitudinal cohorts at risk for lung cancer from routinely collected EHRs.

Quantification of mediation effects of white matter functional characteristics on cognitive decline in aging.

Cognitive decline with aging involves multifactorial processes, including changes in brain structure and function. This study focuses on the role of white matter functional characteristics, as reflected in blood oxygenation level-dependent signals, in age-related cognitive deterioration. Building on previous research confirming the reproducibility and age-dependence of blood oxygenation level-dependent signals acquired via functional magnetic resonance imaging, we here employ mediation analysis to test if aging affects cognition through white matter blood oxygenation level-dependent signal changes, impacting various cognitive domains and specific white matter regions. We used independent component analysis of resting-state blood oxygenation level-dependent signals to segment white matter into coherent hubs, offering a data-driven view of white matter's functional architecture. Through correlation analysis, we constructed a graph network and derived metrics to quantitatively assess regional functional properties based on resting-state blood oxygenation level-dependent fluctuations. Our analysis identified significant mediators in the age-cognition relationship, indicating that aging differentially influences cognitive functions by altering the functional characteristics of distinct white matter regions. These findings enhance our understanding of the neurobiological basis of cognitive aging, highlighting the critical role of white matter in maintaining cognitive integrity and proposing new approaches to assess interventions targeting cognitive decline in older populations.

DeepN4: Learning N4ITK Bias Field Correction for T1-weighted Images.

T1-weighted (T1w) MRI has low frequency intensity artifacts due to magnetic field inhomogeneities. Removal of these biases in T1w MRI images is a critical preprocessing step to ensure spatially consistent image interpretation. N4ITK bias field correction, the current state-of-the-art, is implemented in such a way that makes it difficult to port between different pipelines and workflows, thus making it hard to reimplement and reproduce results across local, cloud, and edge platforms. Moreover, N4ITK is opaque to optimization before and after its application, meaning that methodological development must work around the inhomogeneity correction step. Given the importance of bias fields correction in structural preprocessing and flexible implementation, we pursue a deep learning approximation / reinterpretation of the N4ITK bias fields correction to create a method which is portable, flexible, and fully differentiable. In this paper, we trained a deep learning network "DeepN4" on eight independent cohorts from 72 different scanners and age ranges with N4ITK-corrected T1w MRI and bias field for supervision in log space. We found that we can closely approximate N4ITK bias fields correction with naïve networks. We evaluate the peak signal to noise ratio (PSNR) in test dataset against the N4ITK corrected images. The median PSNR of corrected images between N4ITK and DeepN4 was 47.96 dB. In addition, we assess the DeepN4 model on eight additional external datasets and show the generalizability of the approach. This study establishes that incompatible N4ITK preprocessing steps can be closely approximated by naïve deep neural networks, facilitating more flexibility. All code and models are released at https://github.com/MASILab/DeepN4 .

Assessing depression recurrence, cognitive burden, and neurobiological homeostasis in late life: Design and rationale of the REMBRANDT Study.

Background: Late-life depression is characterized by disability, cognitive impairment and decline, and a high risk of recurrence following remission. Aside from past psychiatric history, prognostic neurobiological and clinical factors influencing recurrence risk are unclear. Moreover, it is unclear if cognitive impairment predisposes to recurrence, or whether recurrent episodes may accelerate brain aging and cognitive decline. The purpose of the REMBRANDT study (Recurrence markers, cognitive burden, and neurobiological homeostasis in late-life depression) is to better elucidate these relationships and identify phenotypic, cognitive, environmental, and neurobiological factors contributing to and predictive of depression recurrence. Methods: Across three sites, REMBRANDT will enroll 300 depressed elders who will receive antidepressant treatment. The goal is to enroll 210 remitted depressed participants and 75 participants with no mental health history into a two-year longitudinal phase focusing on depression recurrence. Participants are evaluated every 2 months with deeper assessments occurring every 8 months, including structural and functional neuroimaging, environmental stress assessments, deep symptom phenotyping, and two weeks of 'burst' ecological momentary assessments to elucidate variability in symptoms and cognitive performance. A broad neuropsychological test battery is completed at the beginning and end of the longitudinal study. Significance: REMBRANDT will improve our understanding of how alterations in neural circuits and cognition that persist during remission contribute to depression recurrence vulnerability. It will also elucidate how these processes may contribute to cognitive impairment and decline. This project will obtain deep phenotypic data that will help identify vulnerability and resilience factors that can help stratify individual clinical risk.

Volumetric brain MRI signatures of heart failure with preserved ejection fraction in the setting of dementia.

Heart failure with preserved ejection fraction (HFpEF) is an important, emerging risk factor for dementia, but it is not clear whether HFpEF contributes to a specific pattern of neuroanatomical changes in dementia. A major challenge to studying this is the relative paucity of datasets of patients with dementia, with/without HFpEF, and relevant neuroimaging. We sought to demonstrate the feasibility of using modern data mining tools to create and analyze clinical imaging datasets and identify the neuroanatomical signature of HFpEF-associated dementia. We leveraged the bioinformatics tools at Vanderbilt University Medical Center to identify patients with a diagnosis of dementia with and without comorbid HFpEF using the electronic health record. We identified high resolution, clinically-acquired neuroimaging data on 30 dementia patients with HFpEF (age 76.9 ± 8.12 years, 61% female) as well as 301 age- and sex-matched patients with dementia but without HFpEF to serve as comparators (age 76.2 ± 8.52 years, 60% female). We used automated image processing pipelines to parcellate the brain into 132 structures and quantify their volume. We found six regions with significant atrophy associated with HFpEF: accumbens area, amygdala, posterior insula, anterior orbital gyrus, angular gyrus, and cerebellar white matter. There were no regions with atrophy inversely associated with HFpEF. Patients with dementia and HFpEF have a distinct neuroimaging signature compared to patients with dementia only. Five of the six regions identified in are in the temporo-parietal region of the brain. Future studies should investigate mechanisms of injury associated with cerebrovascular disease leading to subsequent brain atrophy.

MidRISH: Unbiased harmonization of rotationally invariant harmonics of the diffusion signal.

Data harmonization is necessary for removing confounding effects in multi-site diffusion image analysis. One such harmonization method, LinearRISH, scales rotationally invariant spherical harmonic (RISH) features from one site ("target") to the second ("reference") to reduce confounding scanner effects. However, reference and target site designations are not arbitrary and resultant diffusion metrics (fractional anisotropy, mean diffusivity) are biased by this choice. In this work we propose MidRISH: rather than scaling reference RISH features to target RISH features, we project both sites to a mid-space. We validate MidRISH with the following experiments: harmonizing scanner differences from 37 matched patients free of cognitive impairment, and harmonizing acquisition and study differences on 117 matched patients free of cognitive impairment. We find that MidRISH reduces bias of reference selection while preserving harmonization efficacy of LinearRISH. Users should be cautious when performing LinearRISH harmonization. To select a reference site is to choose diffusion metric effect-size. Our proposed method eliminates the bias-inducing site selection step.

Cross-scale multi-instance learning for pathological image diagnosis.

Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20× magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Lung CT harmonization of paired reconstruction kernel images using generative adversarial networks.

BACKGROUND: The kernel used in CT image reconstruction is an important factor that determines the texture of the CT image. Consistency of reconstruction kernel choice is important for quantitative CT-based assessment as kernel differences can lead to substantial shifts in measurements unrelated to underlying anatomical structures. PURPOSE: In this study, we investigate kernel harmonization in a multi-vendor low-dose CT lung cancer screening cohort and evaluate our approach's validity in quantitative CT-based assessments. METHODS: Using the National Lung Screening Trial, we identified CT scan pairs of the same sessions with one reconstructed from a soft tissue kernel and one from a hard kernel. In total, 1000 pairs of five different paired kernel types (200 each) were identified. We adopt the pix2pix architecture to train models for kernel conversion. Each model was trained on 100 pairs and evaluated on 100 withheld pairs. A total of 10 models were implemented. We evaluated the efficacy of kernel conversion based on image similarity metrics including root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM) as well as the capability of the models to reduce measurement shifts in quantitative emphysema and body composition measurements. Additionally, we study the reproducibility of standard radiomic features for all kernel pairs before and after harmonization. RESULTS: Our approach effectively converts CT images from one kernel to another in all paired kernel types, as indicated by the reduction in RMSE (p < 0.05) and an increase in the PSNR (p < 0.05) and SSIM (p < 0.05) for both directions of conversion for all pair types. In addition, there is an increase in the agreement for percent emphysema, skeletal muscle area, and subcutaneous adipose tissue (SAT) area for both directions of conversion. Furthermore, radiomic features were reproducible when compared with the ground truth features. CONCLUSIONS: Kernel conversion using deep learning reduces measurement variation in percent emphysema, muscle area, and SAT area.

Influence of preprocessing, distortion correction and cardiac triggering on the quality of diffusion MR images of spinal cord.

Diffusion MRI of the spinal cord (SC) is susceptible to geometric distortion caused by field inhomogeneities, and prone to misalignment across time series and signal dropout caused by biological motion. Several modifications of image acquisition and image processing techniques have been introduced to overcome these artifacts, but their specific benefits are largely unproven and warrant further investigations. We aim to evaluate two specific aspects of image acquisition and processing that address image quality in diffusion studies of the spinal cord: susceptibility corrections to reduce geometric distortions, and cardiac triggering to minimize motion artifacts. First, we evaluate 4 distortion preprocessing strategies on 7 datasets of the cervical and lumbar SC and find that while distortion correction techniques increase geometric similarity to structural images, they are largely driven by the high-contrast cerebrospinal fluid, and do not consistently improve the geometry within the cord nor improve white-to-gray matter contrast. We recommend at a minimum to perform bulk-motion correction in preprocessing and posit that improvements/adaptations are needed for spinal cord distortion preprocessing algorithms, which are currently optimized and designed for brain imaging. Second, we design experiments to evaluate the impact of removing cardiac triggering. We show that when triggering is foregone, images are qualitatively similar to triggered sequences, do not have increased prevalence of artifacts, and result in similar diffusion tensor indices with similar reproducibility to triggered acquisitions. When triggering is removed, much shorter acquisitions are possible, which are also qualitatively and quantitatively similar to triggered sequences. We suggest that removing cardiac triggering for cervical SC diffusion can be a reasonable option to save time with minimal sacrifice to image quality.

Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Metrics reloaded: recommendations for image analysis validation.

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Evaluation of Mean Shift, ComBat, and CycleGAN for Harmonizing Brain Connectivity Matrices Across Sites.

Connectivity matrices derived from diffusion MRI (dMRI) provide an interpretable and generalizable way of understanding the human brain connectome. However, dMRI suffers from inter-site and between-scanner variation, which impedes analysis across datasets to improve robustness and reproducibility of results. To evaluate different harmonization approaches on connectivity matrices, we compared graph measures derived from these matrices before and after applying three harmonization techniques: mean shift, ComBat, and CycleGAN. The sample comprises 168 age-matched, sex-matched normal subjects from two studies: the Vanderbilt Memory and Aging Project (VMAP) and the Biomarkers of Cognitive Decline Among Normal Individuals (BIOCARD). First, we plotted the graph measures and used coefficient of variation (CoV) and the Mann-Whitney U test to evaluate different methods' effectiveness in removing site effects on the matrices and the derived graph measures. ComBat effectively eliminated site effects for global efficiency and modularity and outperformed the other two methods. However, all methods exhibited poor performance when harmonizing average betweenness centrality. Second, we tested whether our harmonization methods preserved correlations between age and graph measures. All methods except for CycleGAN in one direction improved correlations between age and global efficiency and between age and modularity from insignificant to significant with p-values less than 0.05.

Identification of functional white matter networks in BOLD fMRI.

White matter signals in resting state blood oxygen level dependent functional magnetic resonance (BOLD-fMRI) have been largely discounted, yet there is growing evidence that these signals are indicative of brain activity. Understanding how these white matter signals capture function can provide insight into brain physiology. Moreover, functional signals could potentially be used as early markers for neurological changes, such as in Alzheimer's Disease. To investigate white matter brain networks, we leveraged the OASIS-3 dataset to extract white matter signals from resting state BOLD-FMRI data on 711 subjects. The imaging was longitudinal with a total of 2,026 images. Hierarchical clustering was performed to investigate clusters of voxel-level correlations on the timeseries data. The stability of clusters was measured with the average Dice coefficients on two different cross fold validations. The first validated the stability between scans, and the second validated the stability between subject populations. Functional clusters at hierarchical levels 4, 9, 13, 18, and 24 had local maximum stability, suggesting better clustered white matter. In comparison with JHU-DTI-SS Type-I Atlas defined regions, clusters at lower hierarchical levels identified well defined anatomical lobes. At higher hierarchical levels, functional clusters mapped motor and memory functional regions, identifying 50.00%, 20.00%, 27.27%, and 35.14% of the frontal, occipital, parietal, and temporal lobe regions respectively.

Perivascular space burden interacts with APOE-ε4 status on cognition in older adults.

Enlarged perivascular spaces (ePVS) may adversely affect cognition. Little is known about how basal ganglia ePVS interact with apolipoprotein (APOE)-ε4 status. Vanderbilt Memory and Aging Project participants (n = 326, 73 ± 7, 59% male) underwent 3 T brain MRI at baseline to assess ePVS and longitudinal neuropsychological assessments. The interaction between ePVS volume and APOE-ε4 carrier status was related to baseline outcomes using ordinary least squares regressions and longitudinal cognition using linear mixed-effects regressions. ePVS volume interacted with APOE-ε4 status on cross-sectional naming performance (ß = -0.002, p = 0.002), and executive function excluding outliers (ß = 0.001, p = 0.009). There were no significant longitudinal interactions (p-values>0.10) except for Coding excluding outliers (ß = 0.002, p = 0.05). While cross-sectional models stratified by APOE-ε4 status indicated greater ePVS related to worse cognition mostly in APOE-ε4 carriers, longitudinal models stratified by APOE-ε4 status showed greater ePVS volume related to worse cognition among APOE-ε4 non-carriers only. Results indicated that greater ePVS volume interacts with APOE-ε4 status on cognition cross-sectionally. Longitudinally, the association of greater ePVS volume and worse cognition appears stronger in APOE-ε4 non-carriers, possibly due to the deleterious effects of APOE-ε4 on cognition across the lifespan.

Robust fiber orientation distribution function estimation using deep constrained spherical deconvolution for diffusion-weighted magnetic resonance imaging.

Purpose: Diffusion-weighted magnetic resonance imaging (DW-MRI) is a critical imaging method for capturing and modeling tissue microarchitecture at a millimeter scale. A common practice to model the measured DW-MRI signal is via fiber orientation distribution function (fODF). This function is the essential first step for the downstream tractography and connectivity analyses. With recent advantages in data sharing, large-scale multisite DW-MRI datasets are being made available for multisite studies. However, measurement variabilities (e.g., inter- and intrasite variability, hardware performance, and sequence design) are inevitable during the acquisition of DW-MRI. Most existing model-based methods [e.g., constrained spherical deconvolution (CSD)] and learning-based methods (e.g., deep learning) do not explicitly consider such variabilities in fODF modeling, which consequently leads to inferior performance on multisite and/or longitudinal diffusion studies. Approach: In this paper, we propose a data-driven deep CSD method to explicitly constrain the scan-rescan variabilities for a more reproducible and robust estimation of brain microstructure from repeated DW-MRI scans. Specifically, the proposed method introduces a three-dimensional volumetric scanner-invariant regularization scheme during the fODF estimation. We study the Human Connectome Project (HCP) young adults test-retest group as well as the MASiVar dataset (with inter- and intrasite scan/rescan data). The Baltimore Longitudinal Study of Aging dataset is employed for external validation. Results: From the experimental results, the proposed data-driven framework outperforms the existing benchmarks in repeated fODF estimation. By introducing the contrastive loss with scan/rescan data, the proposed method achieved a higher consistency while maintaining higher angular correlation coefficients with the CSD modeling. The proposed method is assessing the downstream connectivity analysis and shows increased performance in distinguishing subjects with different biomarkers. Conclusion: We propose a deep CSD method to explicitly reduce the scan-rescan variabilities, so as to model a more reproducible and robust brain microstructure from repeated DW-MRI scans. The plug-and-play design of the proposed approach is potentially applicable to a wider range of data harmonization problems in neuroimaging.

Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

Characterizing patterns of DTI variance in aging brains.

Background: As large analyses merge data across sites, a deeper understanding of variance in statistical assessment across the sources of data becomes critical for valid analyses. Diffusion tensor imaging (DTI) exhibits spatially varying and correlated noise, so care must be taken with distributional assumptions. Purpose: We characterize the role of physiology, subject compliance, and the interaction of subject with the scanner in the understanding of DTI variability, as modeled in spatial variance of derived metrics in homogeneous regions. Methods: We analyze DTI data from 1035 subjects in the Baltimore Longitudinal Study of Aging (BLSA), with ages ranging from 22.4 to 103 years old. For each subject, up to 12 longitudinal sessions were conducted. We assess variance of DTI scalars within regions of interest (ROIs) defined by four segmentation methods and investigate the relationships between the variance and covariates, including baseline age, time from the baseline (referred to as "interval"), motion, sex, and whether it is the first scan or the second scan in the session. Results: Covariate effects are heterogeneous and bilaterally symmetric across ROIs. Inter-session interval is positively related (p ≪ 0.001) to FA variance in the cuneus and occipital gyrus, but negatively (p ≪ 0.001) in the caudate nucleus. Males show significantly (p ≪ 0.001) higher FA variance in the right putamen, thalamus, body of the corpus callosum, and cingulate gyrus. In 62 out of 176 ROIs defined by the Eve type-1 atlas, an increase in motion is associated (p < 0.05) with a decrease in FA variance. Head motion increases during the rescan of DTI (Δµ = 0.045 millimeters per volume). Conclusions: The effects of each covariate on DTI variance, and their relationships across ROIs are complex. Ultimately, we encourage researchers to include estimates of variance when sharing data and consider models of heteroscedasticity in analysis. This work provides a foundation for study planning to account for regional variations in metric variance.