Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging.

OBJECTIVE: Artificial intelligence (AI) models trained using medical images for clinical tasks often exhibit bias in the form of subgroup performance disparities. However, since not all sources of bias in real-world medical imaging data are easily identifiable, it is challenging to comprehensively assess their impacts. In this article, we introduce an analysis framework for systematically and objectively investigating the impact of biases in medical images on AI models. MATERIALS AND METHODS: Our framework utilizes synthetic neuroimages with known disease effects and sources of bias. We evaluated the impact of bias effects and the efficacy of 3 bias mitigation strategies in counterfactual data scenarios on a convolutional neural network (CNN) classifier. RESULTS: The analysis revealed that training a CNN model on the datasets containing bias effects resulted in expected subgroup performance disparities. Moreover, reweighing was the most successful bias mitigation strategy for this setup. Finally, we demonstrated that explainable AI methods can aid in investigating the manifestation of bias in the model using this framework. DISCUSSION: The value of this framework is showcased in our findings on the impact of bias scenarios and efficacy of bias mitigation in a deep learning model pipeline. This systematic analysis can be easily expanded to conduct further controlled in silico trials in other investigations of bias in medical imaging AI. CONCLUSION: Our novel methodology for objectively studying bias in medical imaging AI can help support the development of clinical decision-support tools that are robust and responsible.

Annotation-free prediction of treatment-specific tissue outcome from 4D CT perfusion imaging in acute ischemic stroke.

Acute ischemic stroke is a critical health condition that requires timely intervention. Following admission, clinicians typically use perfusion imaging to facilitate treatment decision-making. While deep learning models leveraging perfusion data have demonstrated the ability to predict post-treatment tissue infarction for individual patients, predictions are often represented as binary or probabilistic masks that are not straightforward to interpret or easy to obtain. Moreover, these models typically rely on large amounts of subjectively segmented data and non-standard perfusion analysis techniques. To address these challenges, we propose a novel deep learning approach that directly predicts follow-up computed tomography images from full spatio-temporal 4D perfusion scans through a temporal compression. The results show that this method leads to realistic follow-up image predictions containing the infarcted tissue outcomes. The proposed compression method achieves comparable prediction results to using perfusion maps as inputs but without the need for perfusion analysis or arterial input function selection. Additionally, separate models trained on 45 patients treated with thrombolysis and 102 treated with thrombectomy showed that each model correctly captured the different patient-specific treatment effects as shown by image difference maps. The findings of this work clearly highlight the potential of our method to provide interpretable stroke treatment decision support without requiring manual annotations.

Exploiting macro- and micro-structural brain changes for improved Parkinson's disease classification from MRI data.

Parkinson's disease (PD) is the second most common neurodegenerative disease. Accurate PD diagnosis is crucial for effective treatment and prognosis but can be challenging, especially at early disease stages. This study aimed to develop and evaluate an explainable deep learning model for PD classification from multimodal neuroimaging data. The model was trained using one of the largest collections of T1-weighted and diffusion-tensor magnetic resonance imaging (MRI) datasets. A total of 1264 datasets from eight different studies were collected, including 611 PD patients and 653 healthy controls (HC). These datasets were pre-processed and non-linearly registered to the MNI PD25 atlas. Six imaging maps describing the macro- and micro-structural integrity of brain tissues complemented with age and sex parameters were used to train a convolutional neural network (CNN) to classify PD/HC subjects. Explainability of the model's decision-making was achieved using SmoothGrad saliency maps, highlighting important brain regions. The CNN was trained using a 75%/10%/15% train/validation/test split stratified by diagnosis, sex, age, and study, achieving a ROC-AUC of 0.89, accuracy of 80.8%, specificity of 82.4%, and sensitivity of 79.1% on the test set. Saliency maps revealed that diffusion tensor imaging data, especially fractional anisotropy, was more important for the classification than T1-weighted data, highlighting subcortical regions such as the brainstem, thalamus, amygdala, hippocampus, and cortical areas. The proposed model, trained on a large multimodal MRI database, can classify PD patients and HC subjects with high accuracy and clinically reasonable explanations, suggesting that micro-structural brain changes play an essential role in the disease course.

A multi-center distributed learning approach for Parkinson's disease classification using the traveling model paradigm.

Distributed learning is a promising alternative to central learning for machine learning (ML) model training, overcoming data-sharing problems in healthcare. Previous studies exploring federated learning (FL) or the traveling model (TM) setup for medical image-based disease classification often relied on large databases with a limited number of centers or simulated artificial centers, raising doubts about real-world applicability. This study develops and evaluates a convolution neural network (CNN) for Parkinson's disease classification using data acquired by 83 diverse real centers around the world, mostly contributing small training samples. Our approach specifically makes use of the TM setup, which has proven effective in scenarios with limited data availability but has never been used for image-based disease classification. Our findings reveal that TM is effective for training CNN models, even in complex real-world scenarios with variable data distributions. After sufficient training cycles, the TM-trained CNN matches or slightly surpasses the performance of the centrally trained counterpart (AUROC of 83% vs. 80%). Our study highlights, for the first time, the effectiveness of TM in 3D medical image classification, especially in scenarios with limited training samples and heterogeneous distributed data. These insights are relevant for situations where ML models are supposed to be trained using data from small or remote medical centers, and rare diseases with sparse cases. The simplicity of this approach enables a broad application to many deep learning tasks, enhancing its clinical utility across various contexts and medical facilities.

Identifying Biases in a Multicenter MRI Database for Parkinson's Disease Classification: Is the Disease Classifier a Secret Site Classifier?

Sharing multicenter imaging datasets can be advantageous to increase data diversity and size but may lead to spurious correlations between site-related biological and non-biological image features and target labels, which machine learning (ML) models may exploit as shortcuts. To date, studies analyzing how and if deep learning models may use such effects as a shortcut are scarce. Thus, the aim of this work was to investigate if site-related effects are encoded in the feature space of an established deep learning model designed for Parkinson's disease (PD) classification based on T1-weighted MRI datasets. Therefore, all layers of the PD classifier were frozen, except for the last layer of the network, which was replaced by a linear layer that was exclusively re-trained to predict three potential bias types (biological sex, scanner type, and originating site). Our findings based on a large database consisting of 1880 MRI scans collected across 41 centers show that the feature space of the established PD model (74% accuracy) can be used to classify sex (75% accuracy), scanner type (79% accuracy), and site location (71% accuracy) with high accuracies despite this information never being explicitly provided to the PD model during original training. Overall, the results of this study suggest that trained image-based classifiers may use unwanted shortcuts that are not meaningful for the actual clinical task at hand. This finding may explain why many image-based deep learning models do not perform well when applied to data from centers not contributing to the training set.

Providing clinical context to the spatio-temporal analysis of 4D CT perfusion to predict acute ischemic stroke lesion outcomes.

Acute ischemic stroke is a leading cause of mortality and morbidity worldwide. Timely identification of the extent of a stroke is crucial for effective treatment, whereas spatio-temporal (4D) Computed Tomography Perfusion (CTP) imaging is playing a critical role in this process. Recently, the first deep learning-based methods that leverage the full spatio-temporal nature of perfusion imaging for predicting stroke lesion outcomes have been proposed. However, clinical information is typically not integrated into the learning process, which may be helpful to improve the tissue outcome prediction given the known influence of various factors (i.e., physiological, demographic, and treatment factors) on lesion growth. Cross-attention, a multimodal fusion strategy, has been successfully used to combine information from multiple sources, but it has yet to be applied to stroke lesion outcome prediction. Therefore, this work aimed to develop and evaluate a novel multimodal and spatio-temporal deep learning model that utilizes cross-attention to combine information from 4D CTP and clinical metadata simultaneously to predict stroke lesion outcomes. The proposed model was evaluated using a dataset of 70 acute ischemic stroke patients, demonstrating significantly improved volume estimates (mean error = 19 ml) compared to a baseline unimodal approach (mean error = 35 ml, p< 0.05). The proposed model allows generating attention maps and counterfactual outcome scenarios to investigate the relevance of clinical variables in predicting stroke lesion outcomes at a patient level, helping to provide a better understanding of the model's decision-making process.

Simulation of neuroplasticity in a CNN-based in-silico model of neurodegeneration of the visual system.

The aim of this work was to enhance the biological feasibility of a deep convolutional neural network-based in-silico model of neurodegeneration of the visual system by equipping it with a mechanism to simulate neuroplasticity. Therefore, deep convolutional networks of multiple sizes were trained for object recognition tasks and progressively lesioned to simulate neurodegeneration of the visual cortex. More specifically, the injured parts of the network remained injured while we investigated how the added retraining steps were able to recover some of the model's object recognition baseline performance. The results showed with retraining, model object recognition abilities are subject to a smoother and more gradual decline with increasing injury levels than without retraining and, therefore, more similar to the longitudinal cognition impairments of patients diagnosed with Alzheimer's disease (AD). Moreover, with retraining, the injured model exhibits internal activation patterns similar to those of the healthy baseline model when compared to the injured model without retraining. Furthermore, we conducted this analysis on a network that had been extensively pruned, resulting in an optimized number of parameters or synapses. Our findings show that this network exhibited remarkably similar capability to recover task performance with decreasingly viable pathways through the network. In conclusion, adding a retraining step to the in-silico setup that simulates neuroplasticity improves the model's biological feasibility considerably and could prove valuable to test different rehabilitation approaches in-silico.

The causal link between cardiometabolic risk factors and gray matter atrophy: An exploratory study.

Although gray matter atrophy is commonly observed with aging, it is highly variable, even among healthy people of the same age. This raises the question of what other factors may contribute to gray matter atrophy. Previous studies have reported that risk factors for cardiometabolic diseases are associated with accelerated brain aging. However, these studies were primarily based on standard correlation analyses, which do not unveil a causal relationship. While randomized controlled trials are typically required to investigate true causality, in this work, we investigated an alternative method by exploring data-driven causal discovery and inference techniques on observational data. Accordingly, this feasibility study used clinical and quantified gray matter volume data from 22,793 subjects from the UK biobank cohort without any known neurological disease. Our method identified that age, sex, body mass index (BMI), body fat percentage (BFP), and smoking exhibit a causal relationship with gray matter volume. Interventions on the causal network revealed that higher BMI and BFP values significantly increased the chance of gray matter atrophy in males, whereas this was not the case in females.

Image-encoded biological and non-biological variables may be used as shortcuts in deep learning models trained on multisite neuroimaging data.

OBJECTIVE: This work investigates if deep learning (DL) models can classify originating site locations directly from magnetic resonance imaging (MRI) scans with and without correction for intensity differences. MATERIAL AND METHODS: A large database of 1880 T1-weighted MRI scans collected across 41 sites originally for Parkinson's disease (PD) classification was used to classify sites in this study. Forty-six percent of the datasets are from PD patients, while 54% are from healthy participants. After preprocessing the T1-weighted scans, 2 additional data types were generated: intensity-harmonized T1-weighted scans and log-Jacobian deformation maps resulting from nonlinear atlas registration. Corresponding DL models were trained to classify sites for each data type. Additionally, logistic regression models were used to investigate the contribution of biological (age, sex, disease status) and non-biological (scanner type) variables to the models' decision. RESULTS: A comparison of the 3 different types of data revealed that DL models trained using T1-weighted and intensity-harmonized T1-weighted scans can classify sites with an accuracy of 85%, while the model using log-Jacobian deformation maps achieved a site classification accuracy of 54%. Disease status and scanner type were found to be significant confounders. DISCUSSION: Our results demonstrate that MRI scans encode relevant site-specific information that models could use as shortcuts that cannot be removed using simple intensity harmonization methods. CONCLUSION: The ability of DL models to exploit site-specific biases as shortcuts raises concerns about their reliability, generalization, and deployability in clinical settings.

Explainable classification of Parkinson's disease using deep learning trained on a large multi-center database of T1-weighted MRI datasets.

INTRODUCTION: Parkinson's disease (PD) is a severe neurodegenerative disease that affects millions of people. Early diagnosis is important to facilitate prompt interventions to slow down disease progression. However, accurate PD diagnosis can be challenging, especially in the early disease stages. The aim of this work was to develop and evaluate a robust explainable deep learning model for PD classification trained from one of the largest collections of T1-weighted magnetic resonance imaging datasets. MATERIALS AND METHODS: A total of 2,041 T1-weighted MRI datasets from 13 different studies were collected, including 1,024 datasets from PD patients and 1,017 datasets from age- and sex-matched healthy controls (HC). The datasets were skull stripped, resampled to isotropic resolution, bias field corrected, and non-linearly registered to the MNI PD25 atlas. The Jacobian maps derived from the deformation fields together with basic clinical parameters were used to train a state-of-the-art convolutional neural network (CNN) to classify PD and HC subjects. Saliency maps were generated to display the brain regions contributing the most to the classification task as a means of explainable artificial intelligence. RESULTS: The CNN model was trained using an 85%/5%/10% train/validation/test split stratified by diagnosis, sex, and study. The model achieved an accuracy of 79.3%, precision of 80.2%, specificity of 81.3%, sensitivity of 77.7%, and AUC-ROC of 0.87 on the test set while performing similarly on an independent test set. Saliency maps computed for the test set data highlighted frontotemporal regions, the orbital-frontal cortex, and multiple deep gray matter structures as most important. CONCLUSION: The developed CNN model, trained on a large heterogenous database, was able to differentiate PD patients from HC subjects with high accuracy with clinically feasible classification explanations. Future research should aim to investigate the combination of multiple imaging modalities with deep learning and on validating these results in a prospective trial as a clinical decision support system.

Comparing 2D and 3D representations for face-based genetic syndrome diagnosis.

Human genetic syndromes are often challenging to diagnose clinically. Facial phenotype is a key diagnostic indicator for hundreds of genetic syndromes and computer-assisted facial phenotyping is a promising approach to assist diagnosis. Most previous approaches to automated face-based syndrome diagnosis have analyzed different datasets of either 2D images or surface mesh-based 3D facial representations, making direct comparisons of performance challenging. In this work, we developed a set of subject-matched 2D and 3D facial representations, which we then analyzed with the aim of comparing the performance of 2D and 3D image-based approaches to computer-assisted syndrome diagnosis. This work represents the most comprehensive subject-matched analyses to date on this topic. In our analyses of 1907 subject faces representing 43 different genetic syndromes, 3D surface-based syndrome classification models significantly outperformed 2D image-based models trained and evaluated on the same subject faces. These results suggest that the clinical adoption of 3D facial scanning technology and continued collection of syndromic 3D facial scan data may substantially improve face-based syndrome diagnosis.

Lesion-preserving unpaired image-to-image translation between MRI and CT from ischemic stroke patients.

PURPOSE: Multiple medical imaging modalities are used for clinical follow-up ischemic stroke analysis. Mixed-modality datasets are challenging, both for clinical rating purposes and for training machine learning models. While image-to-image translation methods have been applied to harmonize stroke patient images to a single modality, they have only been used for paired data so far. In the more common unpaired scenario, the standard cycle-consistent generative adversarial network (CycleGAN) method is not able to translate the stroke lesions properly. Thus, the aim of this work was to develop and evaluate a novel image-to-image translation regularization approach for unpaired 3D follow-up stroke patient datasets. METHODS: A modified CycleGAN was used to translate images between 238 non-contrast computed tomography (NCCT) and 244 fluid-attenuated inversion recovery (FLAIR) MRI datasets, two of the most relevant follow-up modalities in clinical practice. We introduced an additional attention-guided mechanism to encourage an improved translation of the lesion and a gradient-consistency loss to preserve structural brain morphology. RESULTS: The proposed modifications were able to preserve the overall quality provided by the CycleGAN translation. This was confirmed by the FID score and gradient correlation results. Furthermore, the lesion preservation was significantly improved compared to a standard CycleGAN. This was evaluated for location and volume with segmentation models, which were trained on real datasets and applied to the translated test images. Here, the Dice score coefficient resulted in 0.81 and 0.62 for datasets translated to FLAIR and NCCT, respectively, compared to 0.57 and 0.50 for the corresponding datasets translated using a standard CycleGAN. Finally, an analysis of the distribution of mean lesion intensities showed substantial improvements. CONCLUSION: The results of this work show that the proposed image-to-image translation method is effective at preserving stroke lesions in unpaired modality translation, supporting its potential as a tool for stroke image analysis in real-life scenarios.

Dementia in Convolutional Neural Networks: Using Deep Learning Models to Simulate Neurodegeneration of the Visual System.

Although current research aims to improve deep learning networks by applying knowledge about the healthy human brain and vice versa, the potential of using such networks to model and study neurodegenerative diseases remains largely unexplored. In this work, we present an in-depth feasibility study modeling progressive dementia in silico with deep convolutional neural networks. Therefore, networks were trained to perform visual object recognition and then progressively injured by applying neuronal as well as synaptic injury. After each iteration of injury, network object recognition accuracy, saliency map similarity between the intact and injured networks, and internal activations of the degenerating models were evaluated. The evaluation revealed that cognitive function of the network progressively decreased with increasing injury load whereas this effect was much more pronounced for synaptic damage. The effects of neurodegeneration found for the in silico model are especially similar to the loss of visual cognition seen in patients with posterior cortical atrophy.

Detecting 3D syndromic faces as outliers using unsupervised normalizing flow models.

Many genetic syndromes are associated with distinctive facial features. Several computer-assisted methods have been proposed that make use of facial features for syndrome diagnosis. Training supervised classifiers, the most common approach for this purpose, requires large, comprehensive, and difficult to collect databases of syndromic facial images. In this work, we use unsupervised, normalizing flow-based manifold and density estimation models trained entirely on unaffected subjects to detect syndromic 3D faces as statistical outliers. Furthermore, we demonstrate a general, user-friendly, gradient-based interpretability mechanism that enables clinicians and patients to understand model inferences. 3D facial surface scans of 2471 unaffected subjects and 1629 syndromic subjects representing 262 different genetic syndromes were used to train and evaluate the models. The flow-based models outperformed unsupervised comparison methods, with the best model achieving an ROC-AUC of 86.3% on a challenging, age and sex diverse data set. In addition to highlighting the viability of outlier-based syndrome screening tools, our methods generalize and extend previously proposed outlier scores for 3D face-based syndrome detection, resulting in improved performance for unsupervised syndrome detection.

Predicting the tissue outcome of acute ischemic stroke from acute 4D computed tomography perfusion imaging using temporal features and deep learning.

Predicting follow-up lesions from baseline CT perfusion (CTP) datasets in acute ischemic stroke patients is important for clinical decision making. Deep convolutional networks (DCNs) are assumed to be the current state-of-the-art for this task. However, many DCN classifiers have not been validated against the methods currently used in research (random decision forests, RDF) and clinical routine (Tmax thresholding). Specialized DCNs have even been designed to extract complex temporal features directly from spatiotemporal CTP data instead of using standard perfusion parameter maps. However, the benefits of applying deep learning to source or deconvolved CTP data compared to perfusion parameter maps have not been formally investigated so far. In this work, a modular UNet-based DCN is proposed that separates temporal feature extraction from tissue outcome prediction, allowing for both model validation using perfusion parameter maps as well as end-to-end learning from spatiotemporal CTP data. 145 retrospective datasets comprising baseline CTP imaging, perfusion parameter maps, and follow-up non-contrast CT with manual lesion segmentations were assembled from acute ischemic stroke patients treated with intravenous thrombolysis alone (IV; n = 43) or intra-arterial mechanical thrombectomy (IA; n = 102) with or without combined IV. Using the perfusion parameter maps as input, the proposed DCN (mean Dice: 0.287) outperformed the RDF (0.262) and simple Tmax-thresholding (0.249). The performance of the proposed DCN was approximately equal using features optimized from the deconvolved residual curves (0.286) compared to perfusion parameter maps (0.287), while using features optimized from the source concentration-time curves (0.296) provided the best tissue outcome predictions.

An analysis of the effects of limited training data in distributed learning scenarios for brain age prediction.

OBJECTIVE: Distributed learning avoids problems associated with central data collection by training models locally at each site. This can be achieved by federated learning (FL) aggregating multiple models that were trained in parallel or training a single model visiting sites sequentially, the traveling model (TM). While both approaches have been applied to medical imaging tasks, their performance in limited local data scenarios remains unknown. In this study, we specifically analyze FL and TM performances when very small sample sizes are available per site. MATERIALS AND METHODS: 2025 T1-weighted magnetic resonance imaging scans were used to investigate the effect of sample sizes on FL and TM for brain age prediction. We evaluated models across 18 scenarios varying the number of samples per site (1, 2, 5, 10, and 20) and the number of training rounds (20, 40, and 200). RESULTS: Our results demonstrate that the TM outperforms FL, for every sample size examined. In the extreme case when each site provided only one sample, FL achieved a mean absolute error (MAE) of 18.9 ± 0.13 years, while the TM achieved a MAE of 6.21 ± 0.50 years, comparable to central learning (MAE = 5.99 years). DISCUSSION: Although FL is more commonly used, our study demonstrates that TM is the best implementation for small sample sizes. CONCLUSION: The TM offers new opportunities to apply machine learning models in rare diseases and pediatric research but also allows even small hospitals to contribute small datasets.

Predicting treatment-specific lesion outcomes in acute ischemic stroke from 4D CT perfusion imaging using spatio-temporal convolutional neural networks.

For the diagnosis and precise treatment of acute ischemic stroke, predicting the final location and volume of lesions is of great clinical interest. Current deep learning-based prediction methods mainly use perfusion parameter maps, which can be calculated from spatio-temporal (4D) CT perfusion (CTP) imaging data, to estimate the tissue outcome of an acute ischemic stroke. However, this calculation relies on a deconvolution operation, an ill-posed problem requiring strong regularization and definition of an arterial input function. Thus, improved predictions might be achievable if the deep learning models were applied directly to acute 4D CTP data rather than perfusion maps. In this work, a novel deep spatio-temporal convolutional neural network is proposed for predicting treatment-dependent stroke lesion outcomes by making full use of raw 4D CTP data. By merging a U-Net-like architecture with temporal convolutional networks, we efficiently process the spatio-temporal information available in CTP datasets to make a tissue outcome prediction. The proposed method was evaluated on 147 patients using a 10-fold cross validation, which demonstrated that the proposed 3D+time model (mean Dice=0.45) significantly outperforms both a 2D+time variant of our approach (mean Dice=0.43) and a state-of-the-art method that uses perfusion maps (mean Dice=0.38). These results show that 4D CTP datasets include more predictive information than perfusion parameter maps, and that the proposed method is an efficient approach to make use of this complex data.

An exploratory causal analysis of the relationships between the brain age gap and cardiovascular risk factors.

The brain age gap (BAG) has been shown to capture accelerated brain aging patterns and might serve as a biomarker for several neurological diseases. Moreover, it was also shown that it captures other biological information related to modifiable cardiovascular risk factors. Previous studies have explored statistical relationships between the BAG and cardiovascular risk factors. However, none of those studies explored causal relationships between the BAG and cardiovascular risk factors. In this work, we employ causal structure discovery techniques and define a Bayesian network to model the assumed causal relationships between the BAG, estimated using morphometric T1-weighted magnetic resonance imaging brain features from 2025 adults, and several cardiovascular risk factors. This setup allows us to not only assess observed conditional probability distributions of the BAG given cardiovascular risk factors, but also to isolate the causal effect of each cardiovascular risk factor on BAG using causal inference. Results demonstrate the feasibility of the proposed causal analysis approach by illustrating intuitive causal relationships between variables. For example, body-mass-index, waist-to-hip ratio, smoking, and alcohol consumption were found to impact the BAG, with the greatest impact for obesity markers resulting in higher chances of developing accelerated brain aging. Moreover, the findings show that causal effects differ from correlational effects, demonstrating the importance of accounting for variable relationships and confounders when evaluating the information captured by a biomarker. Our work demonstrates the feasibility and advantages of using causal analyses instead of purely correlation-based and univariate statistical analyses in the context of brain aging and related problems.

Fairness-related performance and explainability effects in deep learning models for brain image analysis.

Purpose: Explainability and fairness are two key factors for the effective and ethical clinical implementation of deep learning-based machine learning models in healthcare settings. However, there has been limited work on investigating how unfair performance manifests in explainable artificial intelligence (XAI) methods, and how XAI can be used to investigate potential reasons for unfairness. Thus, the aim of this work was to analyze the effects of previously established sociodemographic-related confounders on classifier performance and explainability methods. Approach: A convolutional neural network (CNN) was trained to predict biological sex from T1-weighted brain MRI datasets of 4547 9- to 10-year-old adolescents from the Adolescent Brain Cognitive Development study. Performance disparities of the trained CNN between White and Black subjects were analyzed and saliency maps were generated for each subgroup at the intersection of sex and race. Results: The classification model demonstrated a significant difference in the percentage of correctly classified White male ( 90.3 % ± 1.7 % ) and Black male ( 81.1 % ± 4.5 % ) children. Conversely, slightly higher performance was found for Black female ( 89.3 % ± 4.8 % ) compared with White female ( 86.5 % ± 2.0 % ) children. Saliency maps showed subgroup-specific differences, corresponding to brain regions previously associated with pubertal development. In line with this finding, average pubertal development scores of subjects used in this study were significantly different between Black and White females ( p < 0.001 ) and males ( p < 0.001 ). Conclusions: We demonstrate that a CNN with significantly different sex classification performance between Black and White adolescents can identify different important brain regions when comparing subgroup saliency maps. Importance scores vary substantially between subgroups within brain structures associated with pubertal development, a race-associated confounder for predicting sex. We illustrate that unfair models can produce different XAI results between subgroups and that these results may explain potential reasons for biased performance.

A Deep Invertible 3-D Facial Shape Model for Interpretable Genetic Syndrome Diagnosis.

One of the primary difficulties in treating patients with genetic syndromes is diagnosing their condition. Many syndromes are associated with characteristic facial features that can be imaged and utilized by computer-assisted diagnosis systems. In this work, we develop a novel 3D facial surface modeling approach with the objective of maximizing diagnostic model interpretability within a flexible deep learning framework. Therefore, an invertible normalizing flow architecture is introduced to enable both inferential and generative tasks in a unified and efficient manner. The proposed model can be used (1) to infer syndrome diagnosis and other demographic variables given a 3D facial surface scan and (2) to explain model inferences to non-technical users via multiple interpretability mechanisms. The model was trained and evaluated on more than 4700 facial surface scans from subjects with 47 different syndromes. For the challenging task of predicting syndrome diagnosis given a new 3D facial surface scan, age, and sex of a subject, the model achieves a competitive overall top-1 accuracy of 71%, and a mean sensitivity of 43% across all syndrome classes. We believe that invertible models such as the one presented in this work can achieve competitive inferential performance while greatly increasing model interpretability in the domain of medical diagnosis.