Distilling BlackBox to Interpretable Models for Efficient Transfer Learning.

Building generalizable AI models is one of the primary challenges in the healthcare domain. While radiologists rely on generalizable descriptive rules of abnormality, Neural Network (NN) models suffer even with a slight shift in input distribution (e.g., scanner type). Fine-tuning a model to transfer knowledge from one domain to another requires a significant amount of labeled data in the target domain. In this paper, we develop an interpretable model that can be efficiently fine-tuned to an unseen target domain with minimal computational cost. We assume the interpretable component of NN to be approximately domain-invariant. However, interpretable models typically underperform compared to their Blackbox (BB) variants. We start with a BB in the source domain and distill it into a mixture of shallow interpretable models using human-understandable concepts. As each interpretable model covers a subset of data, a mixture of interpretable models achieves comparable performance as BB. Further, we use the pseudo-labeling technique from semi-supervised learning (SSL) to learn the concept classifier in the target domain, followed by fine-tuning the interpretable models in the target domain. We evaluate our model using a real-life large-scale chest-X-ray (CXR) classification dataset. The code is available at: https://github.com/batmanlab/MICCAI-2023-Route-interpret-repeat-CXRs.

Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat.

ML model design either starts with an interpretable model or a Blackbox and explains it post hoc. Blackbox models are flexible but difficult to explain, while interpretable models are inherently explainable. Yet, interpretable models require extensive ML knowledge and tend to be less flexible and underperforming than their Blackbox variants. This paper aims to blur the distinction between a post hoc explanation of a Blackbox and constructing interpretable models. Beginning with a Blackbox, we iteratively carve out a mixture of interpretable experts (MoIE) and a residual network. Each interpretable model specializes in a subset of samples and explains them using First Order Logic (FOL), providing basic reasoning on concepts from the Blackbox. We route the remaining samples through a flexible residual. We repeat the method on the residual network until all the interpretable models explain the desired proportion of data. Our extensive experiments show that our route, interpret, and repeat approach (1) identifies a diverse set of instance-specific concepts with high concept completeness via MoIE without compromising in performance, (2) identifies the relatively "harder" samples to explain via residuals, (3) outperforms the interpretable by-design models by significant margins during test-time interventions, and (4) fixes the shortcut learned by the original Blackbox. The code for MoIE is publicly available at: https://github.com/batmanlab/ICML-2023-Route-interpret-repeat.

Calcium signaling in cerebral vasoregulation.

The tight coupling of regional neurometabolic activity with synaptic activity and regional cerebral blood perfusion constitutes a single functional unit, described generally as a neurovascular unit. This is central to any discussion of haemodynamic response linked to any neuronal activation. In normal as well as in pathologic conditions, neurons, astrocytes and endothelial cells of the vasculature interact to generate the complex activity-induced cerebral haemodynamic responses, with astrocytes not only partaking in the signaling but actually controlling it in many cases. Neurons and astrocytes have highly integrated signaling mechanisms, yet they form two separate networks. Bidirectional neuron-astrocyte interactions are crucial for the function and survival of the central nervous system. The primary purpose of such regulation is the homeostasis of the brain's microenvironment. In the maintenance of such homeostasis, astrocytic calcium response is a crucial variable in determining neurovascular control. Future work will be directed towards resolving the nature and extent of astrocytic calcium-mediated mechanisms for gene transcription, in modelling neurovascular control, and in determining calcium sensitive imaging assays that can capture disease variables.

DR-VIDAL - Doubly Robust Variational Information-theoretic Deep Adversarial Learning for Counterfactual Prediction and Treatment Effect Estimation on Real World Data.

Determining causal effects of interventions onto outcomes from real-world, observational (non-randomized) data, e.g., treatment repurposing using electronic health records, is challenging due to underlying bias. Causal deep learning has improved over traditional techniques for estimating individualized treatment effects (ITE). We present the Doubly Robust Variational Information-theoretic Deep Adversarial Learning (DR-VIDAL), a novel generative framework that combines two joint models of treatment and outcome, ensuring an unbiased ITE estimation even when one of the two is misspecified. DR-VIDAL integrates: (i) a variational autoencoder (VAE) to factorize confounders into latent variables according to causal assumptions; (ii) an information-theoretic generative adversarial network (Info-GAN) to generate counterfactuals; (iii) a doubly robust block incorporating treatment propensities for outcome predictions. On synthetic and real-world datasets (Infant Health and Development Program, Twin Birth Registry, and National Supported Work Program), DR-VIDAL achieves better performance than other non-generative and generative methods. In conclusion, DR-VIDAL uniquely fuses causal assumptions, VAE, Info-GAN, and doubly robustness into a comprehensive, per- formant framework. Code is available at: https://github.com/Shantanu48114860/DR-VIDAL-AMIA-22 under MIT license.

Anatomy-Guided Weakly-Supervised Abnormality Localization in Chest X-rays.

Creating a large-scale dataset of abnormality annotation on medical images is a labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack of large-scale data for anomaly detection methods. However, most of the current methods only use image-level pathological observations, failing to utilize the relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined weak labels are noisy due to label sparsity and linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) to address these issues of weak annotation. Our framework consists of a cascade of two networks, one responsible for identifying anatomical abnormalities and the second responsible for pathological observations. The critical component in our framework is an anatomy-guided attention module that aids the downstream observation network in focusing on the relevant anatomical regions generated by the anatomy network. We use Positive Unlabeled (PU) learning to account for the fact that lack of mention does not necessarily mean a negative label. Our quantitative and qualitative results on the MIMIC-CXR dataset demonstrate the effectiveness of AGXNet in disease and anatomical abnormality localization. Experiments on the NIH Chest X-ray dataset show that the learned feature representations are transferable and can achieve the state-of-the-art performances in disease classification and competitive disease localization results. Our code is available at https://github.com/batmanlab/AGXNet.

Propensity score synthetic augmentation matching using generative adversarial networks (PSSAM-GAN).

Understanding causality is of crucial importance in biomedical sciences, where developing prediction models is insufficient because the models need to be actionable. However, data sources, such as electronic health records, are observational and often plagued with various types of biases, e.g. confounding. Although randomized controlled trials are the gold standard to estimate the causal effects of treatment interventions on health outcomes, they are not always possible. Propensity score matching (PSM) is a popular statistical technique for observational data that aims at balancing the characteristics of the population assigned either to a treatment or to a control group, making treatment assignment and outcome independent upon these characteristics. However, matching subjects can reduce the sample size. Inverse probability weighting (IPW) maintains the sample size, but extreme values can lead to instability. While PSM and IPW have been historically used in conjunction with linear regression, machine learning methods -including deep learning with propensity dropout- have been proposed to account for nonlinear treatment assignments. In this work, we propose a novel deep learning approach -the Propensity Score Synthetic Augmentation Matching using Generative Adversarial Networks (PSSAM-GAN)- that aims at keeping the sample size, without IPW, by generating synthetic matches. PSSAM-GAN can be used in conjunction with any other prediction method to estimate treatment effects. Experiments performed on both semi-synthetic (perinatal interventions) and real-world observational data (antibiotic treatments, and job interventions) show that the PSSAM-GAN approach effectively creates balanced datasets, relaxing the weighting/dropout needs for downstream methods, and providing competitive performance in effects estimation as compared to simple GAN and in conjunction with other deep counterfactual learning architectures, e.g. TARNet.

Deep propensity network using a sparse autoencoder for estimation of treatment effects.

OBJECTIVE: Drawing causal estimates from observational data is problematic, because datasets often contain underlying bias (eg, discrimination in treatment assignment). To examine causal effects, it is important to evaluate what-if scenarios-the so-called "counterfactuals." We propose a novel deep learning architecture for propensity score matching and counterfactual prediction-the deep propensity network using a sparse autoencoder (DPN-SA)-to tackle the problems of high dimensionality, nonlinear/nonparallel treatment assignment, and residual confounding when estimating treatment effects. MATERIALS AND METHODS: We used 2 randomized prospective datasets, a semisynthetic one with nonlinear/nonparallel treatment selection bias and simulated counterfactual outcomes from the Infant Health and Development Program and a real-world dataset from the LaLonde's employment training program. We compared different configurations of the DPN-SA against logistic regression and LASSO as well as deep counterfactual networks with propensity dropout (DCN-PD). Models' performances were assessed in terms of average treatment effects, mean squared error in precision on effect's heterogeneity, and average treatment effect on the treated, over multiple training/test runs. RESULTS: The DPN-SA outperformed logistic regression and LASSO by 36%-63%, and DCN-PD by 6%-10% across all datasets. All deep learning architectures yielded average treatment effects close to the true ones with low variance. Results were also robust to noise-injection and addition of correlated variables. Code is publicly available at https://github.com/Shantanu48114860/DPN-SAz. DISCUSSION AND CONCLUSION: Deep sparse autoencoders are particularly suited for treatment effect estimation studies using electronic health records because they can handle high-dimensional covariate sets, large sample sizes, and complex heterogeneity in treatment assignments.

Causal AI with Real World Data: Do Statins Protect from Alzheimer's Disease Onset?

Causal artificial intelligence aims at developing bias-robust models that can be used to intervene on, rather than just be predictive, of risks or outcomes. However, learning interventional models from observational data, including electronic health records (EHR), is challenging due to inherent bias, e.g., protopathic, confounding, collider. When estimating the effects of treatment interventions, classical approaches like propensity score matching are often used, but they pose limitations with large feature sets, nonlinear/nonparallel treatment group assignments, and collider bias. In this work, we used data from a large EHR consortium -OneFlorida- and evaluated causal statistical/machine learning methods for determining the effect of statin treatment on the risk of Alzheimer's disease, a debated clinical research question. We introduced a combination of directed acyclic graph (DAG) learning and comparison with expert's design, with calculation of the generalized adjustment criterion (GAC), to find an optimal set of covariates for estimation of treatment effects -ameliorating collider bias. The DAG/CAC approach was assessed together with traditional propensity score matching, inverse probability weighting, virtual-twin/counterfactual random forests, and deep counterfactual networks. We showed large heterogeneity in effect estimates upon different model configurations. Our results did not exclude a protective effect of statins, where the DAG/GAC point estimate aligned with the maximum credibility estimate, although the 95% credibility interval included a null effect, warranting further studies and replication.

Changes in cytosolic Ca2+ levels correspond to fluctuations of lactate levels in crosstalk of astrocyte-neuron cell lines.

Neurons and astrocytes differentially express isoenzymes of lactate dehydrogenase (LDH). The metabolic consequences for the variations in mRNA expression of LDH isoenzyme subtypes in neurons and astrocytes control cerebral vasoregulation. Moreover, cellular signalling consequences for functional neurovascular control may also be dependent on LDH isoenzyme subtype profiles. Initial computer simulations revealed glutamate-induced calcium waves in connected astrocytes, and showed concomitant changes in the expression of nitric oxide synthase (NOS) and lactic acid metabolism. To validate these findings, the nature and extent of glutamate-dependent signalling crosstalk in murine cell lines were investigated through correlated lactate levels and calcium upregulation. Neuro2A and C8D1A cells were separately treated with timed supernatant extracts from each other and their LDH1 and LDH5 isoenzyme responses were recorded. Western blot analysis showed LDH1/LDH5 isoenzyme ratio in the astrocytes to be positively correlated with Neuro2A-derived lactate levels estimated by the amplitude of 1.33-ppm spectral peak in 1H-NMR, and LDH1/LDH5 isoenzyme ratio in neurons is negatively correlated with CSD1A-derived lactate levels. Significant modulations of the calcium-responsive protein pCamKII levels were also observed in both cell lines, particularly correlations between pCamKII and lactate in C8D1A cells, thus explaining the calcium dependence of the lactate response. Together, these observations indicate that lactate is a key indicator of the metabolic state of these cell types, and may be a determinant of release of vasoregulatory factors.

High-Accuracy Classification of Parkinson's Disease Through Shape Analysis and Surface Fitting in 123I-Ioflupane SPECT Imaging.

Early and accurate identification of Parkinsonian syndromes (PS) involving presynaptic degeneration from nondegenerative variants such as scans without evidence of dopaminergic deficit (SWEDD) and tremor disorders is important for effective patient management as the course, therapy, and prognosis differ substantially between the two groups. In this study, we use single photon emission computed tomography (SPECT) images from healthy normal, early PD, and SWEDD subjects, as obtained from the Parkinson's Progression Markers Initiative (PPMI) database, and process them to compute shape- and surface-fitting-based features. We use these features to develop and compare various classification models that can discriminate between scans showing dopaminergic deficit, as in PD, from scans without the deficit, as in healthy normal or SWEDD. Along with it, we also compare these features with striatal binding ratio (SBR)-based features, which are well established and clinically used, by computing a feature-importance score using random forests technique. We observe that the support vector machine (SVM) classifier gives the best performance with an accuracy of 97.29%. These features also show higher importance than the SBR-based features. We infer from the study that shape analysis and surface fitting are useful and promising methods for extracting discriminatory features that can be used to develop diagnostic models that might have the potential to help clinicians in the diagnostic process.

High-Accuracy Detection of Early Parkinson's Disease through Multimodal Features and Machine Learning.

Early (or preclinical) diagnosis of Parkinson's disease (PD) is crucial for its early management as by the time manifestation of clinical symptoms occur, more than 60% of the dopaminergic neurons have already been lost. It is now established that there exists a premotor stage, before the start of these classic motor symptoms, characterized by a constellation of clinical features, mostly non-motor in nature such as Rapid Eye Movement (REM) sleep Behaviour Disorder (RBD) and olfactory loss. In this paper, we use the non-motor features of RBD and olfactory loss, along with other significant biomarkers such as Cerebrospinal fluid (CSF) measurements and dopaminergic imaging markers from 183 healthy normal and 401 early PD subjects, as obtained from the Parkinson's Progression Markers Initiative (PPMI) database, to classify early PD subjects from normal using Naïve Bayes, Support Vector Machine (SVM), Boosted Trees and Random Forests classifiers. We observe that SVM classifier gave the best performance (96.40% accuracy, 97.03% sensitivity, 95.01% specificity, and 98.88% area under ROC). We infer from the study that a combination of non-motor, CSF and imaging markers may aid in the preclinical diagnosis of PD.

Orbital extension of trigeminal schwannoma.

Schwannomas, also known as neurilemmomas, are benign peripheral nerve sheath tumors. Trigeminal schwannomas are rare intracranial tumors. Here, we report a 35-year-old female presenting with an axial proptosis of right eyeball with right-sided III, IV and VI cranial nerve palsy. Her best corrected visual acuity in the right eye was perception of light absent and in the left eye was 20/20. MRI scan revealed a large right-sided heterogeneous, extra-axial middle cranial fossa mass that extended to the intraconal space of right orbit. A diagnosis of intracranial trigeminal nerve schwannoma with right orbital extension was made. Successful surgical excision of the mass with preservation of the surrounding tissues and orbital exenteration was done. Post-operative period was uneventful.

Functional mapping of language networks in the normal brain using a word-association task.

BACKGROUND: Language functions are known to be affected in diverse neurological conditions, including ischemic stroke, traumatic brain injury, and brain tumors. Because language networks are extensive, interpretation of functional data depends on the task completed during evaluation. AIM: The aim was to map the hemodynamic consequences of word association using functional magnetic resonance imaging (fMRI) in normal human subjects. MATERIALS AND METHODS: Ten healthy subjects underwent fMRI scanning with a postlexical access semantic association task vs lexical processing task. The fMRI protocol involved a T2*-weighted gradient-echo echo-planar imaging (GE-EPI) sequence (TR 4523 ms, TE 64 ms, flip angle 90°) with alternate baseline and activation blocks. A total of 78 scans were taken (interscan interval = 3 s) with a total imaging time of 587 s. Functional data were processed in Statistical Parametric Mapping software (SPM2) with 8-mm Gaussian kernel by convolving the blood oxygenation level-dependent (BOLD) signal with an hemodynamic response function estimated by general linear method to generate SPM{t} and SPM{F} maps. RESULTS: Single subject analysis of the functional data (FWE-corrected, P≤0.001) revealed extensive activation in the frontal lobes, with overlaps among middle frontal gyrus (MFG), superior, and inferior frontal gyri. BOLD activity was also found in the medial frontal gyrus, middle occipital gyrus (MOG), anterior fusiform gyrus, superior and inferior parietal lobules, and to a smaller extent, the thalamus and right anterior cerebellum. Group analysis (FWE-corrected, P≤0.001) revealed neural recruitment of bilateral lingual gyri, left MFG, bilateral MOG, left superior occipital gyrus, left fusiform gyrus, bilateral thalami, and right cerebellar areas. CONCLUSIONS: Group data analysis revealed a cerebellar-occipital-fusiform-thalamic network centered around bilateral lingual gyri for word association, thereby indicating how these areas facilitate language comprehension by activating a semantic association network of words processed postlexical access. This finding is important when assessing the extent of cognitive damage and/or recovery and can be used for presurgical planning after optimization.