Leveraging different learning styles for improved knowledge distillation in biomedical imaging.

Learning style refers to a type of training mechanism adopted by an individual to gain new knowledge. As suggested by the VARK model, humans have different learning preferences, like Visual (V), Auditory (A), Read/Write (R), and Kinesthetic (K), for acquiring and effectively processing information. Our work endeavors to leverage this concept of knowledge diversification to improve the performance of model compression techniques like Knowledge Distillation (KD) and Mutual Learning (ML). Consequently, we use a single-teacher and two-student network in a unified framework that not only allows for the transfer of knowledge from teacher to students (KD) but also encourages collaborative learning between students (ML). Unlike the conventional approach, where the teacher shares the same knowledge in the form of predictions or feature representations with the student network, our proposed approach employs a more diversified strategy by training one student with predictions and the other with feature maps from the teacher. We further extend this knowledge diversification by facilitating the exchange of predictions and feature maps between the two student networks, enriching their learning experiences. We have conducted comprehensive experiments with three benchmark datasets for both classification and segmentation tasks using two different network architecture combinations. These experimental results demonstrate that knowledge diversification in a combined KD and ML framework outperforms conventional KD or ML techniques (with similar network configuration) that only use predictions with an average improvement of 2%. Furthermore, consistent improvement in performance across different tasks, with various network architectures, and over state-of-the-art techniques establishes the robustness and generalizability of the proposed model.

Understanding calibration of deep neural networks for medical image classification.

Background and Objective - In the field of medical image analysis, achieving high accuracy is not enough; ensuring well-calibrated predictions is also crucial. Confidence scores of a deep neural network play a pivotal role in explainability by providing insights into the model's certainty, identifying cases that require attention, and establishing trust in its predictions. Consequently, the significance of a well-calibrated model becomes paramount in the medical imaging domain, where accurate and reliable predictions are of utmost importance. While there has been a significant effort towards training modern deep neural networks to achieve high accuracy on medical imaging tasks, model calibration and factors that affect it remain under-explored. Methods - To address this, we conducted a comprehensive empirical study that explores model performance and calibration under different training regimes. We considered fully supervised training, which is the prevailing approach in the community, as well as rotation-based self-supervised method with and without transfer learning, across various datasets and architecture sizes. Multiple calibration metrics were employed to gain a holistic understanding of model calibration. Results - Our study reveals that factors such as weight distributions and the similarity of learned representations correlate with the calibration trends observed in the models. Notably, models trained using rotation-based self-supervised pretrained regime exhibit significantly better calibration while achieving comparable or even superior performance compared to fully supervised models across different medical imaging datasets. Conclusion - These findings shed light on the importance of model calibration in medical image analysis and highlight the benefits of incorporating self-supervised learning approach to improve both performance and calibration.

Functional connectivity signatures of major depressive disorder: machine learning analysis of two multicenter neuroimaging studies.

The promise of machine learning has fueled the hope for developing diagnostic tools for psychiatry. Initial studies showed high accuracy for the identification of major depressive disorder (MDD) with resting-state connectivity, but progress has been hampered by the absence of large datasets. Here we used regular machine learning and advanced deep learning algorithms to differentiate patients with MDD from healthy controls and identify neurophysiological signatures of depression in two of the largest resting-state datasets for MDD. We obtained resting-state functional magnetic resonance imaging data from the REST-meta-MDD (N = 2338) and PsyMRI (N = 1039) consortia. Classification of functional connectivity matrices was done using support vector machines (SVM) and graph convolutional neural networks (GCN), and performance was evaluated using 5-fold cross-validation. Features were visualized using GCN-Explainer, an ablation study and univariate t-testing. The results showed a mean classification accuracy of 61% for MDD versus controls. Mean accuracy for classifying (non-)medicated subgroups was 62%. Sex classification accuracy was substantially better across datasets (73-81%). Visualization of the results showed that classifications were driven by stronger thalamic connections in both datasets, while nearly all other connections were weaker with small univariate effect sizes. These results suggest that whole brain resting-state connectivity is a reliable though poor biomarker for MDD, presumably due to disease heterogeneity as further supported by the higher accuracy for sex classification using the same methods. Deep learning revealed thalamic hyperconnectivity as a prominent neurophysiological signature of depression in both multicenter studies, which may guide the development of biomarkers in future studies.

Predicting women with depressive symptoms postpartum with machine learning methods.

Postpartum depression (PPD) is a detrimental health condition that affects 12% of new mothers. Despite negative effects on mothers' and children's health, many women do not receive adequate care. Preventive interventions are cost-efficient among high-risk women, but our ability to identify these is poor. We leveraged the power of clinical, demographic, and psychometric data to assess if machine learning methods can make accurate predictions of postpartum depression. Data were obtained from a population-based prospective cohort study in Uppsala, Sweden, collected between 2009 and 2018 (BASIC study, n = 4313). Sub-analyses among women without previous depression were performed. The extremely randomized trees method provided robust performance with highest accuracy and well-balanced sensitivity and specificity (accuracy 73%, sensitivity 72%, specificity 75%, positive predictive value 33%, negative predictive value 94%, area under the curve 81%). Among women without earlier mental health issues, the accuracy was 64%. The variables setting women at most risk for PPD were depression and anxiety during pregnancy, as well as variables related to resilience and personality. Future clinical models that could be implemented directly after delivery might consider including these variables in order to identify women at high risk for postpartum depression to facilitate individualized follow-up and cost-effectiveness.

Investigating the temporal dynamics of electroencephalogram (EEG) microstates using recurrent neural networks.

Electroencephalogram (EEG) microstates that represent quasi-stable, global neuronal activity are considered as the building blocks of brain dynamics. Therefore, the analysis of microstate sequences is a promising approach to understand fast brain dynamics that underlie various mental processes. Recent studies suggest that EEG microstate sequences are non-Markovian and nonstationary, highlighting the importance of the sequential flow of information between different brain states. These findings inspired us to model these sequences using Recurrent Neural Networks (RNNs) consisting of long-short-term-memory (LSTM) units to capture the complex temporal dependencies. Using an LSTM-based auto encoder framework and different encoding schemes, we modeled the microstate sequences at multiple time scales (200-2,000 ms) aiming to capture stably recurring microstate patterns within and across subjects. We show that RNNs can learn underlying microstate patterns with high accuracy and that the microstate trajectories are subject invariant at shorter time scales (≤400 ms) and reproducible across sessions. Significant drop in the reconstruction accuracy was observed for longer sequence lengths of 2,000 ms. These findings indirectly corroborate earlier studies which indicated that EEG microstate sequences exhibit long-range dependencies with finite memory content. Furthermore, we find that the latent representations learned by the RNNs are sensitive to external stimulation such as stress while the conventional univariate microstate measures (e.g., occurrence, mean duration, etc.) fail to capture such changes in brain dynamics. While RNNs cannot be configured to identify the specific discriminating patterns, they have the potential for learning the underlying temporal dynamics and are sensitive to sequence aberrations characterized by changes in metal processes. Empowered with the macroscopic understanding of the temporal dynamics that extends beyond short-term interactions, RNNs offer a reliable alternative for exploring system level brain dynamics using EEG microstate sequences.

REFUGE Challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs.

Glaucoma is one of the leading causes of irreversible but preventable blindness in working age populations. Color fundus photography (CFP) is the most cost-effective imaging modality to screen for retinal disorders. However, its application to glaucoma has been limited to the computation of a few related biomarkers such as the vertical cup-to-disc ratio. Deep learning approaches, although widely applied for medical image analysis, have not been extensively used for glaucoma assessment due to the limited size of the available data sets. Furthermore, the lack of a standardize benchmark strategy makes difficult to compare existing methods in a uniform way. In order to overcome these issues we set up the Retinal Fundus Glaucoma Challenge, REFUGE (https://refuge.grand-challenge.org), held in conjunction with MICCAI 2018. The challenge consisted of two primary tasks, namely optic disc/cup segmentation and glaucoma classification. As part of REFUGE, we have publicly released a data set of 1200 fundus images with ground truth segmentations and clinical glaucoma labels, currently the largest existing one. We have also built an evaluation framework to ease and ensure fairness in the comparison of different models, encouraging the development of novel techniques in the field. 12 teams qualified and participated in the online challenge. This paper summarizes their methods and analyzes their corresponding results. In particular, we observed that two of the top-ranked teams outperformed two human experts in the glaucoma classification task. Furthermore, the segmentation results were in general consistent with the ground truth annotations, with complementary outcomes that can be further exploited by ensembling the results.

Reward circuit connectivity relates to delay discounting in children with attention-deficit/hyperactivity disorder.

Attention-deficit/hyperactivity disorder (ADHD) is a prevalent psychiatric disorder that has poor long-term outcomes and remains a major public health concern. Recent theories have proposed that ADHD arises from alterations in multiple neural pathways. Alterations in reward circuits are hypothesized as one core dysfunction, leading to altered processing of anticipated rewards. The nucleus accumbens (NAcc) is particularly important for reward processes; task-based fMRI studies have found atypical activation of this region while the participants performed a reward task. Understanding how reward circuits are involved with ADHD may be further enhanced by considering how the NAcc interacts with other brain regions. Here we used the technique of resting-state functional connectivity MRI (rs-fcMRI) to examine the alterations in the NAcc interactions and how they relate to impulsive decision making in ADHD. Using rs-fcMRI, this study: examined differences in functional connectivity of the NAcc between children with ADHD and control children; correlated the functional connectivity of NAcc with impulsivity, as measured by a delay discounting task; and combined these two initial segments to identify the atypical NAcc connections that were associated with impulsive decision making in ADHD. We found that functional connectivity of NAcc was atypical in children with ADHD and the ADHD-related increased connectivity between NAcc and the prefrontal cortex was associated with greater impulsivity (steeper delayed-reward discounting). These findings are consistent with the hypothesis that atypical signaling of the NAcc to the prefrontal cortex in ADHD may lead to excessive approach and failure in estimating future consequences; thus, leading to impulsive behavior.

Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD.

Research and clinical investigations in psychiatry largely rely on the de facto assumption that the diagnostic categories identified in the Diagnostic and Statistical Manual (DSM) represent homogeneous syndromes. However, the mechanistic heterogeneity that potentially underlies the existing classification scheme might limit discovery of etiology for most developmental psychiatric disorders. Another, perhaps less palpable, reality may also be interfering with progress-heterogeneity in typically developing populations. In this report we attempt to clarify neuropsychological heterogeneity in a large dataset of typically developing youth and youth with attention deficit/hyperactivity disorder (ADHD), using graph theory and community detection. We sought to determine whether data-driven neuropsychological subtypes could be discerned in children with and without the disorder. Because individual classification is the sine qua non for eventual clinical translation, we also apply support vector machine-based multivariate pattern analysis to identify how well ADHD status in individual children can be identified as defined by the community detection delineated subtypes. The analysis yielded several unique, but similar subtypes across both populations. Just as importantly, comparing typically developing children with ADHD children within each of these distinct subgroups increased diagnostic accuracy. Two important principles were identified that have the potential to advance our understanding of typical development and developmental neuropsychiatric disorders. The first tenet suggests that typically developing children can be classified into distinct neuropsychological subgroups with high precision. The second tenet proposes that some of the heterogeneity in individuals with ADHD might be "nested" in this normal variation.

Altered cortico-striatal-thalamic connectivity in relation to spatial working memory capacity in children with ADHD.

INTRODUCTION: Attention deficit hyperactivity disorder (ADHD) captures a heterogeneous group of children, who are characterized by a range of cognitive and behavioral symptoms. Previous resting-state functional connectivity MRI (rs-fcMRI) studies have sought to understand the neural correlates of ADHD by comparing connectivity measurements between those with and without the disorder, focusing primarily on cortical-striatal circuits mediated by the thalamus. To integrate the multiple phenotypic features associated with ADHD and help resolve its heterogeneity, it is helpful to determine how specific circuits relate to unique cognitive domains of the ADHD syndrome. Spatial working memory has been proposed as a key mechanism in the pathophysiology of ADHD. METHODS: We correlated the rs-fcMRI of five thalamic regions of interest (ROIs) with spatial span working memory scores in a sample of 67 children aged 7-11 years [ADHD and typically developing children (TDC)]. In an independent dataset, we then examined group differences in thalamo-striatal functional connectivity between 70 ADHD and 89 TDC (7-11 years) from the ADHD-200 dataset. Thalamic ROIs were created based on previous methods that utilize known thalamo-cortical loops and rs-fcMRI to identify functional boundaries in the thalamus. RESULTS/CONCLUSION: Using these thalamic regions, we found atypical rs-fcMRI between specific thalamic groupings with the basal ganglia. To identify the thalamic connections that relate to spatial working memory in ADHD, only connections identified in both the correlational and comparative analyses were considered. Multiple connections between the thalamus and basal ganglia, particularly between medial and anterior dorsal thalamus and the putamen, were related to spatial working memory and also altered in ADHD. These thalamo-striatal disruptions may be one of multiple atypical neural and cognitive mechanisms that relate to the ADHD clinical phenotype.

Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data.

In recent years, there has been growing enthusiasm that functional magnetic resonance imaging (MRI) could achieve clinical utility for a broad range of neuropsychiatric disorders. However, several barriers remain. For example, the acquisition of large-scale datasets capable of clarifying the marked heterogeneity that exists in psychiatric illnesses will need to be realized. In addition, there continues to be a need for the development of image processing and analysis methods capable of separating signal from artifact. As a prototypical hyperkinetic disorder, and movement-related artifact being a significant confound in functional imaging studies, ADHD offers a unique challenge. As part of the ADHD-200 Global Competition and this special edition of Frontiers, the ADHD-200 Consortium demonstrates the utility of an aggregate dataset pooled across five institutions in addressing these challenges. The work aimed to (1) examine the impact of emerging techniques for controlling for "micro-movements," and (2) provide novel insights into the neural correlates of ADHD subtypes. Using support vector machine (SVM)-based multivariate pattern analysis (MVPA) we show that functional connectivity patterns in individuals are capable of differentiating the two most prominent ADHD subtypes. The application of graph-theory revealed that the Combined (ADHD-C) and Inattentive (ADHD-I) subtypes demonstrated some overlapping (particularly sensorimotor systems), but unique patterns of atypical connectivity. For ADHD-C, atypical connectivity was prominent in midline default network components, as well as insular cortex; in contrast, the ADHD-I group exhibited atypical patterns within the dlPFC regions and cerebellum. Systematic motion-related artifact was noted, and highlighted the need for stringent motion correction. Findings reported were robust to the specific motion correction strategy employed. These data suggest that resting-state functional connectivity MRI (rs-fcMRI) data can be used to characterize individual patients with ADHD and to identify neural distinctions underlying the clinical heterogeneity of ADHD.

Altered white matter microstructure in children with attention-deficit/hyperactivity disorder.

OBJECTIVE: Identification of biomarkers is a priority for attention-deficit/hyperactivity disorder (ADHD). Studies have documented macrostructural brain alterations in ADHD, but few have examined white matter microstructure, particularly in preadolescent children. Given dramatic white matter maturation across childhood, microstructural differences seen in adolescents and adults with ADHD may reflect compensatory restructuring, rather than early neurophenotypic markers of the disorder. METHOD: Using tract-based spatial statistics, mean fractional anisotropy (FA) maps were created using diffusion tensor imaging. FA, mean diffusivity (MD), and associated axial and radial diffusivities were compared between 16 children with ADHD and 20 healthy children (age 7-9 years). RESULTS: Youth with ADHD showed decreased FA in frontoparietal, frontolimbic, cerebellar, corona radiata, and temporo-occipital white matter compared with controls. In addition, ADHD was associated with lower MD in the posterior limb of the internal capsule and frontoparietal white matter and greater MD in frontolimbic white matter. Lower axial diffusion and/or higher radial diffusion were differentially observed for youth with ADHD in earlier versus later maturing areas of group FA/MD difference. CONCLUSIONS: This study suggests that, even prior to adolescence, ADHD represents a disorder of altered structural connectivity of the brain, characterized by distributed atypical white matter microstructure. In addition, later maturing frontolimbic pathways were abnormal in children with ADHD, likely due to delayed or decreased myelination, a finding not previously demonstrated in the adolescent or adult stages of the disorder. These results suggest that disruptions in white matter microstructure may play a key role in the early pathophysiology of ADHD.

Atypical default network connectivity in youth with attention-deficit/hyperactivity disorder.

BACKGROUND: Attention-deficit/hyperactivity disorder (ADHD) is a major public health concern. It has been suggested that the brain's default network may provide a crucial avenue for understanding the neurobiology of attention deficit/hyperactivity disorder. Evaluations of the default network have increased over recent years with the applied technique of resting-state functional connectivity magnetic resonance imaging (rs-fcMRI). These investigations have established that spontaneous activity in this network is highly correlated at rest in young adult populations. This coherence seems to be reduced in adults with ADHD. This is an intriguing finding, as coherence in spontaneous activity within the default network strengthens with age. Thus, the pathophysiology of ADHD might include delayed or disrupted maturation of the default network. If so, it is important to determine whether an altered developmental picture can be detected using rs-fcMRI in children with ADHD. METHODS: This study used the typical developmental context provided previously by Fair et al. (2008) to examine coherence of brain activity within the default network using rs-fcMRI in children with (n = 23) and without attention deficit/hyperactivity disorder (n = 23). RESULTS: We found that functional connections previously shown as developmentally dynamic in the default network were atypical in children with attention deficit/hyperactivity disorder-consistent with perturbation or failure of the maturational processes. CONCLUSIONS: These findings are consistent with the hypothesis that atypical consolidation of this network over development plays a role in attention deficit/hyperactivity disorder.

Maturing thalamocortical functional connectivity across development.

Recent years have witnessed a surge of investigations examining functional brain organization using resting-state functional connectivity MRI (rs-fcMRI). To date, this method has been used to examine systems organization in typical and atypical developing populations. While the majority of these investigations have focused on cortical-cortical interactions, cortical-subcortical interactions also mature into adulthood. Innovative work by Zhang et al. (2008) in adults have identified methods that utilize rs-fcMRI and known thalamo-cortical topographic segregation to identify functional boundaries in the thalamus that are remarkably similar to known thalamic nuclear grouping. However, despite thalamic nuclei being well formed early in development, the developmental trajectory of functional thalamo-cortical relations remains unexplored. Thalamic maps generated by rs-fcMRI are based on functional relationships, and should modify with the dynamic thalamo-cortical changes that occur throughout maturation. To examine this possibility, we employed a strategy as previously described by Zhang et al. to a sample of healthy children, adolescents, and adults. We found strengthening functional connectivity of the cortex with dorsal/anterior subdivisions of the thalamus, with greater connectivity observed in adults versus children. Temporal lobe connectivity with ventral/midline/posterior subdivisions of the thalamus weakened with age. Changes in sensory and motor thalamo-cortical interactions were also identified but were limited. These findings are consistent with known anatomical and physiological cortical-subcortical changes over development. The methods and developmental context provided here will be important for understanding how cortical-subcortical interactions relate to models of typically developing behavior and developmental neuropsychiatric disorders.

Bayesian analysis of fMRI data with ICA based spatial prior.

Spatial modeling is essential for fMRI analysis due to relatively high noise in the data. Earlier approaches have been primarily concerned with the spatial coherence of the BOLD response in local neighborhoods. In addition to a smoothness constraint, we propose to incorporate prior knowledge of brain activation patterns learned from training samples. This spatially informed prior can significantly enhance the estimation process by inducing sensitivity to task related regions of the brain. As fMRI data exhibits intersubject variability in functional anatomy, we design the prior using Independent Component Analysis (ICA). Due to the non-Gaussian assumption, ICA does not regress to the mean activation pattern and thus avoids suppressing intersubject differences. Results from a real fMRI experiment indicate that our approach provides statistically significant improvement in estimating activation compared to the standard general linear model (GLM) based methods.