Development and validation of an interpretable deep learning framework for Alzheimer's disease classification.

Alzheimer's disease is the primary cause of dementia worldwide, with an increasing morbidity burden that may outstrip diagnosis and management capacity as the population ages. Current methods integrate patient history, neuropsychological testing and MRI to identify likely cases, yet effective practices remain variably applied and lacking in sensitivity and specificity. Here we report an interpretable deep learning strategy that delineates unique Alzheimer's disease signatures from multimodal inputs of MRI, age, gender, and Mini-Mental State Examination score. Our framework linked a fully convolutional network, which constructs high resolution maps of disease probability from local brain structure to a multilayer perceptron and generates precise, intuitive visualization of individual Alzheimer's disease risk en route to accurate diagnosis. The model was trained using clinically diagnosed Alzheimer's disease and cognitively normal subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset (n = 417) and validated on three independent cohorts: the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) (n = 382), the Framingham Heart Study (n = 102), and the National Alzheimer's Coordinating Center (NACC) (n = 582). Performance of the model that used the multimodal inputs was consistent across datasets, with mean area under curve values of 0.996, 0.974, 0.876 and 0.954 for the ADNI study, AIBL, Framingham Heart Study and NACC datasets, respectively. Moreover, our approach exceeded the diagnostic performance of a multi-institutional team of practicing neurologists (n = 11), and high-risk cerebral regions predicted by the model closely tracked post-mortem histopathological findings. This framework provides a clinically adaptable strategy for using routinely available imaging techniques such as MRI to generate nuanced neuroimaging signatures for Alzheimer's disease diagnosis, as well as a generalizable approach for linking deep learning to pathophysiological processes in human disease.

Correction to: Assessment of knee pain from MR imaging using a convolutional Siamese network.

The original version of this article, published on 13 February 2020, unfortunately contained a mistake.

Assessment of knee pain from MR imaging using a convolutional Siamese network.

OBJECTIVES: It remains difficult to characterize the source of pain in knee joints either using radiographs or magnetic resonance imaging (MRI). We sought to determine if advanced machine learning methods such as deep neural networks could distinguish knees with pain from those without it and identify the structural features that are associated with knee pain. METHODS: We constructed a convolutional Siamese network to associate MRI scans obtained on subjects from the Osteoarthritis Initiative (OAI) with frequent unilateral knee pain comparing the knee with frequent pain to the contralateral knee without pain. The Siamese network architecture enabled pairwise learning of information from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices obtained from similar locations on both knees. Class activation mapping (CAM) was utilized to create saliency maps, which highlighted the regions most associated with knee pain. The MRI scans and the CAMs of each subject were reviewed by an expert radiologist to identify the presence of abnormalities within the model-predicted regions of high association. RESULTS: Using 10-fold cross-validation, our model achieved an area under curve (AUC) value of 0.808. When individuals whose knee WOMAC pain scores were not discordant were excluded, model performance increased to 0.853. The radiologist review revealed that about 86% of the cases that were predicted correctly had effusion-synovitis within the regions that were most associated with pain. CONCLUSIONS: This study demonstrates a proof of principle that deep learning can be applied to assess knee pain from MRI scans. KEY POINTS: • Our article is the first to leverage a deep learning framework to associate MR images of the knee with knee pain. • We developed a convolutional Siamese network that had the ability to fuse information from multiple two-dimensional (2D) MRI slices from the knee with pain and the contralateral knee of the same individual without pain to predict unilateral knee pain. • Our model achieved an area under curve (AUC) value of 0.808. When individuals who had WOMAC pain scores that were not discordant for knees (pain discordance < 3) were excluded, model performance increased to 0.853.

Deep learning for risk-based stratification of cognitively impaired individuals.

Quantifying the risk of progression to Alzheimer's disease (AD) could help identify persons who could benefit from early interventions. We used data from the Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 544, discovery cohort) and the National Alzheimer's Coordinating Center (NACC, n = 508, validation cohort), subdividing individuals with mild cognitive impairment (MCI) into risk groups based on cerebrospinal fluid amyloid-ß levels and identifying differential gray matter patterns. We then created models that fused neural networks with survival analysis, trained using non-parcellated T1-weighted brain MRIs from ADNI data, to predict the trajectories of MCI to AD conversion within the NACC cohort (integrated Brier score: 0.192 [discovery], and 0.108 [validation]). Using modern interpretability techniques, we verified that regions important for model prediction are classically associated with AD. We confirmed AD diagnosis labels using postmortem data. We conclude that our framework provides a strategy for risk-based stratification of individuals with MCI and for identifying regions key for disease prognosis.

Multimodal deep learning for Alzheimer's disease dementia assessment.

Worldwide, there are nearly 10 million new cases of dementia annually, of which Alzheimer's disease (AD) is the most common. New measures are needed to improve the diagnosis of individuals with cognitive impairment due to various etiologies. Here, we report a deep learning framework that accomplishes multiple diagnostic steps in successive fashion to identify persons with normal cognition (NC), mild cognitive impairment (MCI), AD, and non-AD dementias (nADD). We demonstrate a range of models capable of accepting flexible combinations of routinely collected clinical information, including demographics, medical history, neuropsychological testing, neuroimaging, and functional assessments. We then show that these frameworks compare favorably with the diagnostic accuracy of practicing neurologists and neuroradiologists. Lastly, we apply interpretability methods in computer vision to show that disease-specific patterns detected by our models track distinct patterns of degenerative changes throughout the brain and correspond closely with the presence of neuropathological lesions on autopsy. Our work demonstrates methodologies for validating computational predictions with established standards of medical diagnosis.

Enhancing magnetic resonance imaging-driven Alzheimer's disease classification performance using generative adversarial learning.

BACKGROUND: Generative adversarial networks (GAN) can produce images of improved quality but their ability to augment image-based classification is not fully explored. We evaluated if a modified GAN can learn from magnetic resonance imaging (MRI) scans of multiple magnetic field strengths to enhance Alzheimer's disease (AD) classification performance. METHODS: T1-weighted brain MRI scans from 151 participants of the Alzheimer's Disease Neuroimaging Initiative (ADNI), who underwent both 1.5-Tesla (1.5-T) and 3-Tesla imaging at the same time were selected to construct a GAN model. This model was trained along with a three-dimensional fully convolutional network (FCN) using the generated images (3T*) as inputs to predict AD status. Quality of the generated images was evaluated using signal to noise ratio (SNR), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Natural Image Quality Evaluator (NIQE). Cases from the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n = 107) and the National Alzheimer's Coordinating Center (NACC, n = 565) were used for model validation. RESULTS: The 3T*-based FCN classifier performed better than the FCN model trained using the 1.5-T scans. Specifically, the mean area under curve increased from 0.907 to 0.932, from 0.934 to 0.940, and from 0.870 to 0.907 on the ADNI test, AIBL, and NACC datasets, respectively. Additionally, we found that the mean quality of the generated (3T*) images was consistently higher than the 1.5-T images, as measured using SNR, BRISQUE, and NIQE on the validation datasets. CONCLUSION: This study demonstrates a proof of principle that GAN frameworks can be constructed to augment AD classification performance and improve image quality.

Fusion of deep learning models of MRI scans, Mini-Mental State Examination, and logical memory test enhances diagnosis of mild cognitive impairment.

INTRODUCTION: Our aim was to investigate if the accuracy of diagnosing mild cognitive impairment (MCI) using the Mini-Mental State Examination (MMSE) and logical memory (LM) test could be enhanced by adding MRI data. METHODS: Data of individuals with normal cognition and MCI were obtained from the National Alzheimer Coordinating Center database (n = 386). Deep learning models trained on MRI slices were combined to generate a fused MRI model using different voting techniques to predict normal cognition versus MCI. Two multilayer perceptron (MLP) models were developed with MMSE and LM test results. Finally, the fused MRI model and the MLP models were combined using majority voting. RESULTS: The fusion model was superior to the individual models alone and achieved an overall accuracy of 90.9%. DISCUSSION: This study is a proof of principle that multimodal fusion of models developed using MRI scans, MMSE, and LM test data is feasible and can better predict MCI.

Improving resolution in quantum subnanometre-gap tip-enhanced Raman nanoimaging.

Two-dimensional (2D) materials beyond graphene such as transition metal dichalcogenides (TMDs) have unique mechanical, optical and electronic properties with promising applications in flexible devices, catalysis and sensing. Optical imaging of TMDs using photoluminescence and Raman spectroscopy can reveal the effects of structure, strain, doping, edge states, and surface functionalization from materials to bioscience. However, Raman signals are inherently weak and so far have been limited in spatial resolution in TMDs to a few hundred nanometres which is much larger than the intrinsic scale of these effects. Here we overcome the diffraction limit by using resonant tip-enhanced Raman scattering (TERS) of few-layer MoS2, and obtain nanoscale optical images with ~20 nm spatial resolution. This becomes possible due to electric field enhancement in an optimized subnanometre-gap resonant tip-substrate configuration. We investigate the limits of signal enhancement by varying the tip-sample gap with sub-Angstrom precision and observe a quantum quenching behavior, as well as a Schottky-Ohmic transition, for subnanometre gaps, which enable surface mapping based on this new contrast mechanism. This quantum regime of plasmonic gap-mode enhancement with a few nanometre thick MoS2 junction may be used for designing new quantum optoelectronic devices and sensors with a wide range of applications.