[Explainable artificial intelligence in pathology]. / Erklärbare Künstliche Intelligenz in der Pathologie.

With the advancements in precision medicine, the demands on pathological diagnostics have increased, requiring standardized, quantitative, and integrated assessments of histomorphological and molecular pathological data. Great hopes are placed in artificial intelligence (AI) methods, which have demonstrated the ability to analyze complex clinical, histological, and molecular data for disease classification, biomarker quantification, and prognosis estimation. This paper provides an overview of the latest developments in pathology AI, discusses the limitations, particularly concerning the black box character of AI, and describes solutions to make decision processes more transparent using methods of so-called explainable AI (XAI).

Toward Explainable Artificial Intelligence for Precision Pathology.

The rapid development of precision medicine in recent years has started to challenge diagnostic pathology with respect to its ability to analyze histological images and increasingly large molecular profiling data in a quantitative, integrative, and standardized way. Artificial intelligence (AI) and, more precisely, deep learning technologies have recently demonstrated the potential to facilitate complex data analysis tasks, including clinical, histological, and molecular data for disease classification; tissue biomarker quantification; and clinical outcome prediction. This review provides a general introduction to AI and describes recent developments with a focus on applications in diagnostic pathology and beyond. We explain limitations including the black-box character of conventional AI and describe solutions to make machine learning decisions more transparent with so-called explainable AI. The purpose of the review is to foster a mutual understanding of both the biomedical and the AI side. To that end, in addition to providing an overview of the relevant foundations in pathology and machine learning, we present worked-through examples for a better practical understanding of what AI can achieve and how it should be done.

Single-cell gene regulatory network prediction by explainable AI.

The molecular heterogeneity of cancer cells contributes to the often partial response to targeted therapies and relapse of disease due to the escape of resistant cell populations. While single-cell sequencing has started to improve our understanding of this heterogeneity, it offers a mostly descriptive view on cellular types and states. To obtain more functional insights, we propose scGeneRAI, an explainable deep learning approach that uses layer-wise relevance propagation (LRP) to infer gene regulatory networks from static single-cell RNA sequencing data for individual cells. We benchmark our method with synthetic data and apply it to single-cell RNA sequencing data of a cohort of human lung cancers. From the predicted single-cell networks our approach reveals characteristic network patterns for tumor cells and normal epithelial cells and identifies subnetworks that are observed only in (subgroups of) tumor cells of certain patients. While current state-of-the-art methods are limited by their ability to only predict average networks for cell populations, our approach facilitates the reconstruction of networks down to the level of single cells which can be utilized to characterize the heterogeneity of gene regulation within and across tumors.

DNA methylation-based classification of sinonasal tumors.

The diagnosis of sinonasal tumors is challenging due to a heterogeneous spectrum of various differential diagnoses as well as poorly defined, disputed entities such as sinonasal undifferentiated carcinomas (SNUCs). In this study, we apply a machine learning algorithm based on DNA methylation patterns to classify sinonasal tumors with clinical-grade reliability. We further show that sinonasal tumors with SNUC morphology are not as undifferentiated as their current terminology suggests but rather reassigned to four distinct molecular classes defined by epigenetic, mutational and proteomic profiles. This includes two classes with neuroendocrine differentiation, characterized by IDH2 or SMARCA4/ARID1A mutations with an overall favorable clinical course, one class composed of highly aggressive SMARCB1-deficient carcinomas and another class with tumors that represent potentially previously misclassified adenoid cystic carcinomas. Our findings can aid in improving the diagnostic classification of sinonasal tumors and could help to change the current perception of SNUCs.

Patient-level proteomic network prediction by explainable artificial intelligence.

Understanding the pathological properties of dysregulated protein networks in individual patients' tumors is the basis for precision therapy. Functional experiments are commonly used, but cover only parts of the oncogenic signaling networks, whereas methods that reconstruct networks from omics data usually only predict average network features across tumors. Here, we show that the explainable AI method layer-wise relevance propagation (LRP) can infer protein interaction networks for individual patients from proteomic profiling data. LRP reconstructs average and individual interaction networks with an AUC of 0.99 and 0.93, respectively, and outperforms state-of-the-art network prediction methods for individual tumors. Using data from The Cancer Proteome Atlas, we identify known and potentially novel oncogenic network features, among which some are cancer-type specific and show only minor variation among patients, while others are present across certain tumor types but differ among individual patients. Our approach may therefore support predictive diagnostics in precision oncology by inferring "patient-level" oncogenic mechanisms.

Differences in Characteristics of Error-Related Potentials Between Individuals With Spinal Cord Injury and Age- and Sex-Matched Able-Bodied Controls.

Background: Non-invasive brain-computer interfaces (BCI) represent an emerging technology for enabling persons with impaired or lost grasping and reaching functions due to high spinal cord injury (SCI) to control assistive devices. A major drawback of BCIs is a high rate of false classifications. The robustness and performance of BCIs might be improved using cerebral electrophysiological correlates of error recognition (error-related potentials, ErrPs). As ErrPs have never been systematically examined in subjects with SCI, this study compares the characteristics of ErrPs in individuals with SCI with those of able-bodied control subjects. Methods: ErrPs at FCz and Cz were analyzed in 11 subjects with SCI (9 male, median age 28 y) and in 11 sex- and age-matched controls. Moving a shoulder joystick according to a visual cue, subjects received feedback about the match/mismatch of the performed movement. ErrPs occurring after "error"-feedback were evaluated by comparing means of voltage values within three consecutive time windows after feedback (wP1, wN1, wP2 containing peak voltages P1, N1, P2) using repeated-measurement analysis of variance. Results: In the control group, mean voltage values for the "error" and "correct" feedback condition differed significantly around N1 (FCz: 254 ms, Cz: 252 ms) and P2 (FCz: 347 ms, Cz: 345 ms), but not around P1 (FCz: 181 ms, Cz: 179 ms). ErrPs of the control and the SCI group showed similar morphology, however mean amplitudes of ErrPs were significantly smaller in individuals with SCI compared to controls for wN1 (FCz: control = -1.55 µV, SCI = -0.27 µV, p = 0.02; Cz: control = -1.03 µV, SCI = 0.11 µV, p = 0.04) and wP2 (FCz: control = 2.79 µV, SCI = 1.29 µV, p = 0.011; Cz: control = 2.12 µV, SCI = 0.81 µV, p = 0.003). Mean voltage values in wP1, wN1, and wP2 did not correlate significantly with either chronicity after or level of injury. Conclusion: The morphology of ErrPs in subjects with and without SCI is comparable, however, with reduced mean amplitude in wN1 and wP2 in the SCI group. Further studies should evaluate whether ErrP-classification can be used for online correction of false BCI-commands in individuals with SCI.

Sensory Feedback Interferes with Mu Rhythm Based Detection of Motor Commands from Electroencephalographic Signals.

Background: Electroencephalogram (EEG)-based brain-computer interfaces (BCI) represent a promising component of restorative motor therapies in individuals with partial paralysis. However, in those patients, sensory functions such as proprioception are at least partly preserved. The aim of this study was to investigate whether afferent feedback interferes with the BCI-based detection of efferent motor commands during execution of movements. Methods: Brain activity of 13 able-bodied subjects (age: 29.1 ± 4.8 years; 11 males) was compared between a motor task (MT) consisting of an isometric, isotonic grip and a somatosensory electrical stimulation (SS) of the fingertips. Modulation of the mu rhythm (8-13 Hz) was investigated to identify changes specifically related to the generation of efferent commands. A linear discriminant analysis (LDA) was used to investigate the activation pattern on a single-trial basis. Classifiers were trained with MT vs. REST (periods without MT/SS) and tested with SS and vice versa to quantify the impact of afferent feedback on the classification results. Results: Few differences in the spatial pattern between MT and SS were found in the modulation of the mu rhythm. All were characterized by event-related desynchronization (ERD) peaks at electrodes C3, C4, and CP3. Execution of the MT was associated with a significantly stronger ERD in the majority of sensorimotor electrodes [C3 (p < 0.01); CP3 (p < 0.05); C4 (p < 0.01)]. Classification accuracy of MT vs. REST was significantly higher than SS vs. REST (77% and 63%; p < 10-8). Classifiers trained on MT vs. REST were able to classify SS trials significantly above chance even though no motor commands were present during SS. Classifiers trained on SS performed better in classifying MT instead of SS. Conclusion: Our results challenge the notion that the modulation of the mu rhythm is a robust phenomenon for detecting efferent commands when afferent feedback is present. Instead, they indicate that the mu ERD caused by the processing of afferent feedback generates ERD patterns in the sensorimotor cortex that are masking the ERD patterns caused by the generation of efferent commands. Thus, processing of afferent feedback represents a considerable source of false positives when the mu rhythm is used for the detection of efferent commands.