Interactive Explainable Deep Learning Model Informs Prostate Cancer Diagnosis at MRI.

Background Clinically significant prostate cancer (PCa) diagnosis at MRI requires accurate and efficient radiologic interpretation. Although artificial intelligence may assist in this task, lack of transparency has limited clinical translation. Purpose To develop an explainable artificial intelligence (XAI) model for clinically significant PCa diagnosis at biparametric MRI using Prostate Imaging Reporting and Data System (PI-RADS) features for classification justification. Materials and Methods This retrospective study included consecutive patients with histopathologic analysis-proven prostatic lesions who underwent biparametric MRI and biopsy between January 2012 and December 2017. After image annotation by two radiologists, a deep learning model was trained to detect the index lesion; classify PCa, clinically significant PCa (Gleason score ≥ 7), and benign lesions (eg, prostatitis); and justify classifications using PI-RADS features. Lesion- and patient-based performance were assessed using fivefold cross validation and areas under the receiver operating characteristic curve. Clinical feasibility was tested in a multireader study and by using the external PROSTATEx data set. Statistical evaluation of the multireader study included Mann-Whitney U and exact Fisher-Yates test. Results Overall, 1224 men (median age, 67 years; IQR, 62-73 years) had 3260 prostatic lesions (372 lesions with Gleason score of 6; 743 lesions with Gleason score of ≥ 7; 2145 benign lesions). XAI reliably detected clinically significant PCa in internal (area under the receiver operating characteristic curve, 0.89) and external test sets (area under the receiver operating characteristic curve, 0.87) with a sensitivity of 93% (95% CI: 87, 98) and an average of one false-positive finding per patient. Accuracy of the visual and textual explanations of XAI classifications was 80% (1080 of 1352), confirmed by experts. XAI-assisted readings improved the confidence (4.1 vs 3.4 on a five-point Likert scale; P = .007) of nonexperts in assessing PI-RADS 3 lesions, reducing reading time by 58 seconds (P = .009). Conclusion The explainable AI model reliably detected and classified clinically significant prostate cancer and improved the confidence and reading time of nonexperts while providing visual and textual explanations using well-established imaging features. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Chapiro in this issue.

Application Scenarios for Artificial Intelligence in Nursing Care: Rapid Review.

BACKGROUND: Artificial intelligence (AI) holds the promise of supporting nurses' clinical decision-making in complex care situations or conducting tasks that are remote from direct patient interaction, such as documentation processes. There has been an increase in the research and development of AI applications for nursing care, but there is a persistent lack of an extensive overview covering the evidence base for promising application scenarios. OBJECTIVE: This study synthesizes literature on application scenarios for AI in nursing care settings as well as highlights adjacent aspects in the ethical, legal, and social discourse surrounding the application of AI in nursing care. METHODS: Following a rapid review design, PubMed, CINAHL, Association for Computing Machinery Digital Library, Institute of Electrical and Electronics Engineers Xplore, Digital Bibliography & Library Project, and Association for Information Systems Library, as well as the libraries of leading AI conferences, were searched in June 2020. Publications of original quantitative and qualitative research, systematic reviews, discussion papers, and essays on the ethical, legal, and social implications published in English were included. Eligible studies were analyzed on the basis of predetermined selection criteria. RESULTS: The titles and abstracts of 7016 publications and 704 full texts were screened, and 292 publications were included. Hospitals were the most prominent study setting, followed by independent living at home; fewer application scenarios were identified for nursing homes or home care. Most studies used machine learning algorithms, whereas expert or hybrid systems were entailed in less than every 10th publication. The application context of focusing on image and signal processing with tracking, monitoring, or the classification of activity and health followed by care coordination and communication, as well as fall detection, was the main purpose of AI applications. Few studies have reported the effects of AI applications on clinical or organizational outcomes, lacking particularly in data gathered outside laboratory conditions. In addition to technological requirements, the reporting and inclusion of certain requirements capture more overarching topics, such as data privacy, safety, and technology acceptance. Ethical, legal, and social implications reflect the discourse on technology use in health care but have mostly not been discussed in meaningful and potentially encompassing detail. CONCLUSIONS: The results highlight the potential for the application of AI systems in different nursing care settings. Considering the lack of findings on the effectiveness and application of AI systems in real-world scenarios, future research should reflect on a more nursing care-specific perspective toward objectives, outcomes, and benefits. We identify that, crucially, an advancement in technological-societal discourse that surrounds the ethical and legal implications of AI applications in nursing care is a necessary next step. Further, we outline the need for greater participation among all of the stakeholders involved.

On the interpretation of weight vectors of linear models in multivariate neuroimaging.

The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain-computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses.

Stereoscopic depth increases intersubject correlations of brain networks.

Three-dimensional movies presented via stereoscopic displays have become more popular in recent years aiming at a more engaging viewing experience. However, neurocognitive processes associated with the perception of stereoscopic depth in complex and dynamic visual stimuli remain understudied. Here, we investigate the influence of stereoscopic depth on both neurophysiology and subjective experience. Using multivariate statistical learning methods, we compare the brain activity of subjects when freely watching the same movies in 2D and in 3D. Subjective reports indicate that 3D movies are more strongly experienced than 2D movies. On the neural level, we observe significantly higher intersubject correlations of cortical networks when subjects are watching 3D movies relative to the same movies in 2D. We demonstrate that increases in intersubject correlations of brain networks can serve as neurophysiological marker for stereoscopic depth and for the strength of the viewing experience.

Metadata-independent classification of MRI sequences using convolutional neural networks: Successful application to prostate MRI.

PURPOSE: The ever-increasing volume of medical imaging data and interest in Big Data research brings challenges to data organization, categorization, and retrieval. Although the radiological value chain is almost entirely digital, data structuring has been widely performed pragmatically, but with insufficient naming and metadata standards for the stringent needs of image analysis. To enable automated data management independent of naming and metadata, this study focused on developing a convolutional neural network (CNN) that classifies medical images based solely on voxel data. METHOD: A 3D CNN (3D-ResNet18) was trained using a dataset of 31,602 prostate MRI volumes with 10 different sequence types of 1243 patients. A five-fold cross-validation approach with patient-based splits was chosen for training and testing. Training was repeated with a gradual reduction in training data assessing classification accuracies to determine the minimum training data required for sufficient performance. The trained model and developed method were tested on three external datasets. RESULTS: The model achieved an overall accuracy of 99.88 % ± 0.13 % in classifying typical prostate MRI sequence types. When being trained with approximately 10 % of the original cohort (112 patients), the CNN still achieved an accuracy of 97.43 % ± 2.10 %. In external testing the model achieved sensitivities of > 90 % for 10/15 tested sequence types. CONCLUSIONS: The herein developed CNN enabled automatic and reliable sequence identification in prostate MRI. Ultimately, such CNN models for voxel-based sequence identification could substantially enhance the management of medical imaging data, improve workflow efficiency and data quality, and allow for robust clinical AI workflows.

Improved decoding of neural activity from fMRI signals using non-separable spatiotemporal deconvolutions.

The goal of most functional Magnetic Resonance Imaging (fMRI) analyses is to investigate neural activity. Many fMRI analysis methods assume that the temporal dynamics of the hemodynamic response function (HRF) to neural activation is separable from its spatial dynamics. Although there is empirical evidence that the HRF is more complex than suggested by space-time separable canonical HRF models, it is difficult to assess how much information about neural activity is lost when assuming space-time separability. In this study we directly test whether spatiotemporal variability in the HRF that is not captured by separable models contains information about neural signals. We predict intracranially measured neural activity from simultaneously recorded fMRI data using separable and non-separable spatiotemporal deconvolutions of voxel time series around the recording electrode. Our results show that abandoning the spatiotemporal separability assumption consistently improves the decoding accuracy of neural signals from fMRI data. We compare our findings with results from optical imaging and fMRI studies and discuss potential implications for classical fMRI analyses without invasive electrophysiological recordings.

Functional and laminar dissociations between muscarinic and nicotinic cholinergic neuromodulation in the tree shrew primary visual cortex.

Acetylcholine is an important neuromodulator involved in cognitive function. The impact of cholinergic neuromodulation on computations within the cortical microcircuit is not well understood. Here we investigate the effects of layer-specific cholinergic drug application in the tree shrew primary visual cortex during visual stimulation with drifting grating stimuli of varying contrast and orientation. We describe differences between muscarinic and nicotinic cholinergic effects in terms of both the layer of cortex and the attribute of visual representation. Nicotinic receptor activation enhanced the contrast response in the granular input layer of the cortex, while tending to reduce neural selectivity for orientation across all cortical layers. Muscarinic activation modestly enhanced the contrast response across cortical layers, and tended to improve orientation tuning. This resulted in highest orientation selectivity in the supra- and infragranular layers, where orientation selectivity was already greatest in the absence of pharmacological stimulation. Our results indicate that laminar position plays a crucial part in functional consequences of cholinergic stimulation, consistent with the differential distribution of cholinergic receptors. Nicotinic receptors function to enhance sensory representations arriving in the cortex, whereas muscarinic receptors act to boost the cortical computation of orientation tuning. Our findings suggest close homology between cholinergic mechanisms in tree shrew and primate visual cortices.

A Benchmark for Data Imputation Methods.

With the increasing importance and complexity of data pipelines, data quality became one of the key challenges in modern software applications. The importance of data quality has been recognized beyond the field of data engineering and database management systems (DBMSs). Also, for machine learning (ML) applications, high data quality standards are crucial to ensure robust predictive performance and responsible usage of automated decision making. One of the most frequent data quality problems is missing values. Incomplete datasets can break data pipelines and can have a devastating impact on downstream ML applications when not detected. While statisticians and, more recently, ML researchers have introduced a variety of approaches to impute missing values, comprehensive benchmarks comparing classical and modern imputation approaches under fair and realistic conditions are underrepresented. Here, we aim to fill this gap. We conduct a comprehensive suite of experiments on a large number of datasets with heterogeneous data and realistic missingness conditions, comparing both novel deep learning approaches and classical ML imputation methods when either only test or train and test data are affected by missing data. Each imputation method is evaluated regarding the imputation quality and the impact imputation has on a downstream ML task. Our results provide valuable insights into the performance of a variety of imputation methods under realistic conditions. We hope that our results help researchers and engineers to guide their data preprocessing method selection for automated data quality improvement.

Regularized Linear Discriminant Analysis of EEG Features in Dementia Patients.

The present study explores if EEG spectral parameters can discriminate between healthy elderly controls (HC), Alzheimer's disease (AD) and vascular dementia (VaD) using. We considered EEG data recorded during normal clinical routine with 114 healthy controls (HC), 114 AD, and 114 VaD patients. The spectral features extracted from the EEG were the absolute delta power, decay from lower to higher frequencies, amplitude, center and dispersion of the alpha power and baseline power of the entire frequency spectrum. For discrimination, we submitted these EEG features to regularized linear discriminant analysis algorithm with a 10-fold cross-validation. To check the consistency of the results obtained by our classifiers, we applied bootstrap statistics. Four binary classifiers were used to discriminate HC from AD, HC from VaD, AD from VaD, and HC from dementia patients (AD or VaD). For each model, we measured the discrimination performance using the area under curve (AUC) and the accuracy of the cross-validation (cv-ACC). We applied this procedure using two different sets of predictors. The first set considered all the features extracted from the 22 channels. For the second set of features, we automatically rejected features poorly correlated with their labels. Fairly good results were obtained when discriminating HC from dementia patients with AD or VaD (AUC = 0.84). We also obtained AUC = 0.74 for discrimination of AD from HC, AUC = 0.77 for discrimination of VaD from HC, and finally AUC = 0.61 for discrimination of AD from VaD. Our models were able to separate HC from dementia patients, and also and to discriminate AD from VaD above chance. Our results suggest that these features may be relevant for the clinical assessment of patients with dementia.

Decoding Three-Dimensional Trajectory of Executed and Imagined Arm Movements From Electroencephalogram Signals.

Decoding motor commands from noninvasively measured neural signals has become important in brain-computer interface (BCI) research. Applications of BCI include neurorehabilitation after stroke and control of limb prostheses. Until now, most studies have tested simple movement trajectories in two dimensions by using constant velocity profiles. However, most real-world scenarios require much more complex movement trajectories and velocity profiles. In this study, we decoded motor commands in three dimensions from electroencephalography (EEG) recordings while the subjects either executed or observed/imagined complex upper limb movement trajectories. We compared the accuracy of simple linear methods and nonlinear methods. In line with previous studies our results showed that linear decoders are an efficient and robust method for decoding motor commands. However, while we took the same precautions as previous studies to suppress eye-movement related EEG contamination, we found that subtracting residual electro-oculogram activity from the EEG data resulted in substantially lower motor decoding accuracy for linear decoders. This effect severely limits the transfer of previous results to practical applications in which neural activation is targeted. We observed that nonlinear methods showed no such drop in decoding performance. Our results demonstrate that eye-movement related contamination of brain signals constitutes a severe problem for decoding motor signals from EEG data. These results are important for developing accurate decoders of motor signal from neural signals for use with BCI-based neural prostheses and neurorehabilitation in real-world scenarios.

Analysis of multimodal neuroimaging data.

Each method for imaging brain activity has technical or physiological limits. Thus, combinations of neuroimaging modalities that can alleviate these limitations such as simultaneous recordings of neurophysiological and hemodynamic activity have become increasingly popular. Multimodal imaging setups can take advantage of complementary views on neural activity and enhance our understanding about how neural information processing is reflected in each modality. However, dedicated analysis methods are needed to exploit the potential of multimodal methods. Many solutions to this data integration problem have been proposed, which often renders both comparisons of results and the choice of the right method for the data at hand difficult. In this review we will discuss different multimodal neuroimaging setups, the advances achieved in basic research and clinical application and the methods used. We will provide a comprehensive overview of mathematical tools reoccurring in multimodal neuroimaging studies for artifact removal, data-driven and model-driven analyses, enabling the practitioner to try established or new combinations from these algorithmic building blocks.

Relationship between neural and hemodynamic signals during spontaneous activity studied with temporal kernel CCA.

Functional magnetic resonance imaging (fMRI) based on the so-called blood oxygen level-dependent (BOLD) contrast is a powerful tool for studying brain function not only locally but also on the large scale. Most studies assume a simple relationship between neural and BOLD activity, in spite of the fact that it is important to elucidate how the "when" and "what" components of neural activity are correlated to the "where" of fMRI data. Here we conducted simultaneous recordings of neural and BOLD signal fluctuations in primary visual (V1) cortex of anesthetized monkeys. We explored the neurovascular relationship during periods of spontaneous activity by using temporal kernel canonical correlation analysis (tkCCA). tkCCA is a multivariate method that can take into account any features in the signals that univariate analysis cannot. The method detects filters in voxel space (for fMRI data) and in frequency-time space (for neural data) that maximize the neurovascular correlation without any assumption of a hemodynamic response function (HRF). Our results showed a positive neurovascular coupling with a lag of 4-5 s and a larger contribution from local field potentials (LFPs) in the γ range than from low-frequency LFPs or spiking activity. The method also detected a higher correlation around the recording site in the concurrent spatial map, even though the pattern covered most of the occipital part of V1. These results are consistent with those of previous studies and represent the first multivariate analysis of intracranial electrophysiology and high-resolution fMRI.