Distribution shift detection for the postmarket surveillance of medical AI algorithms: a retrospective simulation study.

Distribution shifts remain a problem for the safe application of regulated medical AI systems, and may impact their real-world performance if undetected. Postmarket shifts can occur for example if algorithms developed on data from various acquisition settings and a heterogeneous population are predominantly applied in hospitals with lower quality data acquisition or other centre-specific acquisition factors, or where some ethnicities are over-represented. Therefore, distribution shift detection could be important for monitoring AI-based medical products during postmarket surveillance. We implemented and evaluated three deep-learning based shift detection techniques (classifier-based, deep kernel, and multiple univariate kolmogorov-smirnov tests) on simulated shifts in a dataset of 130'486 retinal images. We trained a deep learning classifier for diabetic retinopathy grading. We then simulated population shifts by changing the prevalence of patients' sex, ethnicity, and co-morbidities, and example acquisition shifts by changes in image quality. We observed classification subgroup performance disparities w.r.t. image quality, patient sex, ethnicity and co-morbidity presence. The sensitivity at detecting referable diabetic retinopathy ranged from 0.50 to 0.79 for different ethnicities. This motivates the need for detecting shifts after deployment. Classifier-based tests performed best overall, with perfect detection rates for quality and co-morbidity subgroup shifts at a sample size of 1000. It was the only method to detect shifts in patient sex, but required large sample sizes ( > 3 0 ' 000 ). All methods identified easier-to-detect out-of-distribution shifts with small (≤300) sample sizes. We conclude that effective tools exist for detecting clinically relevant distribution shifts. In particular classifier-based tests can be easily implemented components in the post-market surveillance strategy of medical device manufacturers.

Visualizing single-cell data with the neighbor embedding spectrum.

The two-dimensional embedding methods t-SNE and UMAP are ubiquitously used for visualizing single-cell data. Recent theoretical research in machine learning has shown that, despite their very different formulation and implementation, t-SNE and UMAP are closely connected, and a single parameter suffices to interpolate between them. This leads to a whole spectrum of visualization methods that focus on different aspects of the data. Along the spectrum, this focus changes from representing local structures to representing continuous ones. In single-cell context, this leads to a trade-off between highlighting rare cell types or continuous variation, such as developmental trajectories. Visualizing the entire spectrum as an animation can provide a more nuanced understanding of the high-dimensional dataset than individual visualizations with either t-SNE or UMAP.

Interpretable detection of epiretinal membrane from optical coherence tomography with deep neural networks.

This study aimed to automatically detect epiretinal membranes (ERM) in various OCT-scans of the central and paracentral macula region and classify them by size using deep-neural-networks (DNNs). To this end, 11,061 OCT-images were included and graded according to the presence of an ERM and its size (small 100-1000 µm, large > 1000 µm). The data set was divided into training, validation and test sets (75%, 10%, 15% of the data, respectively). An ensemble of DNNs was trained and saliency maps were generated using Guided-Backprob. OCT-scans were also transformed into a one-dimensional-value using t-SNE analysis. The DNNs' receiver-operating-characteristics on the test set showed a high performance for no-ERM, small-ERM and large-ERM cases (AUC: 0.99, 0.92, 0.99, respectively; 3-way accuracy: 89%), with small-ERMs being the most difficult ones to detect. t-SNE analysis sorted cases by size and, in particular, revealed increased classification uncertainty at the transitions between groups. Saliency maps reliably highlighted ERM, regardless of the presence of other OCT features (i.e. retinal-thickening, intraretinal pseudo-cysts, epiretinal-proliferation) and entities such as ERM-retinoschisis, macular-pseudohole and lamellar-macular-hole. This study showed therefore that DNNs can reliably detect and grade ERMs according to their size not only in the fovea but also in the paracentral region. This is also achieved in cases of hard-to-detect, small-ERMs. In addition, the generated saliency maps can be used to highlight small-ERMs that might otherwise be missed. The proposed model could be used for screening-programs or decision-support-systems in the future.

Most discriminative stimuli for functional cell type clustering.

Identifying cell types and understanding their functional properties is crucial for unraveling the mechanisms underlying perception and cognition. In the retina, functional types can be identified by carefully selected stimuli, but this requires expert domain knowledge and biases the procedure towards previously known cell types. In the visual cortex, it is still unknown what functional types exist and how to identify them. Thus, for unbiased identification of the functional cell types in retina and visual cortex, new approaches are needed. Here we propose an optimization-based clustering approach using deep predictive models to obtain functional clusters of neurons using Most Discriminative Stimuli (MDS). Our approach alternates between stimulus optimization with cluster reassignment akin to an expectation-maximization algorithm. The algorithm recovers functional clusters in mouse retina, marmoset retina and macaque visual area V4. This demonstrates that our approach can successfully find discriminative stimuli across species, stages of the visual system and recording techniques. The resulting most discriminative stimuli can be used to assign functional cell types fast and on the fly, without the need to train complex predictive models or show a large natural scene dataset, paving the way for experiments that were previously limited by experimental time. Crucially, MDS are interpretable: they visualize the distinctive stimulus patterns that most unambiguously identify a specific type of neuron.

The art of seeing the elephant in the room: 2D embeddings of single-cell data do make sense.

A recent paper in PLOS Computational Biology (Chari and Pachter, 2023) claimed that t-SNE and UMAP embeddings of single-cell datasets fail to capture true biological structure. The authors argued that such embeddings are as arbitrary and as misleading as forcing the data into an elephant shape. Here we show that this conclusion was based on inadequate and limited metrics of embedding quality. More appropriate metrics quantifying neighborhood and class preservation reveal the elephant in the room: while t-SNE and UMAP embeddings of single-cell data do not represent high-dimensional distances, they can nevertheless provide biologically relevant information.

A paradigm shift?-On the ethics of medical large language models.

After a wave of breakthroughs in image-based medical diagnostics and risk prediction models, machine learning (ML) has turned into a normal science. However, prominent researchers are claiming that another paradigm shift in medical ML is imminent-due to most recent staggering successes of large language models-from single-purpose applications toward generalist models, driven by natural language. This article investigates the implications of this paradigm shift for the ethical debate. Focusing on issues like trust, transparency, threats of patient autonomy, responsibility issues in the collaboration of clinicians and ML models, fairness, and privacy, it will be argued that the main problems will be continuous with the current debate. However, due to functioning of large language models, the complexity of all these problems increases. In addition, the article discusses some profound challenges for the clinical evaluation of large language models and threats to the reproducibility and replicability of studies about large language models in medicine due to corporate interests.

Compound models and Pearson residuals for normalization of single-cell RNA-seq data without UMIs.

Before downstream analysis can reveal biological signals in single-cell RNA sequencing data, normalization and variance stabilization are required to remove technical noise. Recently, Pearson residuals based on negative binomial models have been suggested as an efficient normalization approach. These methods were developed for UMI-based sequencing protocols, where unique molecular identifiers (UMIs) help to remove PCR amplification noise by keeping track of the original molecules. In contrast, full-length protocols such as Smart-seq2 lack UMIs and retain amplification noise, making negative binomial models inapplicable. Here, we extend Pearson residuals to such read count data by modeling them as a compound process: we assume that the captured RNA molecules follow the negative binomial distribution, but are replicated according to an amplification distribution. Based on this model, we introduce compound Pearson residuals and show that they can be analytically obtained without explicit knowledge of the amplification distribution. Further, we demonstrate that compound Pearson residuals lead to a biologically meaningful gene selection and low-dimensional embeddings of complex Smart-seq2 datasets. Finally, we empirically study amplification distributions across several sequencing protocols, and suggest that they can be described by a broken power law. We show that the resulting compound distribution captures overdispersion and zero-inflation patterns characteristic of read count data. In summary, compound Pearson residuals provide an efficient and effective way to normalize read count data based on simple mechanistic assumptions.

Plasticity of neuronal dynamics in the lateral habenula for cue-punishment associative learning.

The brain's ability to associate threats with external stimuli is vital to execute essential behaviours including avoidance. Disruption of this process contributes instead to the emergence of pathological traits which are common in addiction and depression. However, the mechanisms and neural dynamics at the single-cell resolution underlying the encoding of associative learning remain elusive. Here, employing a Pavlovian discrimination task in mice we investigate how neuronal populations in the lateral habenula (LHb), a subcortical nucleus whose excitation underlies negative affect, encode the association between conditioned stimuli and a punishment (unconditioned stimulus). Large population single-unit recordings in the LHb reveal both excitatory and inhibitory responses to aversive stimuli. Additionally, local optical inhibition prevents the formation of cue discrimination during associative learning, demonstrating a critical role of LHb activity in this process. Accordingly, longitudinal in vivo two-photon imaging tracking LHb calcium neuronal dynamics during conditioning reveals an upward or downward shift of individual neurons' CS-evoked responses. While recordings in acute slices indicate strengthening of synaptic excitation after conditioning, support vector machine algorithms suggest that postsynaptic dynamics to punishment-predictive cues represent behavioral cue discrimination. To examine the presynaptic signaling in LHb participating in learning we monitored neurotransmitter dynamics with genetically-encoded indicators in behaving mice. While glutamate, GABA, and serotonin release in LHb remain stable across associative learning, we observe enhanced acetylcholine signaling developing throughout conditioning. In summary, converging presynaptic and postsynaptic mechanisms in the LHb underlie the transformation of neutral cues in valued signals supporting cue discrimination during learning.

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky.

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While we know how color information is processed in visual brain areas of primates, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of mouse natural scenes. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. This type of color-opponency in the receptive field center was not present at the level of the retinal output and, therefore, is likely computed in the cortex by integrating upstream visual signals. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of "predatory"-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species. More broadly, they support the hypothesis that visual cortex combines upstream information towards computing neuronal selectivity to behaviorally-relevant sensory features.

Multitask Learning for Activity Detection in Neovascular Age-Related Macular Degeneration.

Purpose: The purpose of this study was to provide a comparison of performance and explainability of a multitask convolutional deep neuronal network to single-task networks for activity detection in neovascular age-related macular degeneration (nAMD). Methods: From 70 patients (46 women and 24 men) who attended the University Eye Hospital Tübingen, 3762 optical coherence tomography B-scans (right eye = 2011 and left eye = 1751) were acquired with Heidelberg Spectralis, Heidelberg, Germany. B-scans were graded by a retina specialist and an ophthalmology resident, and then used to develop a multitask deep learning model to predict disease activity in neovascular age-related macular degeneration along with the presence of sub- and intraretinal fluid. We used performance metrics for comparison to single-task networks and visualized the deep neural network (DNN)-based decision with t-distributed stochastic neighbor embedding and clinically validated saliency mapping techniques. Results: The multitask model surpassed single-task networks in accuracy for activity detection (94.2% vs. 91.2%). The area under the curve of the receiver operating curve was 0.984 for the multitask model versus 0.974 for the single-task model. Furthermore, compared to single-task networks, visualizations via t-distributed stochastic neighbor embedding and saliency maps highlighted that multitask networks' decisions for activity detection in neovascular age-related macular degeneration were highly consistent with the presence of both sub- and intraretinal fluid. Conclusions: Multitask learning increases the performance of neuronal networks for predicting disease activity, while providing clinicians with an easily accessible decision control, which resembles human reasoning. Translational Relevance: By improving nAMD activity detection performance and transparency of automated decisions, multitask DNNs can support the translation of machine learning research into clinical decision support systems for nAMD activity detection.

Uncertainty, Evidence, and the Integration of Machine Learning into Medical Practice.

In light of recent advances in machine learning for medical applications, the automation of medical diagnostics is imminent. That said, before machine learning algorithms find their way into clinical practice, various problems at the epistemic level need to be overcome. In this paper, we discuss different sources of uncertainty arising for clinicians trying to evaluate the trustworthiness of algorithmic evidence when making diagnostic judgments. Thereby, we examine many of the limitations of current machine learning algorithms (with deep learning in particular) and highlight their relevance for medical diagnostics. Among the problems we inspect are the theoretical foundations of deep learning (which are not yet adequately understood), the opacity of algorithmic decisions, and the vulnerabilities of machine learning models, as well as concerns regarding the quality of medical data used to train the models. Building on this, we discuss different desiderata for an uncertainty amelioration strategy that ensures that the integration of machine learning into clinical settings proves to be medically beneficial in a meaningful way.

Center-surround interactions underlie bipolar cell motion sensitivity in the mouse retina.

Motion sensing is a critical aspect of vision. We studied the representation of motion in mouse retinal bipolar cells and found that some bipolar cells are radially direction selective, preferring the origin of small object motion trajectories. Using a glutamate sensor, we directly observed bipolar cells synaptic output and found that there are radial direction selective and non-selective bipolar cell types, the majority being selective, and that radial direction selectivity relies on properties of the center-surround receptive field. We used these bipolar cell receptive fields along with connectomics to design biophysical models of downstream cells. The models and additional experiments demonstrated that bipolar cells pass radial direction selective excitation to starburst amacrine cells, which contributes to their directional tuning. As bipolar cells provide excitation to most amacrine and ganglion cells, their radial direction selectivity may contribute to motion processing throughout the visual system.

Probabilistic solvers enable a straight-forward exploration of numerical uncertainty in neuroscience models.

Understanding neural computation on the mechanistic level requires models of neurons and neuronal networks. To analyze such models one typically has to solve coupled ordinary differential equations (ODEs), which describe the dynamics of the underlying neural system. These ODEs are solved numerically with deterministic ODE solvers that yield single solutions with either no, or only a global scalar error indicator on precision. It can therefore be challenging to estimate the effect of numerical uncertainty on quantities of interest, such as spike-times and the number of spikes. To overcome this problem, we propose to use recently developed sampling-based probabilistic solvers, which are able to quantify such numerical uncertainties. They neither require detailed insights into the kinetics of the models, nor are they difficult to implement. We show that numerical uncertainty can affect the outcome of typical neuroscience simulations, e.g. jittering spikes by milliseconds or even adding or removing individual spikes from simulations altogether, and demonstrate that probabilistic solvers reveal these numerical uncertainties with only moderate computational overhead.

Personalized neurorehabilitative precision medicine: from data to therapies (MWKNeuroReha) - a multi-centre prospective observational clinical trial to predict long-term outcome of patients with acute motor stroke.

BACKGROUND: Stroke is one of the most frequent diseases, and half of the stroke survivors are left with permanent impairment. Prediction of individual outcome is still difficult. Many but not all patients with stroke improve by approximately 1.7 times the initial impairment, that has been termed proportional recovery rule. The present study aims at identifying factors predicting motor outcome after stroke more accurately than before, and observe associations of rehabilitation treatment with outcome. METHODS: The study is designed as a multi-centre prospective clinical observational trial. An extensive primary data set of clinical, neuroimaging, electrophysiological, and laboratory data will be collected within 96 h of stroke onset from patients with relevant upper extremity deficit, as indexed by a Fugl-Meyer-Upper Extremity (FM-UE) score ≤ 50. At least 200 patients will be recruited. Clinical scores will include the FM-UE score (range 0-66, unimpaired function is indicated by a score of 66), Action Research Arm Test, modified Rankin Scale, Barthel Index and Stroke-Specific Quality of Life Scale. Follow-up clinical scores and applied types and amount of rehabilitation treatment will be documented in the rehabilitation hospitals. Final follow-up clinical scoring will be performed 90 days after the stroke event. The primary endpoint is the change in FM-UE defined as 90 days FM-UE minus initial FM-UE, divided by initial FM-UE impairment. Changes in the other clinical scores serve as secondary endpoints. Machine learning methods will be employed to analyze the data and predict primary and secondary endpoints based on the primary data set and the different rehabilitation treatments. DISCUSSION: If successful, outcome and relation to rehabilitation treatment in patients with acute motor stroke will be predictable more reliably than currently possible, leading to personalized neurorehabilitation. An important regulatory aspect of this trial is the first-time implementation of systematic patient data transfer between emergency and rehabilitation hospitals, which are divided institutions in Germany. TRIAL REGISTRATION: This study was registered at ClinicalTrials.gov ( NCT04688970 ) on 30 December 2020.

Clinical validation of saliency maps for understanding deep neural networks in ophthalmology.

Deep neural networks (DNNs) have achieved physician-level accuracy on many imaging-based medical diagnostic tasks, for example classification of retinal images in ophthalmology. However, their decision mechanisms are often considered impenetrable leading to a lack of trust by clinicians and patients. To alleviate this issue, a range of explanation methods have been proposed to expose the inner workings of DNNs leading to their decisions. For imaging-based tasks, this is often achieved via saliency maps. The quality of these maps are typically evaluated via perturbation analysis without experts involved. To facilitate the adoption and success of such automated systems, however, it is crucial to validate saliency maps against clinicians. In this study, we used three different network architectures and developed ensembles of DNNs to detect diabetic retinopathy and neovascular age-related macular degeneration from retinal fundus images and optical coherence tomography scans, respectively. We used a variety of explanation methods and obtained a comprehensive set of saliency maps for explaining the ensemble-based diagnostic decisions. Then, we systematically validated saliency maps against clinicians through two main analyses - a direct comparison of saliency maps with the expert annotations of disease-specific pathologies and perturbation analyses using also expert annotations as saliency maps. We found the choice of DNN architecture and explanation method to significantly influence the quality of saliency maps. Guided Backprop showed consistently good performance across disease scenarios and DNN architectures, suggesting that it provides a suitable starting point for explaining the decisions of DNNs on retinal images.

Retinal horizontal cells use different synaptic sites for global feedforward and local feedback signaling.

In the outer plexiform layer (OPL) of the mammalian retina, cone photoreceptors (cones) provide input to more than a dozen types of cone bipolar cells (CBCs). In the mouse, this transmission is modulated by a single horizontal cell (HC) type. HCs perform global signaling within their laterally coupled network but also provide local, cone-specific feedback. However, it is unknown how HCs provide local feedback to cones at the same time as global forward signaling to CBCs and where the underlying synapses are located. To assess how HCs simultaneously perform different modes of signaling, we reconstructed the dendritic trees of five HCs as well as cone axon terminals and CBC dendrites in a serial block-face electron microscopy volume and analyzed their connectivity. In addition to the fine HC dendritic tips invaginating cone axon terminals, we also identified "bulbs," short segments of increased dendritic diameter on the primary dendrites of HCs. These bulbs are in an OPL stratum well below the cone axon terminal base and make contacts with other HCs and CBCs. Our results from immunolabeling, electron microscopy, and glutamate imaging suggest that HC bulbs represent GABAergic synapses that do not receive any direct photoreceptor input. Together, our data suggest the existence of two synaptic strata in the mouse OPL, spatially separating cone-specific feedback and feedforward signaling to CBCs. A biophysical model of a HC dendritic branch and voltage imaging support the hypothesis that this spatial arrangement of synaptic contacts allows for simultaneous local feedback and global feedforward signaling by HCs.

[Ocular changes as a diagnostic tool for malaria]. / Okuläre Veränderungen als Hilfsmittel in der Malariadiagnostik.

BACKGROUND: According to the WHO Malaria Report 2019 a total of 229 million people fall ill with malaria each year and two thirds of deaths involve children under 5 years of age. AIM: To review the fundus changes in the context of malaria and the importance of ophthalmoscopy in the diagnosis. MATERIAL AND METHODS: Summary of changes in cerebral malaria visible on fundus examination, possible underlying pathomechanisms and the value of ophthalmoscopy in practice. RESULTS: Retinal findings in malaria include white or gray staining of the retina (retinal whitening), color change of retinal vessels (orange or white staining), hemorrhages often with a white center, such as Roth's spot and papilledema. DISCUSSION: The retinal changes in malaria are specific and may help to differentiate malaria from other causes of coma and fever. Smartphone-based fundus photography and artificial intelligence could support malaria diagnostics particularly in resource-poor regions.

How competitors become collaborators-Bridging the gap(s) between machine learning algorithms and clinicians.

For some years, we have been witnessing a steady stream of high-profile studies about machine learning (ML) algorithms achieving high diagnostic accuracy in the analysis of medical images. That said, facilitating successful collaboration between ML algorithms and clinicians proves to be a recalcitrant problem that may exacerbate ethical problems in clinical medicine. In this paper, we consider different epistemic and normative factors that may lead to algorithmic overreliance within clinical decision-making. These factors are false expectations, the miscalibration of uncertainties, non-explainability, and the socio-technical context within which the algorithms are utilized. Moreover, we identify different desiderata for bridging the gap between ML algorithms and clinicians. Further, we argue that there is an intriguing dialectic in the collaboration between clinicians and ML algorithms. While it is the algorithm that is supposed to assist the clinician in diagnostic tasks, successful collaboration will also depend on adjustments on the side of the clinician.

Ancestral circuits for vertebrate color vision emerge at the first retinal synapse.

For color vision, retinal circuits separate information about intensity and wavelength. In vertebrates that use the full complement of four "ancestral" cone types, the nature and implementation of this computation remain poorly understood. Here, we establish the complete circuit architecture of outer retinal circuits underlying color processing in larval zebrafish. We find that the synaptic outputs of red and green cones efficiently rotate the encoding of natural daylight in a principal components analysis­like manner to yield primary achromatic and spectrally opponent axes, respectively. Blue cones are tuned to capture most remaining variance when opposed to green cones, while UV cone present a UV achromatic axis for prey capture. We note that fruitflies use essentially the same strategy. Therefore, rotating color space into primary achromatic and chromatic axes at the eye's first synapse may thus be a fundamental principle of color vision when using more than two spectrally well-separated photoreceptor types.