Validating Automatic Concept-Based Explanations for AI-Based Digital Histopathology.

Digital histopathology poses several challenges such as label noise, class imbalance, limited availability of labelled data, and several latent biases to deep learning, negatively influencing transparency, reproducibility, and classification performance. In particular, biases are well known to cause poor generalization. Proposed tools from explainable artificial intelligence (XAI), bias detection, and bias discovery suffer from technical challenges, complexity, unintuitive usage, inherent biases, or a semantic gap. A promising XAI method, not studied in the context of digital histopathology is automated concept-based explanation (ACE). It automatically extracts visual concepts from image data. Our objective is to evaluate ACE's technical validity following design science principals and to compare it to Guided Gradient-weighted Class Activation Mapping (Grad-CAM), a conventional pixel-wise explanation method. To that extent, we created and studied five convolutional neural networks (CNNs) in four different skin cancer settings. Our results demonstrate that ACE is a valid tool for gaining insights into the decision process of histopathological CNNs that can go beyond explanations from the control method. ACE validly visualized a class sampling ratio bias, measurement bias, sampling bias, and class-correlated bias. Furthermore, the complementary use with Guided Grad-CAM offers several benefits. Finally, we propose practical solutions for several technical challenges. In contradiction to results from the literature, we noticed lower intuitiveness in some dermatopathology scenarios as compared to concept-based explanations on real-world images.

Evaluation of Open-Source and Pre-Trained Deep Convolutional Neural Networks Suitable for Player Detection and Motion Analysis in Squash.

In sport science, athlete tracking and motion analysis are essential for monitoring and optimizing training programs, with the goal of increasing success in competition and preventing injury. At present, contact-free, camera-based, multi-athlete detection and tracking have become a reality, mainly due to the advances in machine learning regarding computer vision and, specifically, advances in artificial convolutional neural networks (CNN), used for human pose estimation (HPE-CNN) in image sequences. Sport science in general, as well as coaches and athletes in particular, would greatly benefit from HPE-CNN-based tracking, but the sheer amount of HPE-CNNs available, as well as their complexity, pose a hurdle to the adoption of this new technology. It is unclear how many HPE-CNNs which are available at present are ready to use in out-of-the-box inference to squash, to what extent they allow motion analysis and if detections can easily be used to provide insight to coaches and athletes. Therefore, we conducted a systematic investigation of more than 250 HPE-CNNs. After applying our selection criteria of open-source, pre-trained, state-of-the-art and ready-to-use, five variants of three HPE-CNNs remained, and were evaluated in the context of motion analysis for the racket sport of squash. Specifically, we are interested in detecting player's feet in videos from a single camera and investigated the detection accuracy of all HPE-CNNs. To that end, we created a ground-truth dataset from publicly available squash videos by developing our own annotation tool and manually labeling frames and events. We present heatmaps, which depict the court floor using a color scale and highlight areas according to the relative time for which a player occupied that location during matchplay. These are used to provide insight into detections. Finally, we created a decision flow chart to help sport scientists, coaches and athletes to decide which HPE-CNN is best for player detection and tracking in a given application scenario.

Initial evaluation of prospective cardiac triggering using photoplethysmography signals recorded with a video camera compared to pulse oximetry and electrocardiography at 7T MRI.

BACKGROUND: Accurate synchronization between magnetic resonance imaging data acquisition and a subject's cardiac activity ("triggering") is essential for reducing image artifacts but conventional, contact-based methods for this task are limited by several factors, including preparation time, patient inconvenience, and susceptibility to signal degradation. The purpose of this work is to evaluate the performance of a new contact-free triggering method developed with the aim to eventually replace conventional methods in non-cardiac imaging applications. In this study, the method's performance is evaluated in the context of 7 Tesla non-enhanced angiography of the lower extremities. METHODS: Our main contribution is a basic algorithm capable of estimating in real-time the phase of the cardiac cycle from reflection photoplethysmography signals obtained from skin color variations of the forehead recorded with a video camera. Instead of finding the algorithm's parameters heuristically, they were optimized using videos of the forehead as well as electrocardiography and pulse oximetry signals that were recorded from eight healthy volunteers in and outside the scanner, with and without active radio frequency and gradient coils. Based on the video characteristics, synthetic signals were generated and the "best available" values of an objective function were determined using mathematical optimization. The performance of the proposed method with optimized algorithm parameters was evaluated by applying it to the recorded videos and comparing the computed triggers to those of contact-based methods. Additionally, the method was evaluated by using its triggers for acquiring images from a healthy volunteer and comparing the result to images obtained using pulse oximetry triggering. RESULTS: During evaluation of the videos recorded inside the bore with active radio frequency and gradient coils, the pulse oximeter triggers were labeled in 62.5% as "potentially usable" for cardiac triggering, the electrocardiography triggers in 12.5%, and the proposed method's triggers in 62.5%. Evaluation of the angiography images demonstrated that under appropriate conditions the method is feasible to produce an image quality comparable to pulse oximetry. CONCLUSION: We conclude that cardiac triggering using the proposed method is technically feasible. However, for improved reliability the signal-to-noise ratio of the videos will have to be addressed by either replacing the camera sensor, improving the illumination, or by use of additional signal filtering techniques.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review.

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

A Systematic Comparison of Task Adaptation Techniques for Digital Histopathology.

Due to an insufficient amount of image annotation, artificial intelligence in computational histopathology usually relies on fine-tuning pre-trained neural networks. While vanilla fine-tuning has shown to be effective, research on computer vision has recently proposed improved algorithms, promising better accuracy. While initial studies have demonstrated the benefits of these algorithms for medical AI, in particular for radiology, there is no empirical evidence for improved accuracy in histopathology. Therefore, based on the ConvNeXt architecture, our study performs a systematic comparison of nine task adaptation techniques, namely, DELTA, L2-SP, MARS-PGM, Bi-Tuning, BSS, MultiTune, SpotTune, Co-Tuning, and vanilla fine-tuning, on five histopathological classification tasks using eight datasets. The results are based on external testing and statistical validation and reveal a multifaceted picture: some techniques are better suited for histopathology than others, but depending on the classification task, a significant relative improvement in accuracy was observed for five advanced task adaptation techniques over the control method, i.e., vanilla fine-tuning (e.g., Co-Tuning: P(≫) = 0.942, d = 2.623). Furthermore, we studied the classification accuracy for three of the nine methods with respect to the training set size (e.g., Co-Tuning: P(≫) = 0.951, γ = 0.748). Overall, our results show that the performance of advanced task adaptation techniques in histopathology is affected by influencing factors such as the specific classification task or the size of the training dataset.

Delineation of Electrocardiograms Using Multiscale Parameter Estimation.

The continuing interest in unobtrusive electrocardiography requires the development of algorithms, compensating for an increased number of artifacts. In previous work, we proposed a framework for robust parameter estimation of signals following a piecewise Gaussian derivative model, well suited for describing all waves of a heartbeat. The framework is based on a numeric and analytic representation of applying the Wavelet Transform at arbitrary scale to the input model. For robustly estimating model parameters, it processes lines of zero-crossings in scale-space, showing high accuracy for various noise models in synthetic signals. An initial evaluation with electrocardiography signals revealed that our basic classifier for identifying the correct lines often fails, leading to false parameter estimates. In this work, we propose a general delineation method based on a solid mathematical framework that treats each heartbeat, wave and fiducial point in the same way, tailored only by intuitive parameters and not relying on any heuristically found decision rules. The steps include a novel line classifier based on pre-filtering using domain knowledge, followed by an exhaustive search among all possible combinations of zero-crossing lines and an error-measure quantifying their agreement with the model. The combination with highest agreement is processed by the parameter estimation framework, customized to the computation of all nine fiducial points. Evaluation using the expert-annotated QT database, shows high sensitivity (P: 99.91%, QRS: 99.92%, T: 99.89%) and mean errors below 1 ms for all onset and offset fiducial points. The proposed combination of line classification and parameter estimation is well suited for delineating electrocardiograms.

ECG delineation using a piecewise Gaussian derivative model with parameters estimated from scale-dependent algebraic expressions.

Automatic methods for the detection of characteristic points in electrocardiography signals support cardiologists in assessing the state of a patient's cardiovascular system. In this work, we apply a general method for parameter estimation to the specific problem of QRS complex, P-, and T-wave delineation, i.e. the computation of their on- and offset points in time. As input the method expects a piecewise Gaussian derivative model that is potentially a good fit for the morphology of electrocardiography waves, but a thorough investigation is needed. The model parameters are estimated by substituting zero-crossings of the input signals' scale-space representation into scale-dependent algebraic expressions and are further refined by fitting the model to the electrocardiography signal in a least-squares sense. Validating the results on the QT database and comparing to state-of-the-art algorithms shows smallest mean error for 3 out of 9 fiducial points and for the others only small differences to the respective best competitors.

An "optimal" k-needle placement strategy and its application to guiding transbronchial needle aspirations.

This article addresses the problem of finding an "optimal" strategy for placing k biopsy needles, given a large number of possible initial needle positions. We consider two variations of the problem: (1) Calculate the smallest set of needles necessary to guarantee a successful biopsy; and (2) Given a number k, calculate k needles such that the probability of a successful biopsy is maximized. Note that "needle" is used as shorthand for the parameter vector that specifies the needle placement. Both problems are formulated in terms of two general, NP-hard optimization problems. Our k-needle placement strategy can be considered as "optimal" in the sense that we are able to formulate it as a known NP-hard problem for which it is believed (NP not equal P conjecture) that no efficient algorithm exists that computes the optimal solution. In other words, our strategy is "optimal" with respect to the best approximative algorithm known for the respective NP-hard problem. For the second variation we have implemented an approximative algorithm that is guaranteed to be within a factor of approximately 0.63 of the exact solution. Given a number k, the algorithm calculates k sets of parameters, each set specifying the placement of a needle and the corresponding probability of success. The resulting probabilities show that our approach can provide valuable decision support for the physician in choosing how many needles to place and how to place them.A typical example of a biopsy where the initial needle position is known approximately is a transbronchial needle aspiration (TBNA). We demonstrate how our "optimal" needle placement strategy can be used to achieve sensor-less guidance of TBNA. The basic idea is to use a patient-specific model of the tracheobronchial tree (from CT/MR) and our model for flexible endoscopes to preoperatively estimate the unknown position of the bronchoscope. The result is a set of candidate shapes for the unknown shape of the bronchoscope before needle placement or, in other words, a (large) number of possible initial needle positions. By parameterizing the handling of the bronchoscope, including the insertion of the biopsy needle, we are able to apply our "optimal" strategy. The result is a TBNA protocol that, if executed during the procedure, prescribes how to handle the bronchoscope to maneuver the needle into the target. With the aforementioned endoscope model, we present a new way of modeling long, flexible instruments. The algorithm requires no initialization or preprocessing and calculates the workspace of an instrument based on its insertion depth and a set of internal and external constraints.

Rotational roadmapping: a new image-based navigation technique for the interventional room.

For decades, conventional 2D-roadmaping has been the method of choice for image-based guidewire navigation during endovascular procedures. Only recently have 3D-roadmapping techniques become available that are based on the acquisition and reconstruction of a 3D image of the vascular tree. In this paper, we present a new image-based navigation technique called RoRo (Rotational Roadmapping) that eliminates the guess-work inherent to the conventional 2D method, but does not require a 3D image. Our preliminary clinical results show that there are situations in which RoRo is preferred over the existing two methods, thus demonstrating potential for filling a clinical niche and complementing the spectrum of available navigation tools.