Deep Learning-Based Automated Thrombolysis in Cerebral Infarction Scoring: A Timely Proof-of-Principle Study.

Background and Purpose: Mechanical thrombectomy is an established procedure for treatment of acute ischemic stroke. Mechanical thrombectomy success is commonly assessed by the Thrombolysis in Cerebral Infarction (TICI) score, assigned by visual inspection of X-ray digital subtraction angiography data. However, expert-based TICI scoring is highly observer-dependent. This represents a major obstacle for mechanical thrombectomy outcome comparison in, for instance, multicentric clinical studies. Focusing on occlusions of the M1 segment of the middle cerebral artery, the present study aimed to develop a deep learning (DL) solution to automated and, therefore, objective TICI scoring, to evaluate the agreement of DL- and expert-based scoring, and to compare corresponding numbers to published scoring variability of clinical experts. Methods: The study comprises 2 independent datasets. For DL system training and initial evaluation, an in-house dataset of 491 digital subtraction angiography series and modified TICI scores of 236 patients with M1 occlusions was collected. To test the model generalization capability, an independent external dataset with 95 digital subtraction angiography series was analyzed. Characteristics of the DL system were modeling TICI scoring as ordinal regression, explicit consideration of the temporal image information, integration of physiological knowledge, and modeling of inherent TICI scoring uncertainties. Results: For the in-house dataset, the DL system yields Cohen's kappa, overall accuracy, and specific agreement values of 0.61, 71%, and 63% to 84%, respectively, compared with the gold standard: the expert rating. Values slightly drop to 0.52/64%/43% to 87% when the model is, without changes, applied to the external dataset. After model updating, they increase to 0.65/74%/60% to 90%. Literature Cohen's kappa values for expert-based TICI scoring agreement are in the order of 0.6. Conclusions: The agreement of DL- and expert-based modified TICI scores in the range of published interobserver variability of clinical experts highlights the potential of the proposed DL solution to automated TICI scoring.

Benchmarking machine learning-based real-time respiratory signal predictors in 4D SBRT.

BACKGROUND: Stereotactic body radiotherapy of thoracic and abdominal tumors has to account for respiratory intrafractional tumor motion. Commonly, an external breathing signal is continuously acquired that serves as a surrogate of the tumor motion and forms the basis of strategies like breathing-guided imaging and gated dose delivery. However, due to inherent system latencies, there exists a temporal lag between the acquired respiratory signal and the system response. Respiratory signal prediction models aim to compensate for the time delays and to improve imaging and dose delivery. PURPOSE: The present study explores and compares six state-of-the-art machine and deep learning-based prediction models, focusing on real-time and real-world applicability. All models and data are provided as open source and data to ensure reproducibility of the results and foster reuse. METHODS: The study was based on 2502 breathing signals ( t t o t a l ≈ 90 $t_{total} \approx 90$  h) acquired during clinical routine, split into independent training (50%), validation (20%), and test sets (30%). Input signal values were sampled from noisy signals, and the target signal values were selected from corresponding denoised signals. A standard linear prediction model (Linear), two state-of-the-art models in general univariate signal prediction (Dlinear, Xgboost), and three deep learning models (Lstm, Trans-Enc, Trans-TSF) were chosen. The prediction performance was evaluated for three different prediction horizons (480, 680, and 920 ms). Moreover, the robustness of the different models when applied to atypical, that is, out-of-distribution (OOD) signals, was analyzed. RESULTS: The Lstm model achieved the lowest normalized root mean square error for all prediction horizons. The prediction errors only slightly increased for longer horizons. However, a substantial spread of the error values across the test signals was observed. Compared to typical, that is, in-distribution test signals, the prediction accuracy of all models decreased when applied to OOD signals. The more complex deep learning models Lstm and Trans-Enc showed the least performance loss, while the performance of simpler models like Linear dropped the most. Except for Trans-Enc, inference times for the different models allowed for real-time application. CONCLUSION: The application of the Lstm model achieved the lowest prediction errors. Simpler prediction filters suffer from limited signal history access, resulting in a drop in performance for OOD signals.

Self-Supervision for Medical Image Classification: State-of-the-Art Performance with ~100 Labeled Training Samples per Class.

Is self-supervised deep learning (DL) for medical image analysis already a serious alternative to the de facto standard of end-to-end trained supervised DL? We tackle this question for medical image classification, with a particular focus on one of the currently most limiting factor of the field: the (non-)availability of labeled data. Based on three common medical imaging modalities (bone marrow microscopy, gastrointestinal endoscopy, dermoscopy) and publicly available data sets, we analyze the performance of self-supervised DL within the self-distillation with no labels (DINO) framework. After learning an image representation without use of image labels, conventional machine learning classifiers are applied. The classifiers are fit using a systematically varied number of labeled data (1-1000 samples per class). Exploiting the learned image representation, we achieve state-of-the-art classification performance for all three imaging modalities and data sets with only a fraction of between 1% and 10% of the available labeled data and about 100 labeled samples per class.

Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture.

Histopathologic diagnosis relies on simultaneous integration of information from a broad range of scales, ranging from nuclear aberrations (≈O(0.1µm)) through cellular structures (≈O(10µm)) to the global tissue architecture (⪆O(1mm)). To explicitly mimic how human pathologists combine multi-scale information, we introduce a family of multi-encoder fully-convolutional neural networks with deep fusion. We present a simple block for merging model paths with differing spatial scales in a spatial relationship-preserving fashion, which can readily be included in standard encoder-decoder networks. Additionally, a context classification gate block is proposed as an alternative for the incorporation of global context. Our experiments were performed on three publicly available whole-slide images of recent challenges (PAIP 2019: hepatocellular carcinoma segmentation; BACH 2020: breast cancer segmentation; CAMELYON 2016: metastasis detection in lymph nodes). The multi-scale architectures consistently outperformed the baseline single-scale U-Nets by a large margin. They benefit from local as well as global context and particularly a combination of both. If feature maps from different scales are fused, doing so in a manner preserving spatial relationships was found to be beneficial. Deep guidance by a context classification loss appeared to improve model training at low computational costs. All multi-scale models had a reduced GPU memory footprint compared to ensembles of individual U-Nets trained on different image scales. Additional path fusions were shown to be possible at low computational cost, opening up possibilities for further, systematic and task-specific architecture optimisation. The findings demonstrate the potential of the presented family of human-inspired, end-to-end trainable, multi-scale multi-encoder fully-convolutional neural networks to improve deep histopathologic diagnosis by extensive integration of largely different spatial scales.

Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data.

In this paper, we describe our method for the ISIC 2019 Skin Lesion Classification Challenge. The challenge comes with two tasks. For task 1, skin lesions have to be classified based on dermoscopic images. For task 2, dermoscopic images and additional patient meta data are used. Our deep learning-based method achieved first place for both tasks. The are several problems we address with our method. First, there is an unknown class in the test set which we cover with a data-driven approach. Second, there is a severe class imbalance that we address with loss balancing. Third, there are images with different resolutions which motivates two different cropping strategies and multi-crop evaluation. Last, there is patient meta data available which we incorporate with a dense neural network branch. • We address skin lesion classification with an ensemble of deep learning models including EfficientNets, SENet, and ResNeXt WSL, selected by a search strategy. • We rely on multiple model input resolutions and employ two cropping strategies for training. We counter severe class imbalance with a loss balancing approach. • We predict an additional, unknown class with a data-driven approach and we make use of patient meta data with an additional input branch.