Simulation of acquisition shifts in T2 weighted fluid-attenuated inversion recovery magnetic resonance images to stress test artificial intelligence segmentation networks.

Purpose: To provide a simulation framework for routine neuroimaging test data, which allows for "stress testing" of deep segmentation networks against acquisition shifts that commonly occur in clinical practice for T2 weighted (T2w) fluid-attenuated inversion recovery magnetic resonance imaging protocols. Approach: The approach simulates "acquisition shift derivatives" of MR images based on MR signal equations. Experiments comprise the validation of the simulated images by real MR scans and example stress tests on state-of-the-art multiple sclerosis lesion segmentation networks to explore a generic model function to describe the F1 score in dependence of the contrast-affecting sequence parameters echo time (TE) and inversion time (TI). Results: The differences between real and simulated images range up to 19% in gray and white matter for extreme parameter settings. For the segmentation networks under test, the F1 score dependency on TE and TI can be well described by quadratic model functions (R2>0.9). The coefficients of the model functions indicate that changes of TE have more influence on the model performance than TI. Conclusions: We show that these deviations are in the range of values as may be caused by erroneous or individual differences in relaxation times as described by literature. The coefficients of the F1 model function allow for a quantitative comparison of the influences of TE and TI. Limitations arise mainly from tissues with a low baseline signal (like cerebrospinal fluid) and when the protocol contains contrast-affecting measures that cannot be modeled due to missing information in the DICOM header.

Faster and Better: How Anomaly Detection Can Accelerate and Improve Reporting of Head Computed Tomography.

BACKGROUND: Most artificial intelligence (AI) systems are restricted to solving a pre-defined task, thus limiting their generalizability to unselected datasets. Anomaly detection relieves this shortfall by flagging all pathologies as deviations from a learned norm. Here, we investigate whether diagnostic accuracy and reporting times can be improved by an anomaly detection tool for head computed tomography (CT), tailored to provide patient-level triage and voxel-based highlighting of pathologies. METHODS: Four neuroradiologists with 1-10 years of experience each investigated a set of 80 routinely acquired head CTs containing 40 normal scans and 40 scans with common pathologies. In a random order, scans were investigated with and without AI-predictions. A 4-week wash-out period between runs was included to prevent a reminiscence effect. Performance metrics for identifying pathologies, reporting times, and subjectively assessed diagnostic confidence were determined for both runs. RESULTS: AI-support significantly increased the share of correctly classified scans (normal/pathological) from 309/320 scans to 317/320 scans (p = 0.0045), with a corresponding sensitivity, specificity, negative- and positive- predictive value of 100%, 98.1%, 98.2% and 100%, respectively. Further, reporting was significantly accelerated with AI-support, as evidenced by the 15.7% reduction in reporting times (65.1 ± 8.9 s vs. 54.9 ± 7.1 s; p < 0.0001). Diagnostic confidence was similar in both runs. CONCLUSION: Our study shows that AI-based triage of CTs can improve the diagnostic accuracy and accelerate reporting for experienced and inexperienced radiologists alike. Through ad hoc identification of normal CTs, anomaly detection promises to guide clinicians towards scans requiring urgent attention.

Automated Detection of Ischemic Stroke and Subsequent Patient Triage in Routinely Acquired Head CT.

PURPOSE: Advanced machine-learning (ML) techniques can potentially detect the entire spectrum of pathology through deviations from a learned norm. We investigated the utility of a weakly supervised ML tool to detect characteristic findings related to ischemic stroke in head CT and provide subsequent patient triage. METHODS: Patients having undergone non-enhanced head CT at a tertiary care hospital in April 2020 with either no anomalies, subacute or chronic ischemia, lacunar infarcts of the deep white matter or hyperdense vessel signs were retrospectively analyzed. Anomaly detection was performed using a weakly supervised ML classifier. Findings were displayed on a voxel-level (heatmap) and pooled to an anomaly score. Thresholds for this score classified patients into i) normal, ii) inconclusive, iii) pathological. Expert-validated radiological reports were considered as ground truth. Test assessment was performed with ROC analysis; inconclusive results were pooled to pathological predictions for accuracy measurements. RESULTS: During the investigation period 208 patients were referred for head CT of which 111 could be included. Definite ratings into normal/pathological were feasible in 77 (69.4%) patients. Based on anomaly scores, the AUC to differentiate normal from pathological scans was 0.98 (95% CI 0.97-1.00). The sensitivity, specificity, positive and negative predictive values were 100%, 40.6%, 80.6% and 100%, respectively. CONCLUSION: Our study demonstrates the potential of a weakly supervised anomaly-detection tool to detect stroke findings in head CT. Definite classification into normal/pathological was made with high accuracy in > 2/3 of patients. Anomaly heatmaps further provide guidance towards pathologies, also in cases with inconclusive ratings.

Automated Pathology Detection and Patient Triage in Routinely Acquired Head Computed Tomography Scans.

OBJECTIVES: Anomaly detection systems can potentially uncover the entire spectrum of pathologies through deviations from a learned norm, meaningfully supporting the radiologist's workflow. We aim to report on the utility of a weakly supervised machine learning (ML) tool to detect pathologies in head computed tomography (CT) and adequately triage patients in an unselected patient cohort. MATERIALS AND METHODS: All patients having undergone a head CT at a tertiary care hospital in March 2020 were eligible for retrospective analysis. Only the first scan of each patient was included. Anomaly detection was performed using a weakly supervised ML technique. Anomalous findings were displayed on voxel-level and pooled to an anomaly score ranging from 0 to 1. Thresholds for this score classified patients into the 3 classes: "normal," "pathological," or "inconclusive." Expert-validated radiological reports with multiclass pathology labels were considered as ground truth. Test assessment was performed with receiver operator characteristics analysis; inconclusive results were pooled to "pathological" predictions for accuracy measurements. External validity was tested in a publicly available external data set (CQ500). RESULTS: During the investigation period, 297 patients were referred for head CT of which 248 could be included. Definite ratings into normal/pathological were feasible in 167 patients (67.3%); 81 scans (32.7%) remained inconclusive. The area under the curve to differentiate normal from pathological scans was 0.95 (95% confidence interval, 0.92-0.98) for the study data set and 0.87 (95% confidence interval, 0.81-0.94) in external validation. The negative predictive value to exclude pathology if a scan was classified as "normal" was 100% (25/25), and the positive predictive value was 97.6% (137/141). Sensitivity and specificity were 100% and 86%, respectively. In patients with inconclusive ratings, pathologies were found in 26 (63%) of 41 cases. CONCLUSIONS: Our study provides the first clinical evaluation of a weakly supervised anomaly detection system for brain imaging. In an unselected, consecutive patient cohort, definite classification into normal/diseased was feasible in approximately two thirds of scans, going along with an excellent diagnostic accuracy and perfect negative predictive value for excluding pathology. Moreover, anomaly heat maps provide important guidance toward pathology interpretation, also in cases with inconclusive ratings.

Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer.

Tissue Phenomics is the discipline of mining tissue images to identify patterns that are related to clinical outcome providing potential prognostic and predictive value. This involves the discovery process from assay development, image analysis, and data mining to the final interpretation and validation of the findings. Importantly, this process is not linear but allows backward steps and optimization loops over multiple sub-processes. We provide a detailed description of the Tissue Phenomics methodology while exemplifying each step on the application of prostate cancer recurrence prediction. In particular, we automatically identified tissue-based biomarkers having significant prognostic value for low- and intermediate-risk prostate cancer patients (Gleason scores 6-7b) after radical prostatectomy. We found that promising phenes were related to CD8(+) and CD68(+) cells in the microenvironment of cancerous glands in combination with the local micro-vascularization. Recurrence prediction based on the selected phenes yielded accuracies up to 83% thereby clearly outperforming prediction based on the Gleason score. Moreover, we compared different machine learning algorithms to combine the most relevant phenes resulting in increased accuracies of 88% for tumor progression prediction. These findings will be of potential use for future prognostic tests for prostate cancer patients and provide a proof-of-principle of the Tissue Phenomics approach.

Deformable reconstruction of histology sections using structural probability maps.

The reconstruction of a 3D volume from a stack of 2D histology slices is still a challenging problem especially if no external references are available. Without a reference, standard registration approaches tend to align structures that should not be perfectly aligned. In this work we introduce a deformable, reference-free reconstruction method that uses an internal structural probability map (SPM) to regularize a free-form deformation. The SPM gives an estimate of the original 3D structure of the sample from the misaligned and possibly corrupted 2D slices. We present a consecutive as well as a simultaneous reconstruction approach that incorporates this estimate in a deformable registration framework. Experiments on synthetic and mouse brain datasets indicate that our method produces similar results compared to reference-based techniques on synthetic datasets. Moreover, it improves the smoothness of the reconstruction compared to standard registration techniques on real data.

Hough space parametrization: ensuring global consistency in intensity-based registration.

Intensity based registration is a challenge when images to be registered have insufficient amount of information in their overlapping region. Especially, in the absence of dominant structures such as strong edges in this region, obtaining a solution that satisfies global structural consistency becomes difficult. In this work, we propose to exploit the vast amount of available information beyond the overlapping region to support the registration process. To this end, a novel global regularization term using Generalized Hough Transform is designed that ensures the global consistency when the local information in the overlap region is insufficient to drive the registration. Using prior data, we learn a parametrization of the target anatomy in Hough space. This parametrization is then used as a regularization for registering the observed partial images without using any prior data. Experiments on synthetic as well as on sample real medical images demonstrate the good performance and potential use of the proposed concept.

Structure propagation for image registration.

Mosaicing is a commonly used technique in many medical imaging applications where subimages are stitched together in order to obtain a larger field of view. However, stitching, which involves alignment or registration in overlapping regions, is often challenging when the information shared by subimages is absent or small. While it is not possible to perform an alignment without overlap using existing techniques, imaging artifacts such as distortions towards image boundaries present further complications during registration by decreasing the reliability of available information. Without taking these into consideration, a registration approach might violate the continuity and the smoothness of structures across subimages. In this paper, we propose a novel registration approach for the stitching of subimages in such challenging scenarios. By using a perceptual grouping approach, we extend subimages beyond their boundaries by propagating available structures in order to obtain structural maps in the extended regions. These maps are then used to establish correspondences between subimages when the shared information is absent, small or unreliable. Using our approach ensures the continuity and the smoothness of structures across subimage boundaries. Furthermore, since only structures are used, the proposed method can also be used for the stitching of multi-modal images. Our approach is unique in that it also enables contactless stitching. We demonstrate the effectiveness of the proposed method by performing several experiments on synthetic and medical images. Moreover, we show how stitching is possible in the presence of a physical gap between subimages.

Manifold learning for image-based breathing gating in ultrasound and MRI.

Respiratory motion is a challenging factor for image acquisition and image-guided procedures in the abdominal and thoracic region. In order to address the issues arising from respiratory motion, it is often necessary to detect the respiratory signal. In this article, we propose a novel, purely image-based retrospective respiratory gating method for ultrasound and MRI. Further, we apply this technique to acquire breathing-affected 4D ultrasound with a wobbler probe and, similarly, to create 4D MR with a slice stacking approach. We achieve the gating with Laplacian eigenmaps, a manifold learning technique, to determine the low-dimensional manifold embedded in the high-dimensional image space. Since Laplacian eigenmaps assign to each image frame a coordinate in low-dimensional space by respecting the neighborhood relationship, they are well suited for analyzing the breathing cycle. We perform the image-based gating on several 2D and 3D ultrasound datasets over time, and quantify its very good performance by comparing it to measurements from an external gating system. For MRI, we perform the manifold learning on several datasets for various orientations and positions. We achieve very high correlations by a comparison to an alternative gating with diaphragm tracking.

Temporal groupwise registration for motion modeling.

We propose a novel method for the registration of time-resolved image sequences, called Spatio-Temporal groupwise non-rigid Registration using free-form deforMations (STORM). It is a groupwise registration method, with a group of images being considered simultaneously, in order to prevent bias introduction. This is different from pairwise registration methods where only two images are registered to each other. Furthermore, STORM is a spatio-temporal registration method, where both, the spatial and the temporal information are utilized during the registration. This ensures the smoothness and consistency of the resulting deformation fields, which is especially important for motion modeling on medical data. Moreover, popular free-form deformations are applied to model the non-rigid motion. Experiments are conducted on both synthetic and medical images. Results show the good performance and the robustness of the proposed approach with respect to outliers and imaging artifacts, and moreover, its ability to correct for larger deformation in comparison to standard pairwise techniques.

Manifold learning for shape guided segmentation of cardiac boundaries: application to 3D+t cardiac MRI.

In this paper we propose a new method for shape guided segmentation of cardiac boundaries based on manifold learning of the shapes represented by the phase field approximation of the Mumford-Shah functional. A novel distance is defined to measure the similarity of shapes without requiring deformable registration. Cardiac motion is compensated and phases are mapped into one reference phase, that is the end of diastole, to avoid time warping and synchronization at all cardiac phases. Non-linear embedding of these 3D shapes extracts the manifold of the inter-subject variation of the heart shape to be used for guiding the segmentation for a new subject. For validation the method is applied to a comprehensive dataset of 3D+t cardiac Cine MRI from normal subjects and patients.

Manifold learning for image-based breathing gating with application to 4D ultrasound.

Breathing motion leads to a significant displacement and deformation of organs in the abdominal region. This makes the detection of the breathing phase for numerous applications necessary. We propose a new, purely image-based respiratory gating method for ultrasound. Further, we use this technique to provide a solution for breathing affected 4D ultrasound acquisitions with a wobbler probe. We achieve the gating with Laplacian eigenmaps, a manifold learning technique, to determine the low-dimensional manifold embedded in the high-dimensional image space. Since Laplacian eigenmaps assign each ultrasound frame a coordinate in low-dimensional space by respecting the neighborhood relationship, they are well suited for analyzing the breathing cycle. For the 4D application, we perform the manifold learning for each angle, and consecutively, align all the local curves and perform a curve fitting to achieve a globally consistent breathing signal. We performed the image-based gating on several 2D and 3D ultrasound datasets over time, and quantified its very good performance by comparing it to measurements from an external gating system.