Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation.

PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review.

Background/purpose: The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods: We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results: We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion: Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation.

BACKGROUND: Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS: Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS: After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

A simple single-cycle interactive strategy to improve deep learning-based segmentation of organs-at-risk in head-and-neck cancer.

Background and purpose: Interactive segmentation seeks to incorporate human knowledge into segmentation models and thereby reducing the total amount of editing of auto-segmentations. By performing only interactions which provide new information, segmentation performance may increase cost-effectively. The aim of this study was to develop, evaluate and test feasibility of a deep learning-based single-cycle interactive segmentation model with the input being computer tomography (CT) and a small amount of information rich contours. Methods and Materials: A single-cycle interactive segmentation model, which took CT and the most cranial and caudal contour slices for each of 16 organs-at-risk for head-and-neck cancer as input, was developed. A CT-only model served as control. The models were evaluated with Dice similarity coefficient, Hausdorff Distance 95th percentile and average symmetric surface distance. A subset of 8 organs-at-risk were selected for a feasibility test. In this, a designated radiation oncologist used both single-cycle interactive segmentation and atlas-based auto-contouring for three cases. Contouring time and added path length were recorded. Results: The medians of Dice coefficients increased with single-cycle interactive segmentation in the range of 0.004 (Brain)-0.90 (EyeBack_merged) when compared to CT-only. In the feasibility test, contouring time and added path length were reduced for all three cases as compared to editing atlas-based auto-segmentations. Conclusion: Single-cycle interactive segmentation improved segmentation metrics when compared to the CT-only model and was clinically feasible from a technical and usability point of view. The study suggests that it may be cost-effective to add a small amount of contouring input to deep learning-based segmentation models.

Time until diagnosis of clinical events with different remote monitoring systems in implantable cardioverter-defibrillator patients.

BACKGROUND: Remote monitoring (RM) is an established technology integrated into routine follow-up of patients with an implantable cardioverter-defibrillator (ICD). Current RM systems differ according to transmission frequency and alert definition. OBJECTIVE: The purpose of this study was to compare the time difference between detection and acknowledgment of clinically relevant events between 4 RM systems. METHODS: We analyzed time delay between detection of ventricular arrhythmic and technical events by the ICD and acknowledgment by hospital staff in 1802 consecutive patients followed with RM between September 2014 and August 2016. Devices from Biotronik (BIO; n = 374), Boston Scientific (BSC; n = 196), Medtronic (MDT; n = 468), and St Jude Medical (SJM; n = 764) were included. We identified all events from RM web pages and their acknowledgment with RM or at in-clinic follow-up. Events that occurred during weekends were excluded. RESULTS: We included 3472 events. Proportion of events acknowledged within 24 hours was 72%, 23%, 18%, and 65% with BIO, BSC, MDT, and SJM, respectively, with median times of 13, 222, 163, and 18 hours from detection to acknowledgment (P <.001 for both comparisons between manufacturers). Including only events transmitted as alerts by RM, 72%, 68%, 61%, and 65% for BIO, BSC, MDT and SJM, respectively, were acknowledged within 24 hours. Variation in time to acknowledgment of ventricular tachyarrhythmia episodes not treated with shock therapy was the primary cause for the difference between manufacturers. CONCLUSION: Significant and clinically relevant differences in time delay from event detection to acknowledgment exist between RM systems. Varying definitions of which events RM transmits as alerts are important for the differences observed.

Remote Ischemic Conditioning for Patients With STEMI.

ST-segment elevation myocardial infarction (STEMI) remains a leading cause of death and morbidity, despite declining incidence and improved short-term outcome in many countries. Although mortality declines in developed countries with easy and fast access to optimized treatment, development of heart failure often remains a challenge in survivors and still approaches 10% at 1 year. Rapid admission and acute interventional treatment combined with modern antithrombotic pharmacologic therapy frequently establish complete reperfusion and acutely stabilize the patient, but the reperfusion itself adds further to the damage in the myocardium compromising the long-term outcome. Reperfusion injury is believed to be a significant-if not the dominant-contributor to the net injury resulting from STEMI and has become a major focus of research in recent years. Despite a plethora of pharmacological and mechanical interventions showing consistent reduction of reperfusion injury in experimental models, translation into a clinical setting has been challenging. In patients, attempts to modify reperfusion injury by pharmacological strategies have largely been unsuccessful, and focus is increasingly directed toward mechanical modalities. Remote ischemic conditioning of the heart is achieved by repeated brief interruption of the blood supply to a distant part of the body, most frequently the arm. At present, remote ischemic conditioning is the most promising adjuvant therapy to reduce reperfusion injury in patients with STEMI. In this review, we discuss the results of clinical trials investigating the effect of remote ischemic conditioning in patients admitted with STEMI and potential reasons for its apparent superiority to current pharmacologic adjuvant therapies.