Development of a national deep learning-based auto-segmentation model for the heart on clinical delineations from the DBCG RT nation cohort.

BACKGROUND: This study aimed at investigating the feasibility of developing a deep learning-based auto-segmentation model for the heart trained on clinical delineations. MATERIAL AND METHODS: This study included two different datasets. The first dataset contained clinical heart delineations from the DBCG RT Nation study (1,561 patients). The second dataset was smaller (114 patients), but with corrected heart delineations. Before training the model on the clinical delineations an outlier-detection was performed, to remove cases with gross deviations from the delineation guideline. No outlier detection was performed for the dataset with corrected heart delineations. Both models were trained with a 3D full resolution nnUNet. The models were evaluated with the dice similarity coefficient (DSC), 95% Hausdorff distance (HD95) and Mean Surface Distance (MSD). The difference between the models were tested with the Mann-Whitney U-test. The balance of dataset quantity versus quality was investigated, by stepwise reducing the cohort size for the model trained on clinical delineations. RESULTS: During the outlier-detection 137 patients were excluded from the clinical cohort due to non-compliance with delineation guidelines. The model trained on the curated clinical cohort performed with a median DSC of 0.96 (IQR 0.94-0.96), median HD95 of 4.00 mm (IQR 3.00 mm-6.00 mm) and a median MSD of 1.49 mm (IQR 1.12 mm-2.02 mm). The model trained on the dedicated and corrected cohort performed with a median DSC of 0.95 (IQR 0.93-0.96), median HD95 of 5.65 mm (IQR 3.37 mm-8.62 mm) and median MSD of 1.63 mm (IQR 1.35 mm-2.11 mm). The difference between the two models were found non-significant for all metrics (p > 0.05). Reduction of cohort size showed no significant difference for all metrics (p > 0.05). However, with the smallest cohort size, a few outlier structures were found. CONCLUSIONS: This study demonstrated a deep learning-based auto-segmentation model trained on curated clinical delineations which performs on par with a model trained on dedicated delineations, making it easier to develop multi-institutional auto-segmentation models.

Target coverage and local recurrences after radiotherapy for sinonasal cancer in Denmark 2008-2015. A DAHANCA study.

PURPOSE: The study aimed to investigate the pattern of failure and describe compromises in the definition and coverage of the target for patients treated with curatively intended radiotherapy (RT) for sinonasal cancer (SNC). METHODS AND MATERIAL: Patients treated with curatively intended RT in 2008-2015 in Denmark for SNC were eligible for the retrospective cohort study. Information regarding diagnosis and treatment was retrieved from the national database of the Danish Head and Neck Cancer Group (DAHANCA). Imaging from the diagnosis of recurrences was collected, and the point of origin (PO) of the recurrent tumour was estimated. All treatment plans were collected and reviewed with the focus on target coverage, manual modifications of target volumes, and dose to organs at risk (OARs) above defined constraints. RESULTS: A total of 184 patients were included in the analysis, and 76 (41%) relapsed. The majority of recurrences involved T-site (76%). Recurrence imaging of 39 patients was evaluated, and PO was established. Twenty-nine POs (74%) were located within the CTV, and the minimum dose to the PO was median 64.1 Gy (3.1-70.7). The criteria for target coverage (V95%) was not met in 89/184 (48%) of the CTV and 131/184 (71%) of the PTV. A total of 24% of CTVs had been manually modified to spare OARs of high-dose irradiation. No difference in target volume modifications was observed between patients who suffered recurrence and patients with lasting remission. CONCLUSION: The majority of relapses after radical treatment of SNC were located in the T-site (the primary tumour site). Multiple compromises with regards to target coverage and tolerance levels for OARs in the sinonasal region, as defined from RT guidelines, were taken. No common practice in this respect could be derived from the study.

Comparing different CT, PET and MRI multi-modality image combinations for deep learning-based head and neck tumor segmentation.

BACKGROUND: Manual delineation of gross tumor volume (GTV) is essential for radiotherapy treatment planning, but it is time-consuming and suffers inter-observer variability (IOV). In clinics, CT, PET, and MRI are used to inform delineation accuracy due to their different complementary characteristics. This study aimed to investigate deep learning to assist GTV delineation in head and neck squamous cell carcinoma (HNSCC) by comparing various modality combinations. MATERIALS AND METHODS: This retrospective study had 153 patients with multiple sites of HNSCC including their planning CT, PET, and MRI (T1-weighted and T2-weighted). Clinical delineations of gross tumor volume (GTV-T) and involved lymph nodes (GTV-N) were collected as the ground truth. The dataset was randomly divided into 92 patients for training, 31 for validation, and 30 for testing. We applied a residual 3 D UNet as the deep learning architecture. We independently trained the UNet with four different modality combinations (CT-PET-MRI, CT-MRI, CT-PET, and PET-MRI). Additionally, analogical to post-processing, an average fusion of three bi-modality combinations (CT-PET, CT-MRI, and PET-MRI) was produced as an ensemble. Segmentation accuracy was evaluated on the test set, using Dice similarity coefficient (Dice), Hausdorff Distance 95 percentile (HD95), and Mean Surface Distance (MSD). RESULTS: All imaging combinations including PET provided similar average scores in range of Dice: 0.72-0.74, HD95: 8.8-9.5 mm, MSD: 2.6-2.8 mm. Only CT-MRI had a lower score with Dice: 0.58, HD95: 12.9 mm, MSD: 3.7 mm. The average of three bi-modality combinations reached Dice: 0.74, HD95: 7.9 mm, MSD: 2.4 mm. CONCLUSION: Multimodal deep learning-based auto segmentation of HNSCC GTV was demonstrated and inclusion of the PET image was shown to be crucial. Training on combined MRI, PET, and CT data provided limited improvements over CT-PET and PET-MRI. However, when combining three bimodal trained networks into an ensemble, promising improvements were shown.

Enhancing the reliability of deep learning-based head and neck tumour segmentation using uncertainty estimation with multi-modal images.

Objective.Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors.Approach.We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC).Main results.Evaluated on the hold-out test dataset (n= 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network-PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC.Significance.Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.

RadDeploy: A framework for integrating in-house developed software and artificial intelligence models seamlessly into radiotherapy workflows.

The use of and research in automation and artificial intelligence (AI) in radiotherapy is moving with incredible pace. Many innovations do, however, not make it into the clinic. One technical reason for this may be the lack of a platform to deploy such software into clinical practice. We suggest RadDeploy as a framework for integrating containerized software in clinical workflows outside of treatment planning systems. RadDeploy supports multiple DICOM as input for model containers and can run model containers asynchronously across GPUs and computers. This technical note summarizes the inner workings of RadDeploy and demonstrates three use-cases with varying complexity.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review.

Background/purpose: The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods: We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results: We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion: Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Evaluating Danish Breast Cancer Group locoregional radiotherapy guideline adherence in clinical treatment data 2008-2016: The DBCG RT Nation study.

BACKGROUND AND PURPOSE: Guideline adherence in radiotherapy is crucial for maintaining treatment quality and consistency, particularly in non-trial patient settings where most treatments occur. The study aimed to assess the impact of guideline changes on treatment planning practices and compare manual registry data accuracy with treatment planning data. MATERIALS AND METHODS: This study utilised the DBCG RT Nation cohort, a collection of breast cancer radiotherapy data in Denmark, to evaluate adherence to guidelines from 2008 to 2016. The cohort included 7448 high-risk breast cancer patients. National guideline changes included, fractionation, introduction of respiratory gating, irradiation of the internal mammary lymph nodes, use of the simultaneous integrated boost technique and inclusion of the Left Anterior Descending coronary artery in delineation practice. Methods for structure name mapping, laterality detection, detection of temporal changes in population mean lung volume, and dose evaluation were presented and applied. Manually registered treatment characteristic data was obtained from the Danish Breast Cancer Database for comparison. RESULTS: The study found immediate and consistent adherence to guideline changes across Danish radiotherapy centres. Treatment practices before guideline implementation were documented and showed a variation among centres. Discrepancies between manual registry data and actual treatment planning data were as high as 10% for some measures. CONCLUSION: National guideline changes could be detected in the routine treatment data, with a high degree of compliance and short implementation time. Data extracted from treatment planning data files provides a more accurate and detailed characterisation of treatments and guideline adherence than medical register data.

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy.

BACKGROUND AND PURPOSE: Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap. METHODS AND MATERIALS: A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended. RESULTS: The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated. CONCLUSION: A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation.

PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Evaluation of methods for selecting the midventilation bin in 4DCT scans of lung cancer patients.

BACKGROUND: In lung cancer radiotherapy, planning on the midventilation (MidV) bin of a four-dimensional (4D) CT scan can reduce the systematic errors introduced by respiratory tumour motion compared to conventional CT. In this study four different methods for MidV bin selection are evaluated. MATERIAL AND METHODS: The study is based on 4DCT scans of 19 patients with a total of 23 peripheral lung tumours having peak-to-peak displacement ≥ 5 mm in at least one of the left-right (LR), anterior-posterior (AP) or cranio-caudal (CC) directions. For each tumour, the MidV bin was selected based on: 1) visual evaluation of tumour displacement; 2) rigid registration of tumour position; 3) diaphragm displacement in the CC direction; and 4) carina displacement in the CC direction. Determination of the MidV bin based on the displacement of the manually delineated gross tumour volume (GTV) was used as a reference method. The accuracy of each method was evaluated by the distance between GTV position in the selected MidV bin and the time-weighted mean position of GTV throughout the bins (i.e. the geometric MidV error). RESULTS: Median (range) geometric MidV error was 1.4 (0.4-5.4) mm, 1.4 (0.4-5.4) mm, 1.9 (0.5-6.9) mm, 2.0 (0.5-12.3) mm and 1.1 (0.4-5.4) mm for the visual, rigid registration, diaphragm, carina, and reference method. Median (range) absolute difference between geometric MidV error for the evaluated methods and the reference method was 0.0 (0.0-1.2) mm, 0.0 (0.0-1.7) mm, 0.7 (0.0-3.9) mm and 1.0 (0.0-6.9) mm for the visual, rigid registration, diaphragm and carina method. CONCLUSION: The visual and semi-automatic rigid registration methods were equivalent in accuracy for selecting the MidV bin of a 4DCT scan. The methods based on diaphragm and carina displacement cannot be recommended without modifications.

Stability of percutaneously implanted markers for lung stereotactic radiotherapy.

The purpose of this study was to evaluate the stability of complex markers implanted into lung tumors throughout a course of stereotactic body radiotherapy (SBRT). Fifteen patients referred for lung SBRT were prospectively included. Radio-opaque markers were implanted percutaneously, guided by computed tomography (CT). Deep inspiration breath-hold CT scans (BHCT) were acquired at planning and on three treatment days. The treatment days' BHCTs were registered to the planning BHCT. Intraobserver uncertainty in both tumor and marker registration was determined. Deviations in the difference between tumor and marker-based image registrations of the BHCT scans during treatment quantified the marker stability. Marker position deviation relative to tumor position of less than 2 mm in all three dimensions was considered acceptable for treatment delivery precision. Intra observer uncertainties for image registration in the left-right (LR), anterior-posterior (AP), craniocaudal (CC) directions and three-dimensional vector (3D) were 0.9 mm, 0.9 mm, 1.0 mm, and 1.1 mm (SD) for tumor registration and 0.3 mm, 0.5 mm, 0.7 mm, and 0.7 mm (SD) for marker registration. Mean 3D differences for tumor registrations on all days were significantly larger than for 3D marker registrations (p = 0.007). Overall median differences between tumor and marker position were 0.0 mm (range -2.9 to 2.6 mm) in LR, 0.0 mm (-1.8 to 1.5 mm) in AP, and -0.2 mm (-2.6 to 2.8 mm) in CC directions. Four patients had deviations exceeding 2 mm in one or more registrations throughout the SBRT course. This is the first study to evaluate stability of complex markers implanted percutaneously into lung tumors for image guidance in SBRT. We conclude that the observed stability of marker position within the tumor indicates that complex markers can be used as surrogates for tumor position during a short course of SBRT as long as the uncertainties related to their position within the tumor are incorporated into the planning target volume.

Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy.

Background and purpose: With deep-learning, gross tumour volume (GTV) auto-segmentation has substantially been improved, but still substantial manual corrections are needed. With interactive deep-learning (iDL), manual corrections can be used to update a deep-learning tool while delineating, minimising the input to achieve acceptable segmentations. We present an iDL tool for GTV segmentation that took annotated slices as input and simulated its performance on a head and neck cancer (HNC) dataset. Materials and methods: Multimodal image data of 204 HNC patients with clinical tumour and lymph node GTV delineations were used. A baseline convolutional neural network (CNN) was trained (n = 107 training, n = 22 validation) and tested (n = 24). Subsequently, user input was simulated on initial test set by replacing one or more of predicted slices with ground truth delineation, followed by re-training the CNN. The objective was to optimise re-training parameters and simulate slice selection scenarios while limiting annotations to maximally-five slices. The remaining 51 patients were used as an independent test set, where Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD95%) were assessed at baseline and after every update. Results: Median segmentation accuracy at baseline was DSC = 0.65, MSD = 4.3 mm, HD95% = 17.5 mm. Updating CNN using three slices equally sampled from the craniocaudal axis of the GTV in the first round, followed by two rounds of annotating one extra slice, gave the best results. The accuracy improved to DSC = 0.82, MSD = 1.6 mm, HD95% = 4.8 mm. Every CNN update took 30 s. Conclusions: The presented iDL tool achieved substantial segmentation improvement with only five annotated slices.

A simple single-cycle interactive strategy to improve deep learning-based segmentation of organs-at-risk in head-and-neck cancer.

Background and purpose: Interactive segmentation seeks to incorporate human knowledge into segmentation models and thereby reducing the total amount of editing of auto-segmentations. By performing only interactions which provide new information, segmentation performance may increase cost-effectively. The aim of this study was to develop, evaluate and test feasibility of a deep learning-based single-cycle interactive segmentation model with the input being computer tomography (CT) and a small amount of information rich contours. Methods and Materials: A single-cycle interactive segmentation model, which took CT and the most cranial and caudal contour slices for each of 16 organs-at-risk for head-and-neck cancer as input, was developed. A CT-only model served as control. The models were evaluated with Dice similarity coefficient, Hausdorff Distance 95th percentile and average symmetric surface distance. A subset of 8 organs-at-risk were selected for a feasibility test. In this, a designated radiation oncologist used both single-cycle interactive segmentation and atlas-based auto-contouring for three cases. Contouring time and added path length were recorded. Results: The medians of Dice coefficients increased with single-cycle interactive segmentation in the range of 0.004 (Brain)-0.90 (EyeBack_merged) when compared to CT-only. In the feasibility test, contouring time and added path length were reduced for all three cases as compared to editing atlas-based auto-segmentations. Conclusion: Single-cycle interactive segmentation improved segmentation metrics when compared to the CT-only model and was clinically feasible from a technical and usability point of view. The study suggests that it may be cost-effective to add a small amount of contouring input to deep learning-based segmentation models.

Patient anatomy-specific trade-offs between sub-clinical disease coverage and normal tissue dose reduction in head-and-neck cancer.

PURPOSE: Risk of subclinical disease decreases with increasing distance from the GTV in head- and-neck squamous cell carcinoma (HNSCC). Depending on individual patient anatomy, OAR sparing could be improved by reducing target coverage in regions with low risk of subclinical spread. Using automated multi-criteria optimization, we investigate patient-specific optimal trade-offs between target periphery coverage and OAR sparing. METHODS: VMAT plans for 39 HNSCC patients were retrospectively created following our clinical three-target-level protocol: high-risk (PTV1), intermediate-risk (PTV2, 5 mm expansion from PTV1), and elective (PTV3). A baseline plan fulfilling clinical constraints (D 99 % ≥95 % for all PTVs) was compared to three plans with reduced PTV2 coverage (goals: PTV2 D 99 % ≥90 % or 85 %, or no PTV2) at the outer edge of PTV2. Plans were compared on PTV D 99 %, OAR D mean, and NTCP (xerostomia/dysphagia). RESULTS: Trade-offs between PTV2 coverage and OAR doses varied considerably between patients. For plans with PTV2 D 99 % -goal 90 %, median PTV2 D 99 % was 91.5 % resulting in xerostomia (≥grade 4) and dysphagia (≥grade 2) NTCP decrease of median [maximum] 1.9 % [5.3 %] and 1.1 % [4.1 %], respectively, compared to nominal PTV2 D 99 % -goal 95 %. For PTV2 D 99 % -goal 85 % median PTV D 99 % was 87 % with NTCP improvements of 4.6 % [9.9 %] and 1.5 % [5.4 %]. For no-margin plans, PTV2 D 99 % decreased to 83.3 % with NTCP reductions of 5.1 % [10.2 %] and 1.4 % [6.1 %]. CONCLUSION: Clinically relevant, patient-specific reductions in OARs and NTCP were observed at limited cost in target under-coverage at the outermost PTV edge. Given the observed inter-patient variations, individual evaluation is warranted to determine whether trade- offs would benefit a specific patient.

End-to-end framework for automated collection of large multicentre radiotherapy datasets demonstrated in a Danish Breast Cancer Group cohort.

Large Digital Imaging and Communications in Medicine (DICOM) datasets are key to support research and the development of machine learning technology in radiotherapy (RT). However, the tools for multi-centre data collection, curation and standardisation are not readily available. Automated batch DICOM export solutions were demonstrated for a multicentre setup. A Python solution, Collaborative DICOM analysis for RT (CORDIAL-RT) was developed for curation, standardisation, and analysis of the collected data. The setup was demonstrated in the DBCG RT-Nation study, where 86% (n = 7748) of treatments in the inclusion period were collected and quality assured, supporting the applicability of the end-to-end framework.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation.

BACKGROUND: Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS: Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS: After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.