Framework for Radiation Oncology Department-wide Evaluation and Implementation of Commercial Artificial Intelligence Autocontouring.

PURPOSE: Artificial intelligence (AI)-based autocontouring in radiation oncology has potential benefits such as standardization and time savings. However, commercial AI solutions require careful evaluation before clinical integration. We developed a multidimensional evaluation method to test pretrained AI-based automated contouring solutions across a network of clinics. METHODS AND MATERIALS: Curated data included 121 patient planning computed tomography (CT) scans with a total of 859 clinically approved contours used for treatment from 4 clinics. Regions of interest (ROIs) were generated with 3 commercial AI-based automated contouring software solutions (AI1, AI2, AI3) spanning the following disease sites: brain, head and neck (H&N), thorax, abdomen, and pelvis. Quantitative agreement between AI-generated and clinical contours was measured by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Qualitative assessment was performed by multiple experts scoring blinded AI-contours using a Likert scale. Workflow and usability surveying was also conducted. RESULTS: AI1, AI2, and AI3 contours had high quantitative agreement in 27.8%, 32.8%, and 34.1% of cases (DSC >0.9), performing well in pelvis (median DSC = 0.86/0.88/0.91) and thorax (median DSC = 0.91/0.89/0.91). All 3 solutions had low quantitative agreement in 7.4%, 8.8%, and 6.1% of cases (DSC <0.5), performing worse in brain (median DSC = 0.65/0.78/0.75) and H&N (median DSC = 0.76/0.80/0.81). Qualitatively, AI1 and AI2 contours were acceptable (rated 1-2) with at most minor edits in 70.7% and 74.6% of ROIs (2906 ratings), higher for abdomen (AI1: 79.2%) and thorax (AI2: 90.2%), and lower for H&N (29.0/35.6%). An end-user survey showed strong user preference for full automation and mixed preferences for accuracy versus total number of structures generated. CONCLUSIONS: Our evaluation method provided a comprehensive analysis of both quantitative and qualitative measures of commercially available pretrained AI autocontouring algorithms. The evaluation framework served as a roadmap for clinical integration that aligned with user workflow preference.

Automated planning stage tracking and analysis through an integrated whiteboard system.

PURPOSE: One of the common challenges in delivering complex healthcare procedures such as radiation oncology is the organization and sharing of information in ways that facilitate workflow and prevent treatment delays. Within the major vendors of Oncology Information Systems (OIS) is a lack of tools and displays to assist in task timing and workflow processes. To address this issue, we developed an electronic whiteboard integrated with a local OIS to track, record, and evaluate time frames associated with clinical radiation oncology treatment planning processes. METHODS: We developed software using an R environment hosted on a local web-server at Seattle Cancer Care Alliance (SCCA) in 2017. The planning process was divided into stages, and time-stamped moves between planning stages were recorded automatically via Mosaiq (Elekta, Sweden) Quality Check Lists (QCLs). Whiteboard logs were merged with Mosaiq-extracted diagnostic factors and evaluated for significance. Interventional changes to task time expectations were evaluated for 6 months in 2021 and compared with 6 month periods in 2018 and 2019. RESULTS: Whiteboard/Mosaiq data from the SCCA show that treatment intent, number of prescriptions, and nodal involvement were main factors influencing overall time to plan completion. Contouring and Planning times were improved by 2.6 days (p<10-14) and 2.5 days (p<10-11), respectively. Overall time to plan completion was reduced by 33% (5.1 days; p<10-11). CONCLUSIONS: This report establishes the utility of real-time task tracking tools in a radiotherapy planning process. The whiteboard results provide data-driven evidence to add justification for practice change implementations.

Cardiac sparing characteristics of internal mammary chain radiotherapy using deep inspiration breath hold for left-sided breast cancer.

BACKGROUND: While breast radiotherapy typically includes regional nodal basins, the treatment of the internal mammary nodes (IMN) has been controversial due to concern for long-term cardiac toxicity. For high risk patients where IMN treatment is warranted, there is limited data with regards to the degree of heart sparing conferred by modern techniques. In this study, we sought to analyze the specific heart sparing metrics conferred by deep inspiration breath hold (DIBH) in the setting of IMN irradiation. METHODS: From 2012 to 2015, 168 consecutive patients were treated with adjuvant left-sided radiotherapy using DIBH. Retrospective review identified 49 patients who received nodal irradiation, either to a supraclavicular field (SCF) and IMN (16), or to the SCF alone (33). Cardiac mean dose and dose volumes were calculated from free breathing (FB) and DIBH treatment plans, and compared by Wilcoxon signed-rank and Mann-Whitney U tests. RESULTS: DIBH achieved significant reductions in mean heart dose (p < 0.001) in both the IMN treated group from 6.73 Gy to 2.79 Gy (- 56.4%) and the IMN untreated group from 4.77 Gy to 1.55 Gy (- 63.7%). There was a 7.3% difference in relative reduction that was not statistically significant (p = 0.216). Relative reductions in heart dose volume measures were all significantly lower for IMN-irradiated patients (p ≤ 0.012), with the greatest deficits at V5 that gradually diminish with increasing dose (V25). CONCLUSIONS: The relative heart sparing benefits of the DIBH technique are retained even with IMN inclusion. However, the addition of IMN irradiation is associated with an intrinsically greater heart dose, which translates to an estimated 9.2% proportional increase in the risk of a subsequent major coronary event. In the setting of effective cardiac sparing techniques, clinicians should take these considerations into account to guide when IMN treatment is warranted.

Dosimetric comparison of single-beam multi-arc and 2-beam multi-arc VMAT optimization in the Monaco treatment planning system.

The purpose of this study was to evaluate the dosimetric and practical effects of the Monaco treatment planning system "max arcs-per-beam" optimization parameter in pelvic radiotherapy treatments. We selected for this study a total of 17 previously treated patients with a range of pelvic disease sites including prostate (9), bladder (1), uterus (3), rectum (3), and cervix (1). For each patient, 2 plans were generated, one using an arc-per-beam setting of "1" and another with an arc-per-beam setting of "2" using the volumes and constraints established from the initial clinical treatments. All constraints and dose coverage objects were kept the same between plans, and all plans were normalized to 99.7% to ensure 100% of the planning target volume (PTV) received 95% of the prescription dose. Plans were evaluated for PTV conformity, homogeneity, number of monitor units, number of control points, and overall plan acceptability. Treatment delivery time, patient-specific quality assurance procedures, and the impact on clinical workflow were also assessed. We found that for complex-shaped target volumes (small central volumes with extending arms to cover nodal regions), the use of 2 arc-per-beam (2APB) parameter setting achieved significantly lower average dose-volume histogram values for the rectum V20 (p = 0.0012) and bladder V30 (p = 0.0036) while meeting the high dose target constraints. For simple PTV shapes, we found reduced monitor units (13.47%, p = 0.0009) and control points (8.77%, p = 0.0004) using 2APB planning. In addition, we found a beam delivery time reduction of approximately 25%. In summary, the dosimetric benefit, although moderate, was improved over a 1APB setting for complex PTV, and equivalent in other cases. The overall reduced delivery time suggests that the use of mulitple arcs per beam could lead to reduced patient-on-table time, increased clinical throughput, and reduced medical physics quality assurance effort.