A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy.

BACKGROUND AND PURPOSE: Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap. METHODS AND MATERIALS: A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended. RESULTS: The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated. CONCLUSION: A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

Stereotactic body radiotherapy for centrally located inoperable early-stage NSCLC: EORTC 22113-08113 LungTech phase II trial results.

BACKGROUND: The international EORTC phase II single-arm LungTech trial 22113-08113 assessed safety and efficacy of stereotactic body radiotherapy (SBRT) in patients with centrally located early-stage non-small cell lung cancer (NSCLC). PATIENTS AND METHODS: Patients with inoperable non-metastatic central NSCLC (T1-T3 N0 M0, ≤7cm) were included. After prospective central imaging review and radiation therapy quality assurance (RTQA) for any eligible patient, SBRT (8x7.5 Gy, ICRU 83) was delivered. The primary endpoint was freedom from local progression probability at three years after start of SBRT. RESULTS: The trial was closed earlier due to poor accrual related to repeated safety-related pauses in recruitment. Between 08/2015 and 12/2017, 39 patients from 6 European countries were included and 31 were treated per protocol and analyzed. Patients were mainly male (58%) with a median age of 75 years. Baseline comorbidities were mainly respiratory (68%) and cardiac (48%). Median tumor size was 2.6 cm (range, 1.2-5.5) and most cancers were T1 (51.6%) or T2a (38.7%) N0 M0 and of squamous cell origin (48.4%). Median follow-up was 3.6 years. The 3-year freedom from local progression and overall survival rates were 81.5% (90% CI: 62.7-91.4%) and 61.1% (90%CI: 44.1-74.4%), respectively. Cumulative incidence rates of local, regional and distant progression at 3 years were 6.7% (90% CI: 1.6-17.1%), 3.3% (90% CI: 0.4 - 12.4%) and 29.8% (90% CI: 16.8 - 44.1%), respectively. SBRT-related acute and late AEs ≥ G3 were reported in 6.5% (n=2, including one G5 pneumonitis in a patient with prior interstitial lung disease) and 19.4 % (n=6, including one lethal hemoptysis after a lung biopsy in a patient receiving anticoagulants), respectively. CONCLUSION: The LungTech trial suggests that SBRT with 8×7.5Gy for central lung tumors in inoperable patients is associated with acceptable local control rates. However, late severe adverse events may occur after completion of treatment. This SBRT regimen is a viable treatment option after thorough risk-benefit discussion with patients. To minimize potentially fatal toxicity, careful management of dose constraints and post-SBRT interventions is crucial.

Assessment of bias in scoring of AI-based radiotherapy segmentation and planning studies using modified TRIPOD and PROBAST guidelines as an example.

BACKGROUND AND PURPOSE: Studies investigating the application of Artificial Intelligence (AI) in the field of radiotherapy exhibit substantial variations in terms of quality. The goal of this study was to assess the amount of transparency and bias in scoring articles with a specific focus on AI based segmentation and treatment planning, using modified PROBAST and TRIPOD checklists, in order to provide recommendations for future guideline developers and reviewers. MATERIALS AND METHODS: The TRIPOD and PROBAST checklist items were discussed and modified using a Delphi process. After consensus was reached, 2 groups of 3 co-authors scored 2 articles to evaluate usability and further optimize the adapted checklists. Finally, 10 articles were scored by all co-authors. Fleiss' kappa was calculated to assess the reliability of agreement between observers. RESULTS: Three of the 37 TRIPOD items and 5 of the 32 PROBAST items were deemed irrelevant. General terminology in the items (e.g., multivariable prediction model, predictors) was modified to align with AI-specific terms. After the first scoring round, further improvements of the items were formulated, e.g., by preventing the use of sub-questions or subjective words and adding clarifications on how to score an item. Using the final consensus list to score the 10 articles, only 2 out of the 61 items resulted in a statistically significant kappa of 0.4 or more demonstrating substantial agreement. For 41 items no statistically significant kappa was obtained indicating that the level of agreement among multiple observers is due to chance alone. CONCLUSION: Our study showed low reliability scores with the adapted TRIPOD and PROBAST checklists. Although such checklists have shown great value during development and reporting, this raises concerns about the applicability of such checklists to objectively score scientific articles for AI applications. When developing or revising guidelines, it is essential to consider their applicability to score articles without introducing bias.

Knowledge-based versus deep learning based treatment planning for breast radiotherapy.

Background and Purpose: To improve radiotherapy (RT) planning efficiency and plan quality, knowledge-based planning (KBP) and deep learning (DL) solutions have been developed. We aimed to make a direct comparison of these models for breast cancer planning using the same training, validation, and testing sets. Materials and Methods: Two KBP models were trained and validated with 90 RT plans for left-sided breast cancer with 15 fractions of 2.6 Gy. The versions either used the full dataset (non-clean model) or a cleaned dataset (clean model), thus eliminating geometric and dosimetric outliers. Results were compared with a DL U-net model (previously trained and validated with the same 90 RT plans) and manually produced RT plans, for the same independent dataset of 15 patients. Clinically relevant dose volume histogram parameters were evaluated according to established consensus criteria. Results: Both KBP models underestimated the mean heart and lung dose equally 0.4 Gy (0.3-1.1 Gy) and 1.4 Gy (1.1-2.8 Gy) compared to the clinical plans 0.8 Gy (0.5-1.8 Gy) and 1.7 Gy (1.3-3.2 Gy) while in the final calculations the mean lung dose was higher 1.9-2.0 Gy (1.5-3.5 Gy) for both KPB models. The U-Net model resulted in a mean planning target volume dose of 40.7 Gy (40.4-41.3 Gy), slightly higher than the clinical plans 40.5 Gy (40.1-41.0 Gy). Conclusions: Only small differences were observed between the estimated and final dose calculation and the clinical results for both KPB models and the DL model. With a good set of breast plans, the data cleaning module is not needed and both KPB and DL models lead to clinically acceptable results.

Real-time motion management in MRI-guided radiotherapy: Current status and AI-enabled prospects.

MRI-guided radiotherapy (MRIgRT) is a highly complex treatment modality, allowing adaptation to anatomical changes occurring from one treatment day to the other (inter-fractional), but also to motion occurring during a treatment fraction (intra-fractional). In this vision paper, we describe the different steps of intra-fractional motion management during MRIgRT, from imaging to beam adaptation, and the solutions currently available both clinically and at a research level. Furthermore, considering the latest developments in the literature, a workflow is foreseen in which motion-induced over- and/or under-dosage is compensated in 3D, with minimal impact to the radiotherapy treatment time. Considering the time constraints of real-time adaptation, a particular focus is put on artificial intelligence (AI) solutions as a fast and accurate alternative to conventional algorithms.

Evaluation of a clinically introduced deep learning model for radiotherapy treatment planning of breast cancer.

Deep learning (DL) models are increasingly studied to automate the process of radiotherapy treatment planning. This study evaluates the clinical use of such a model for whole breast radiotherapy. Treatment plans were automatically generated, after which planners were allowed to manually adapt them. Plans were evaluated based on clinical goals and DVH parameters. Thirty-seven of 50plans did fulfill all clinical goals without adjustments. Thirteen of these 37 plans were still adjusted but did not improve mean heart or lung dose. These results leave room for improvement of both the DL model as well as education on clinically relevant adjustments.

Inter-observer variability in library plan selection on iterative CBCT and synthetic CT images of cervical cancer patients.

INTRODUCTION: In the Library-of-Plans (LoP) approach, correct plan selection is essential for delivering radiotherapy treatment accurately. However, poor image quality of the cone-beam computed tomography (CBCT) may introduce inter-observer variability and thereby hamper accurate plan selection. In this study, we investigated whether new techniques to improve the CBCT image quality and improve consistency in plan selection, affects the accuracy of LoP selection in cervical cancer patients. MATERIALS AND METHODS: CBCT images of 12 patients were used to investigate the inter-observer variability of plan selection based on different CBCT image types. Six observers were asked to individually select a plan based on clinical X-ray Volumetric Imaging (XVI) CBCT, iterative reconstructed CBCT (iCBCT) and synthetic CTs (sCT). Selections were performed before and after a consensus meeting with the entire group, in which guidelines were created. A scoring by all observers on the image quality and plan selection procedure was also included. For plan selection, Fleiss' kappa (κ) statistical test was used to determine the inter-observer variability within one image type. RESULTS: The agreement between observers was significantly higher on sCT compared to CBCT. The consensus meeting improved the duration and inter-observer variability. In this manuscript, the guidelines attributed the overall results in the plan selection. Before the meeting, the gold standard was selected in 76% of the cases on XVI CBCT, 74% on iCBCT, and 76% on sCT. After the meeting, the gold standard was selected in 83% of the cases on XVI CBCT, 81% on iCBCT, and 90% on sCT. CONCLUSION: The use of sCTs can increase the agreement of plan selection among observers and the gold standard was indicated to be selected more often. It is important that clear guidelines for plan selection are implemented in order to benefit from the increased image quality, accurate selection, and decrease inter-observer variability.

Re-irradiation in clinical practice: Results of an international patterns of care survey within the framework of the ESTRO-EORTC E2-RADIatE platform.

BACKGROUND: Re-irradiation is an increasingly utilized treatment for recurrent, metastatic or new malignancies after previous radiotherapy. It is unclear how re-irradiation is applied in clinical practice. We aimed to investigate the patterns of care of re-irradiation internationally. MATERIAL/METHODS: A cross-sectional survey conducted between March and September 2022. The survey was structured into six sections, each corresponding to a specific anatomical region. Participants were instructed to complete the sections of their clinical expertise. A total of 15 multiple-choice questions were included in each section, addressing various aspects of the re-irradiation process. The online survey targeted radiation and clinical oncologists and was endorsed by the European Society for Radiotherapy and Oncology (ESTRO) and the European Organisation for Research and Treatment of Cancer (EORTC). RESULTS: 371 physicians from 55 countries across six continents participated. Participants had a median professional experience of 16 years, and the majority (60%) were affiliated with an academic hospital. The brain region was the most common site for re-irradiation (77%), followed by the pelvis (65%) and head and neck (63%). Prolonging local control was the most common goal (90-96% across anatomical regions). The most common minimum interval between previous radiotherapy and re-irradiation was 6-12 months (45-55%). Persistent grade 3 or greater radiation-induced toxicity (77-80%) was the leading contraindication. Variability in organs at risk dose constraints for re-irradiation was observed. Advanced imaging modalities and conformal radiotherapy techniques were predominantly used. A scarcity of institutional guidelines for re-irradiation was reported (16-19%). Participants from European centers more frequently applied thoracic and abdominal re-irradiation. Indications did not differ between academic and non-academic hospitals. CONCLUSION: This study highlights the heterogeneity in re-irradiation practices across anatomical regions and emphasizes the need for high-quality evidence from prospective studies to guide treatment decisions and derive safe cumulative dose constraints.

Clinical evaluation of a deep learning segmentation model including manual adjustments afterwards for locally advanced breast cancer.

Introduction: Deep learning (DL) models are increasingly developed for auto-segmentation in radiotherapy. Qualitative analysis is of great importance for clinical implementation, next to quantitative. This study evaluates a DL segmentation model for left- and right-sided locally advanced breast cancer both quantitatively and qualitatively. Methods: For each side a DL model was trained, including primary breast CTV (CTVp), lymph node levels 1-4, heart, lungs, humeral head, thyroid and esophagus. For evaluation, both automatic segmentation, including correction of contours when needed, and manual delineation was performed and both processes were timed. Quantitative scoring with dice-similarity coefficient (DSC), 95% Hausdorff Distance (95%HD) and surface DSC (sDSC) was used to compare both the automatic (not-corrected) and corrected contours with the manual contours. Qualitative scoring was performed by five radiotherapy technologists and five radiation oncologists using a 3-point Likert scale. Results: Time reduction was achieved using auto-segmentation in 95% of the cases, including correction. The time reduction (mean ± std) was 42.4% ± 26.5% and 58.5% ± 19.1% for OARs and CTVs, respectively, corresponding to an absolute mean reduction (hh:mm:ss) of 00:08:51 and 00:25:38. Good quantitative results were achieved before correction, e.g. mean DSC for the right-sided CTVp was 0.92 ± 0.06, whereas correction statistically significantly improved this contour by only 0.02 ± 0.05, respectively. In 92% of the cases, auto-contours were scored as clinically acceptable, with or without corrections. Conclusions: A DL segmentation model was trained and was shown to be a time-efficient way to generate clinically acceptable contours for locally advanced breast cancer.

Comparison of the output of a deep learning segmentation model for locoregional breast cancer radiotherapy trained on 2 different datasets.

Introduction: The development of deep learning (DL) models for auto-segmentation is increasing and more models become commercially available. Mostly, commercial models are trained on external data. To study the effect of using a model trained on external data, compared to the same model trained on in-house collected data, the performance of these two DL models was evaluated. Methods: The evaluation was performed using in-house collected data of 30 breast cancer patients. Quantitative analysis was performed using Dice similarity coefficient (DSC), surface DSC (sDSC) and 95th percentile of Hausdorff Distance (95% HD). These values were compared with previously reported inter-observer variations (IOV). Results: For a number of structures, statistically significant differences were found between the two models. For organs at risk, mean values for DSC ranged from 0.63 to 0.98 and 0.71 to 0.96 for the in-house and external model, respectively. For target volumes, mean DSC values of 0.57 to 0.94 and 0.33 to 0.92 were found. The difference of 95% HD values ranged 0.08 to 3.23 mm between the two models, except for CTVn4 with 9.95 mm. For the external model, both DSC and 95% HD are outside the range of IOV for CTVn4, whereas this is the case for the DSC found for the thyroid of the in-house model. Conclusions: Statistically significant differences were found between both models, which were mostly within published inter-observer variations, showing clinical usefulness of both models. Our findings could encourage discussion and revision of existing guidelines, to further decrease inter-observer, but also inter-institute variability.

STereotactic Arrhythmia Radioablation (STAR): the Standardized Treatment and Outcome Platform for Stereotactic Therapy Of Re-entrant tachycardia by a Multidisciplinary consortium (STOPSTORM.eu) and review of current patterns of STAR practice in Europe.

The EU Horizon 2020 Framework-funded Standardized Treatment and Outcome Platform for Stereotactic Therapy Of Re-entrant tachycardia by a Multidisciplinary (STOPSTORM) consortium has been established as a large research network for investigating STereotactic Arrhythmia Radioablation (STAR) for ventricular tachycardia (VT). The aim is to provide a pooled treatment database to evaluate patterns of practice and outcomes of STAR and finally to harmonize STAR within Europe. The consortium comprises 31 clinical and research institutions. The project is divided into nine work packages (WPs): (i) observational cohort; (ii) standardization and harmonization of target delineation; (iii) harmonized prospective cohort; (iv) quality assurance (QA); (v) analysis and evaluation; (vi, ix) ethics and regulations; and (vii, viii) project coordination and dissemination. To provide a review of current clinical STAR practice in Europe, a comprehensive questionnaire was performed at project start. The STOPSTORM Institutions' experience in VT catheter ablation (83% ≥ 20 ann.) and stereotactic body radiotherapy (59% > 200 ann.) was adequate, and 84 STAR treatments were performed until project launch, while 8/22 centres already recruited VT patients in national clinical trials. The majority currently base their target definition on mapping during VT (96%) and/or pace mapping (75%), reduced voltage areas (63%), or late ventricular potentials (75%) during sinus rhythm. The majority currently apply a single-fraction dose of 25 Gy while planning techniques and dose prescription methods vary greatly. The current clinical STAR practice in the STOPSTORM consortium highlights potential areas of optimization and harmonization for substrate mapping, target delineation, motion management, dosimetry, and QA, which will be addressed in the various WPs.

Anatomical evaluation of deep-learning synthetic computed tomography images generated from male pelvis cone-beam computed tomography.

Background and purpose: To improve cone-beam computed tomography (CBCT), deep-learning (DL)-models are being explored to generate synthetic CTs (sCT). The sCT evaluation is mainly focused on image quality and CT number accuracy. However, correct representation of daily anatomy of the CBCT is also important for sCTs in adaptive radiotherapy. The aim of this study was to emphasize the importance of anatomical correctness by quantitatively assessing sCT scans generated from CBCT scans using different paired and unpaired dl-models. Materials and methods: Planning CTs (pCT) and CBCTs of 56 prostate cancer patients were included to generate sCTs. Three different dl-models, Dual-UNet, Single-UNet and Cycle-consistent Generative Adversarial Network (CycleGAN), were evaluated on image quality and anatomical correctness. The image quality was assessed using image metrics, such as Mean Absolute Error (MAE). The anatomical correctness between sCT and CBCT was quantified using organs-at-risk volumes and average surface distances (ASD). Results: MAE was 24 Hounsfield Unit (HU) [range:19-30 HU] for Dual-UNet, 40 HU [range:34-56 HU] for Single-UNet and 41HU [range:37-46 HU] for CycleGAN. Bladder ASD was 4.5 mm [range:1.6-12.3 mm] for Dual-UNet, 0.7 mm [range:0.4-1.2 mm] for Single-UNet and 0.9 mm [range:0.4-1.1 mm] CycleGAN. Conclusions: Although Dual-UNet performed best in standard image quality measures, such as MAE, the contour based anatomical feature comparison with the CBCT showed that Dual-UNet performed worst on anatomical comparison. This emphasizes the importance of adding anatomy based evaluation of sCTs generated by dl-models. For applications in the pelvic area, direct anatomical comparison with the CBCT may provide a useful method to assess the clinical applicability of dl-based sCT generation methods.

Corrigendum to "Harmonization of breast cancer radiotherapy treatment planning in the Netherlands" [Tech. Innov. Patient Support Radiat. Oncol. 19 (2021) 26-32].

[This corrects the article DOI: 10.1016/j.tipsro.2021.06.004.].

European Society for Radiotherapy and Oncology and European Organisation for Research and Treatment of Cancer consensus on re-irradiation: definition, reporting, and clinical decision making.

Re-irradiation can be considered for local recurrence or new tumours adjacent to a previously irradiated site to achieve durable local control for patients with cancer who have otherwise few therapeutic options. With the use of new radiotherapy techniques, which allow for conformal treatment plans, image guidance, and short fractionation schemes, the use of re-irradiation for different sites is increasing in clinical settings. Yet, prospective evidence on re-irradiation is scarce and our understanding of the underlying radiobiology is poor. Our consensus on re-irradiation aims to assist in re-irradiation decision making, and to standardise the classification of different forms of re-irradiation and reporting. The consensus has been endorsed by the European Society for Radiotherapy and Oncology and the European Organisation for Research and Treatment of Cancer. The use of this classification in daily clinical practice and research will facilitate accurate understanding of the clinical implications of re-irradiation and allow for cross-study comparisons. Data gathered in a uniform manner could be used in the future to make recommendations for re-irradiation on the basis of clinical evidence. The consensus document is based on an adapted Delphi process and a systematic review of the literature was done according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

Critical impact of radiotherapy protocol compliance and quality in the treatment of retroperitoneal sarcomas: Results from the EORTC 62092-22092 STRASS trial.

BACKGROUND: The European Organization for Research and Treatment of Cancer 22092-62092 STRASS trial failed to demonstrate the superiority of neoadjuvant radiotherapy (RT) over surgery alone in patients with retroperitoneal sarcoma. Therefore, an RT quality-assurance program was added to the study protocol to detect and correct RT deviations. The authors report results from the trial RT quality-assurance program and its potential effect on patient outcomes. METHODS: To evaluate the effect of RT compliance on survival outcomes, a composite end point was created. It combined the information related to planning target volume coverage, target delineation, total dose received, and overall treatment time into 2 groups: non-RT-compliant (NRC) for patients who had unacceptable deviation(s) in any of the previous categories and RT-compliant (RC) otherwise. Abdominal recurrence-free survival (ARFS) and overall survival were compared between the 2 groups using a Cox proportional hazard model adjusted for known prognostic factors. RESULTS: Thirty-six of 125 patients (28.8%) were classified as NRC, and the remaining 89 patients (71.2%) were classified as RC. The 3-year ARFS rate was 66.8% (95% confidence interval [CI], 55.8%-75.7%) and 49.8% (95% CI, 32.7%-64.8%) for the RC and NRC groups, respectively (adjusted hazard ratio, 2.32; 95% CI, 1.25-4.32; P = .008). Local recurrence after macroscopic complete resection occurred in 13 of 89 patients (14.6%) versus 2 of 36 patients (5.6%) in the RC and NRC groups, respectively. CONCLUSIONS: The current analysis suggests a significant benefit in terms of ARFS in favor of the RC group. This association did not translate into less local relapses after complete resection in the RC group. Multidisciplinary collaboration and review of cases are critical to avoid geographic misses, especially for rare tumors like retroperitoneal sarcoma.

Transparency in quality of radiotherapy for breast cancer in the Netherlands: a national registration of radiotherapy-parameters.

BACKGROUND: Radiotherapy (RT) is part of the curative treatment of approximately 70% of breast cancer (BC) patients. Wide practice variation has been reported in RT dose, fractionation and its treatment planning for BC. To decrease this practice variation, it is essential to first gain insight into the current variation in RT treatment between institutes. This paper describes the development of the NABON Breast Cancer Audit-Radiotherapy (NBCA-R), a structural nationwide registry of BC RT data of all BC patients treated with at least surgery and RT. METHODS: A working group consisting of representatives of the BC Platform of the Dutch Radiotherapy Society selected a set of dose volume parameters deemed to be surrogate outcome parameters, both for tumour control and toxicity. Two pilot studies were carried out in six RT institutes. In the first pilot study, data were manually entered into a secured web-based system. In the second pilot study, an automatic Digital Imaging and Communications in Medicine (DICOM) RT upload module was created and tested. RESULTS: The NBCA-R dataset was created by selecting RT parameters describing given dose, target volumes, coverage and homogeneity, and dose to organs at risk (OAR). Entering the data was made mandatory for all Dutch RT departments. In the first pilot study (N = 1093), quite some variation was already detected. Application of partial breast irradiation varied from 0 to 17% between the 6 institutes and boost to the tumour bed from 26.5 to 70.2%. For patients treated to the left breast or chest wall only, the average mean heart dose (MHD) varied from 0.80 to 1.82 Gy; for patients treated to the breast/chest wall only, the average mean lung dose (MLD) varied from 2.06 to 3.3 Gy. In the second pilot study 6 departments implemented the DICOM-RT upload module in daily practice. Anonymised data will be available for researchers via a FAIR (Findable, Accessible, Interoperable, Reusable) framework. CONCLUSIONS: We have developed a set of RT parameters and implemented registration for all Dutch BC patients. With the use of an automated upload module registration burden will be minimized. Based on the data in the NBCA-R analyses of the practice variation will be done, with the ultimate aim to improve quality of BC RT. Trial registration Retrospectively registered.

Clinical evaluation of two AI models for automated breast cancer plan generation.

BACKGROUND: Artificial intelligence (AI) shows great potential to streamline the treatment planning process. However, its clinical adoption is slow due to the limited number of clinical evaluation studies and because often, the translation of the predicted dose distribution to a deliverable plan is lacking. This study evaluates two different, deliverable AI plans in terms of their clinical acceptability based on quantitative parameters and qualitative evaluation by four radiation oncologists. METHODS: For 20 left-sided node-negative breast cancer patients, treated with a prescribed dose of 40.05 Gy, using tangential beam intensity modulated radiotherapy, two model-based treatment plans were evaluated against the corresponding manual plan. The two models used were an in-house developed U-net model and a vendor-developed contextual atlas regression forest model (cARF). Radiation oncologists evaluated the clinical acceptability of each blinded plan and ranked plans according to preference. Furthermore, a comparison with the manual plan was made based on dose volume histogram parameters, clinical evaluation criteria and preparation time. RESULTS: The U-net model resulted in a higher average and maximum dose to the PTV (median difference 0.37 Gy and 0.47 Gy respectively) and a slightly higher mean heart dose (MHD) (0.01 Gy). The cARF model led to higher average and maximum doses to the PTV (0.30 and 0.39 Gy respectively) and a slightly higher MHD (0.02 Gy) and mean lung dose (MLD, 0.04 Gy). The maximum MHD/MLD difference was ≤ 0.5 Gy for both AI plans. Regardless of these dose differences, 90-95% of the AI plans were considered clinically acceptable versus 90% of the manual plans. Preferences varied between the radiation oncologists. Plan preparation time was comparable between the U-net model and the manual plan (287 s vs 253 s) while the cARF model took longer (471 s). When only considering user interaction, plan generation time was 121 s for the cARF model and 137 s for the U-net model. CONCLUSIONS: Two AI models were used to generate deliverable plans for breast cancer patients, in a time-efficient manner, requiring minimal user interaction. Although the AI plans resulted in slightly higher doses overall, radiation oncologists considered 90-95% of the AI plans clinically acceptable.