Wide-Scale Clinical Implementation of Knowledge-Based Planning: An Investigation of Workforce Efficiency, Need for Post-automation Refinement, and Data-Driven Model Maintenance.

PURPOSE: Our purpose was to investigate the effect of automated knowledge-based planning (KBP) on real-world clinical workflow efficiency, assess whether manual refinement of KBP plans improves plan quality across multiple disease sites, and develop a data-driven method to periodically improve KBP automated planning routines. METHODS AND MATERIALS: Using clinical knowledge-based automated planning routines for prostate, prostatic fossa, head and neck, and hypofractionated lung disease sites in a commercial KBP solution, workflow efficiency was compared in terms of planning time in a pre-KBP (n = 145 plans) and post-KBP (n = 503) patient cohort. Post-KBP, planning was initialized with KBP (KBP-only) and subsequently manually refined (KBP +human). Differences in planning time were tested for significance using a 2-tailed Mann-Whitney U test (P < .05, null hypothesis: planning time unchanged). Post-refinement plan quality was assessed using site-specific dosimetric parameters of the original KBP-only plan versus KBP +human; 2-tailed paired t test quantified statistical significance (Bonferroni-corrected P < .05, null hypothesis: no dosimetric difference after refinement). If KBP +human significantly improved plans across the cohort, optimization objectives were changed to create an updated KBP routine (KBP'). Patients were replanned with KBP' and plan quality was compared with KBP +human as described previously. RESULTS: KBP significantly reduced planning time in all disease sites: prostate (median: 7.6 hrs â†’ 2.1 hrs; P < .001), prostatic fossa (11.1 hrs â†’ 3.7 hrs; P = .001), lung (9.9 hrs â†’ 2.0 hrs; P < .001), and head and neck (12.9 hrs â†’ 3.5 hrs; P <.001). In prostate, prostatic fossa, and lung disease sites, organ-at-risk dose changes in KBP +human versus KBP-only were minimal (<1% prescription dose). In head and neck, KBP +human did achieve clinically relevant dose reductions in some parameters. The head and neck routine was updated (KBP'HN) to incorporate dose improvements from manual refinement. The only significant dosimetric differences to KBP +human after replanning with KBP'HN were in favor of the new routine. CONCLUSIONS: KBP increased clinical efficiency by significantly reducing planning time. On average, human refinement offered minimal dose improvements over KBP-only plans. In the single disease site where KBP +human was superior to KBP-only, differences were eliminated by adjusting optimization parameters in a revised KBP routine.

Automated Closed- and Open-Loop Validation of Knowledge-Based Planning Routines Across Multiple Disease Sites.

PURPOSE: Knowledge-based planning (KBP) clinical implementation necessitates significant upfront effort, even within a single disease site. The purpose of this study was to demonstrate an efficient method for clinicians to assess the noninferiority of KBP across multiple disease sites and estimate any systematic dosimetric differences after implementation. We sought to establish these endpoints in a plurality of previously treated patients (validation set) with both closed-loop (training set overlapping validation set) and open-loop (independent training set) KBP routines. METHODS AND MATERIALS: We identified 53 prostate, 24 prostatic fossa, 54 hypofractionated lung, and 52 head and neck patients treated with volumetric modulated arc therapy in the year directly preceding our clinic's broad adoption of RapidPlan (Varian Medical Systems, Palo Alto, CA). Using the Varian Eclipse Scripting API, our program takes as input a list of patients, then performs semiautomated structure matching, fully automated RapidPlan-driven optimization, and plan comparison. All plans were normalized to the planning target volume (PTV) D95% = 100%. Dose metric differences (ΔDx = Dx,clinical - Dx,KBP) were computed for standard PTV and organ-at-risk (OAR) dose-volume histogram parameters across disease sites. A 2-tailed paired t test quantified statistical significance (P < .001). RESULTS: Statistically significant organ dose-volume histogram improvements were observed in the KBP cohort: the rectum, bladder, and penile bulb in prostate/prostatic fossa; and the larynx, esophagus, cricopharyngeus, parotid glands, and cochlea in head and neck. No OAR dose metric was statistically worse in any KBP sample. PTV ΔD1% increases in prostatic fossa were deemed acceptable given organ-sparing gains. PTV ΔD1% and internal target volume ΔD99% increase for the lung was by design owing to the prescription normalization variance in the pre-KBP lung sample. CONCLUSIONS: Our automated method showed multiple disease sites' KBP routines to be noninferior to manual planning, with statistically significant superiority in some aspects of OAR sparing. This method is applicable to any institution implementing either closed-loop or open-loop KBP autoplanning routines.