Wide-Scale Clinical Implementation of Knowledge-Based Planning: An Investigation of Workforce Efficiency, Need for Post-automation Refinement, and Data-Driven Model Maintenance.

PURPOSE: Our purpose was to investigate the effect of automated knowledge-based planning (KBP) on real-world clinical workflow efficiency, assess whether manual refinement of KBP plans improves plan quality across multiple disease sites, and develop a data-driven method to periodically improve KBP automated planning routines. METHODS AND MATERIALS: Using clinical knowledge-based automated planning routines for prostate, prostatic fossa, head and neck, and hypofractionated lung disease sites in a commercial KBP solution, workflow efficiency was compared in terms of planning time in a pre-KBP (n = 145 plans) and post-KBP (n = 503) patient cohort. Post-KBP, planning was initialized with KBP (KBP-only) and subsequently manually refined (KBP +human). Differences in planning time were tested for significance using a 2-tailed Mann-Whitney U test (P < .05, null hypothesis: planning time unchanged). Post-refinement plan quality was assessed using site-specific dosimetric parameters of the original KBP-only plan versus KBP +human; 2-tailed paired t test quantified statistical significance (Bonferroni-corrected P < .05, null hypothesis: no dosimetric difference after refinement). If KBP +human significantly improved plans across the cohort, optimization objectives were changed to create an updated KBP routine (KBP'). Patients were replanned with KBP' and plan quality was compared with KBP +human as described previously. RESULTS: KBP significantly reduced planning time in all disease sites: prostate (median: 7.6 hrs â†’ 2.1 hrs; P < .001), prostatic fossa (11.1 hrs â†’ 3.7 hrs; P = .001), lung (9.9 hrs â†’ 2.0 hrs; P < .001), and head and neck (12.9 hrs â†’ 3.5 hrs; P <.001). In prostate, prostatic fossa, and lung disease sites, organ-at-risk dose changes in KBP +human versus KBP-only were minimal (<1% prescription dose). In head and neck, KBP +human did achieve clinically relevant dose reductions in some parameters. The head and neck routine was updated (KBP'HN) to incorporate dose improvements from manual refinement. The only significant dosimetric differences to KBP +human after replanning with KBP'HN were in favor of the new routine. CONCLUSIONS: KBP increased clinical efficiency by significantly reducing planning time. On average, human refinement offered minimal dose improvements over KBP-only plans. In the single disease site where KBP +human was superior to KBP-only, differences were eliminated by adjusting optimization parameters in a revised KBP routine.

Noninferiority Study of Automated Knowledge-Based Planning Versus Human-Driven Optimization Across Multiple Disease Sites.

PURPOSE: To evaluate whether automated knowledge-based planning (KBP) (a) is noninferior to human-driven planning across multiple disease sites and (b) systematically affects dosimetric plan quality and variability. METHODS AND MATERIALS: Clinical KBP automated planning routines were developed for prostate, prostatic fossa, hypofractionated lung, and head and neck. Clinical implementation consisted of independent generation of human-generated and KBP plans (145 cases across all sites), followed by blinded plan selection. Reviewing physicians were prompted to select a single plan; when plan equivalence was volunteered, this scored as KBP selection. Plan selection analysis used a noninferiority framework testing the hypothesis that KBP is not worse than human-driven planning (threshold: lower 95% confidence interval [CI] > 0.45 = noninferiority; > 0.5 = superiority). Target and organ-at-risk metrics were compared by dose differencing: ΔDx = Dx, human-Dx, KBP (2-tailed paired t test, Bonferroni-corrected P < .05 significance threshold). To evaluate the aggregated effect of KBP on planning performance, we examined post-KBP dosimetric parameters against 183 plans generated just before KBP implementation (2-tailed unpaired t test, Bonferroni-corrected P < .05). RESULTS: Across all disease sites, the KBP success rate (physician preferred + equivalent) was noninferior compared with human-driven planning (83 of 145 = 57.2%; range, 49.2%-65.3%) but did not cross the threshold for superiority. The KBP success rate in respective disease sites was superior with head and neck ([22 + 2]/36 = 66.7%; 95% CI, 51%-82%) and noninferior for lung stereotactic body radiation therapy ([21 + 2]/36 = 63.9%; 95% CI, 48%-80%) but did not meet noninferiority criteria with prostate ([16 + 3]/41 = 46.3%; 95% CI, 31%-62%) or prostatic fossa ([17 + 0]/32 = 53.1%; 95% CI, 36%-70%). Prostate, prostatic fossa, and head and neck showed significant differences in KBP-selected plans versus human-selected plans, with KBP generally exhibiting greater organ-at-risk sparing and human plans exhibiting better target homogeneity. Analysis of plan quality pre- and post-KBP showed some reductions in organ doses and quality metric variability in prostate and head and neck. CONCLUSIONS: Fully automated KBP was noninferior to human-driven plan optimization across multiple disease sites. Dosimetric analysis of treatment plans before and after KBP implementation showed a systematic shift to higher plan quality and lower variability with the introduction of KBP.