Health Care Cost Reductions with Machine Learning-Directed Evaluations during Radiation Therapy - An Economic Analysis of a Randomized Controlled Study.

BACKGROUND: Machine learning (ML) may cost-effectively direct health care by identifying patients most likely to benefit from preventative interventions to avoid negative and expensive outcomes. System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT; NCT04277650) was a single-institution, randomized controlled study in which electronic health record-based ML accurately identified patients at high risk for acute care (emergency visit or hospitalization) during radiotherapy (RT) and targeted them for supplemental clinical evaluations. This ML-directed intervention resulted in decreased acute care utilization. Given the limited prospective data showing the ability of ML to direct interventions cost-efficiently, an economic analysis was performed. METHODS: A post hoc economic analysis was conducted of SHIELD-RT that included RT courses from January 7, 2019, to June 30, 2019. ML-identified high-risk courses (≥10% risk of acute care during RT) were randomized to receive standard of care weekly clinical evaluations with ad hoc supplemental evaluations per clinician discretion versus mandatory twice-weekly evaluations. The primary outcome was difference in mean total medical costs during and 15 days after RT. Acute care costs were obtained via institutional cost accounting. Physician and intervention costs were estimated via Medicare and Medicaid data. Negative binomial regression was used to estimate cost outcomes after adjustment for patient and disease factors. RESULTS: A total of 311 high-risk RT courses among 305 patients were randomized to the standard (n=157) or the intervention (n=154) group. Unadjusted mean intervention group supplemental visit costs were $155 per course (95% confidence interval, $142 to $168). The intervention group had fewer acute care visits per course (standard, 0.47; intervention, 0.31; P=0.04). Total mean adjusted costs were $3110 per course for the standard group and $1494 for the intervention group (difference in means, $1616 [95% confidence interval, $1450 to $1783]; P=0.03). CONCLUSIONS: In this economic analysis of a randomized controlled, health care ML study, mandatory supplemental evaluations for ML-identified high-risk patients were associated with both reduced total medical costs and improved clinical outcomes. Further study is needed to determine whether economic results are generalizable. (Funded in part by The Duke Endowment, The Conquer Cancer Foundation, the Duke Department of Radiation Oncology, and the National Cancer Institute of the National Institutes of Health [R01CA277782]; ClinicalTrials.gov number, NCT04277650.).

Healthcare provider evaluation of machine learning-directed care: reactions to deployment on a randomised controlled study.

OBJECTIVES: Clinical artificial intelligence and machine learning (ML) face barriers related to implementation and trust. There have been few prospective opportunities to evaluate these concerns. System for High Intensity EvaLuation During Radiotherapy (NCT03775265) was a randomised controlled study demonstrating that ML accurately directed clinical evaluations to reduce acute care during cancer radiotherapy. We characterised subsequent perceptions and barriers to implementation. METHODS: An anonymous 7-question Likert-type scale survey with optional free text was administered to multidisciplinary staff focused on workflow, agreement with ML and patient experience. RESULTS: 59/71 (83%) responded. 81% disagreed/strongly disagreed their workflow was disrupted. 67% agreed/strongly agreed patients undergoing intervention were high risk. 75% agreed/strongly agreed they would implement the ML approach routinely if the study was positive. Free-text feedback focused on patient education and ML predictions. CONCLUSIONS: Randomised data and firsthand experience support positive reception of clinical ML. Providers highlighted future priorities, including patient counselling and workflow optimisation.

Implementation of machine learning in the clinic: challenges and lessons in prospective deployment from the System for High Intensity EvaLuation During Radiation Therapy (SHIELD-RT) randomized controlled study.

BACKGROUND: Artificial intelligence (AI) and machine learning (ML) have resulted in significant enthusiasm for their promise in healthcare. Despite this, prospective randomized controlled trials and successful clinical implementation remain limited. One clinical application of ML is mitigation of the increased risk for acute care during outpatient cancer therapy. We previously reported the results of the System for High Intensity EvaLuation During Radiation Therapy (SHIELD-RT) study (NCT04277650), which was a prospective, randomized quality improvement study demonstrating that ML based on electronic health record (EHR) data can direct supplemental clinical evaluations and reduce the rate of acute care during cancer radiotherapy with and without chemotherapy. The objective of this study is to report the workflow and operational challenges encountered during ML implementation on the SHIELD-RT study. RESULTS: Data extraction and manual review steps in the workflow represented significant time commitments for implementation of clinical ML on a prospective, randomized study. Barriers include limited data availability through the standard clinical workflow and commercial products, the need to aggregate data from multiple sources, and logistical challenges from altering the standard clinical workflow to deliver adaptive care. CONCLUSIONS: The SHIELD-RT study was an early randomized controlled study which enabled assessment of barriers to clinical ML implementation, specifically those which leverage the EHR. These challenges build on a growing body of literature and may provide lessons for future healthcare ML adoption. TRIAL REGISTRATION: NCT04277650. Registered 20 February 2020. Retrospectively registered quality improvement study.

System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning-Directed Clinical Evaluations During Radiation and Chemoradiation.

PURPOSE: Patients undergoing outpatient radiotherapy (RT) or chemoradiation (CRT) frequently require acute care (emergency department evaluation or hospitalization). Machine learning (ML) may guide interventions to reduce this risk. There are limited prospective studies investigating the clinical impact of ML in health care. The objective of this study was to determine whether ML can identify high-risk patients and direct mandatory twice-weekly clinical evaluation to reduce acute care visits during treatment. PATIENTS AND METHODS: During this single-institution randomized quality improvement study (ClinicalTrials.gov identifier: NCT04277650), 963 outpatient adult courses of RT and CRT started from January 7 to June 30, 2019, were evaluated by an ML algorithm. Among these, 311 courses identified by ML as high risk (> 10% risk of acute care during treatment) were randomized to standard once-weekly clinical evaluation (n = 157) or mandatory twice-weekly evaluation (n = 154). Both arms allowed additional evaluations on the basis of clinician discretion. The primary end point was the rate of acute care visits during RT. Model performance was evaluated using receiver operating characteristic area under the curve (AUC) and decile calibration plots. RESULTS: Twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (difference, -10.0%; 95% CI, -18.3 to -1.6; relative risk, 0.556; 95% CI, 0.332 to 0.924; P = .02). Low-risk patients had a 2.7% acute care rate. Model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC, 0.851). CONCLUSION: In this prospective randomized study, ML accurately triaged patients undergoing RT and CRT, directing clinical management with reduced acute care rates versus standard of care. This prospective study demonstrates the potential benefit of ML in health care and offers opportunities to enhance care quality and reduce health care costs.

Mid-radiotherapy PET/CT for prognostication and detection of early progression in patients with stage III non-small cell lung cancer.

BACKGROUND AND PURPOSE: Pre- and mid-radiotherapy FDG-PET metrics have been proposed as biomarkers of recurrence and survival in patients treated for stage III non-small cell lung cancer. We evaluated these metrics in patients treated with definitive radiation therapy (RT). We also evaluated outcomes after progression on mid-radiotherapy PET/CT. MATERIAL AND METHODS: Seventy-seven patients treated with RT with or without chemotherapy were included in this retrospective study. Primary tumor and involved nodes were delineated. PET metrics included metabolic tumor volume (MTV), total lesion glycolysis (TLG), and SUVmax. For mid-radiotherapy PET, both absolute value of these metrics and percentage decrease were analyzed. The influence of PET metrics on time to death, local recurrence, and regional/distant recurrence was assessed using Cox regression. RESULTS: 91% of patients had concurrent chemotherapy. Median follow-up was 14months. None of the PET metrics were associated with overall survival. Several were positively associated with local recurrence: pre-radiotherapy MTV, and mid-radiotherapy MTV and TLG (p=0.03-0.05). Ratio of mid- to pre-treatment SUVmax was associated with regional/distant recurrence (p=0.02). 5/77 mid-radiotherapy scans showed early out-of-field progression. All of these patients died. CONCLUSIONS: Several PET metrics were associated with risk of recurrence. Progression on mid-radiotherapy PET/CT was a poor prognostic factor.

Pulmonary ventilation imaging based on 4-dimensional computed tomography: comparison with pulmonary function tests and SPECT ventilation images.

PURPOSE: 4-dimensional computed tomography (4D-CT)-based pulmonary ventilation imaging is an emerging functional imaging modality. The purpose of this study was to investigate the physiological significance of 4D-CT ventilation imaging by comparison with pulmonary function test (PFT) measurements and single-photon emission CT (SPECT) ventilation images, which are the clinical references for global and regional lung function, respectively. METHODS AND MATERIALS: In an institutional review board-approved prospective clinical trial, 4D-CT imaging and PFT and/or SPECT ventilation imaging were performed in thoracic cancer patients. Regional ventilation (V4DCT) was calculated by deformable image registration of 4D-CT images and quantitative analysis for regional volume change. V4DCT defect parameters were compared with the PFT measurements (forced expiratory volume in 1 second (FEV1; % predicted) and FEV1/forced vital capacity (FVC; %). V4DCT was also compared with SPECT ventilation (VSPECT) to (1) test whether V4DCT in VSPECT defect regions is significantly lower than in nondefect regions by using the 2-tailed t test; (2) to quantify the spatial overlap between V4DCT and VSPECT defect regions with Dice similarity coefficient (DSC); and (3) to test ventral-to-dorsal gradients by using the 2-tailed t test. RESULTS: Of 21 patients enrolled in the study, 18 patients for whom 4D-CT and either PFT or SPECT were acquired were included in the analysis. V4DCT defect parameters were found to have significant, moderate correlations with PFT measurements. For example, V4DCT(HU) defect volume increased significantly with decreasing FEV1/FVC (R=-0.65, P<.01). V4DCT in VSPECT defect regions was significantly lower than in nondefect regions (mean V4DCT(HU) 0.049 vs 0.076, P<.01). The average DSCs for the spatial overlap with SPECT ventilation defect regions were only moderate (V4DCT(HU)0.39 ± 0.11). Furthermore, ventral-to-dorsal gradients of V4DCT were strong (V4DCT(HU) R(2) = 0.69, P=.08), which was similar to VSPECT (R(2) = 0.96, P<.01). CONCLUSIONS: An 18-patient study demonstrated significant correlations between 4D-CT ventilation and PFT measurements as well as SPECT ventilation, providing evidence toward the validation of 4D-CT ventilation imaging.

Feasibility and potential utility of multicomponent exhaled breath analysis for predicting development of radiation pneumonitis after stereotactic ablative radiotherapy.

INTRODUCTION: In this prospective pilot study, we evaluated the feasibility and potential utility of measuring multiple exhaled gases as biomarkers of radiation pneumonitis (RP) in patients receiving stereotactic ablative radiotherapy (SABR) for lung tumors. METHODS: Breath analysis was performed for 26 patients receiving SABR for lung tumors. Concentrations of exhaled nitric oxide (eNO), carbon monoxide (eCO), nitrous oxide (eN2O), and carbon dioxide (eCO2) were measured before and immediately after each fraction using real-time, infrared laser spectroscopy. RP development (CTCAE grade ≥2) was correlated with baseline gas concentrations, acute changes in gas concentrations after each SABR fraction, and dosimetric parameters. RESULTS: Exhaled breath analysis was successfully completed in 77% of patients. Five of 20 evaluable patients developed RP at a mean of 5.4 months after SABR. Acute changes in eNO and eCO concentrations, defined as percent changes between each pre-fraction and post-fraction measurement, were significantly smaller in RP versus non-RP cases (p = 0.022 and 0.015, respectively). In an exploratory analysis, a combined predictor of baseline eNO greater than 24 parts per billion and acute decrease in eCO less than 5.5% strongly correlated with RP incidence (p =0.0099). Neither eN2O nor eCO2 concentrations were significantly associated with RP development. Although generally higher in patients destined to develop RP, dosimetric parameters were not significantly associated with RP development. CONCLUSIONS: The majority of SABR patients in this pilot study were able to complete exhaled breath analysis. Baseline concentrations and acute changes in concentrations of exhaled breath components were associated with RP development after SABR. If our findings are validated, exhaled breath analysis may become a useful approach for noninvasive identification of patients at highest risk for developing RP after SABR.

An ultrasensitive method for quantitating circulating tumor DNA with broad patient coverage.

Circulating tumor DNA (ctDNA) is a promising biomarker for noninvasive assessment of cancer burden, but existing ctDNA detection methods have insufficient sensitivity or patient coverage for broad clinical applicability. Here we introduce cancer personalized profiling by deep sequencing (CAPP-Seq), an economical and ultrasensitive method for quantifying ctDNA. We implemented CAPP-Seq for non-small-cell lung cancer (NSCLC) with a design covering multiple classes of somatic alterations that identified mutations in >95% of tumors. We detected ctDNA in 100% of patients with stage II-IV NSCLC and in 50% of patients with stage I, with 96% specificity for mutant allele fractions down to ∼0.02%. Levels of ctDNA were highly correlated with tumor volume and distinguished between residual disease and treatment-related imaging changes, and measurement of ctDNA levels allowed for earlier response assessment than radiographic approaches. Finally, we evaluated biopsy-free tumor screening and genotyping with CAPP-Seq. We envision that CAPP-Seq could be routinely applied clinically to detect and monitor diverse malignancies, thus facilitating personalized cancer therapy.

4D CT lung ventilation images are affected by the 4D CT sorting method.

PURPOSE: Four-dimensional (4D) computed tomography (CT) ventilation imaging is a novel promising technique for lung functional imaging. The current standard 4D CT technique using phase-based sorting frequently results in artifacts, which may deteriorate the accuracy of ventilation imaging. The purpose of this study was to quantify the variability of 4D CT ventilation imaging due to 4D CT sorting. METHODS: 4D CT image sets from nine lung cancer patients were each sorted by the phase-based method and anatomic similarity-based method, designed to reduce artifacts, with corresponding ventilation images created for each method. Artifacts in the resulting 4D CT images were quantified with the artifact score which was defined based on the difference between the normalized cross correlation for CT slices within a CT data segment and that for CT slices bordering the interface between adjacent CT data segments. The ventilation variation was quantified using voxel-based Spearman rank correlation coefficients for all lung voxels, and Dice similarity coefficients (DSC) for the spatial overlap of low-functional lung volumes. Furthermore, the correlations with matching single-photon emission CT (SPECT) ventilation images (assumed ground truth) were evaluated for three patients to investigate which sorting method provides higher physiologic accuracy. RESULTS: Anatomic similarity-based sorting reduced 4D CT artifacts compared to phase-based sorting (artifact score, 0.45 ± 0.14 vs 0.58 ± 0.24, p = 0.10 at peak-exhale; 0.63 ± 0.19 vs 0.71 ± 0.31, p = 0.25 at peak-inhale). The voxel-based correlation between the two ventilation images was 0.69 ± 0.26 on average, ranging from 0.03 to 0.85. The DSC was 0.71 ± 0.13 on average. Anatomic similarity-based sorting yielded significantly fewer lung voxels with paradoxical negative ventilation values than phase-based sorting (5.0 ± 2.6% vs 9.7 ± 8.4%, p = 0.05), and improved the correlation with SPECT ventilation regionally. CONCLUSIONS: The variability of 4D CT ventilation imaging due to 4D CT sorting was moderate overall and substantial in some cases, suggesting that 4D CT artifacts are an important source of variations in 4D CT ventilation imaging. Reduction of 4D CT artifacts provided more physiologically convincing and accurate ventilation estimates. Further studies are needed to confirm this result.

Safety of 90Y radioembolization in patients who have undergone previous external beam radiation therapy.

PURPOSE: Previous external beam radiation therapy (EBRT) is theoretically contraindicated for yttrium-90 ((90)Y) radioembolization (RE) because the liver has a lifetime tolerance to radiation before becoming vulnerable to radiation-induced liver disease. We analyzed the safety of RE as salvage treatment in patients who had previously undergone EBRT. METHODS AND MATERIALS: Between June 2004 and December 2010, a total of 31 patients who had previously undergone EBRT were treated with RE. Three-dimensional treatment planning with dose-volume histogram (DVH) analysis of the liver was used to calculate the EBRT liver dose. Liver-related toxicities including RE-induced liver disease (REILD) were reviewed and classified according to Common Terminology Criteria for Adverse Events version 4.02. RESULTS: The mean EBRT and RE liver doses were 4.40 Gy (range, 0-23.13 Gy) and 57.9 Gy (range, 27.0-125.9 Gy), respectively. Patients who experienced hepatotoxicity (≥grade2; n=12) had higher EBRT mean liver doses (7.96 ± 8.55 Gy vs 1.62 ± 3.39 Gy; P=.037), the only independent predictor in multivariate analysis. DVH analysis showed that the fraction of liver exposed to ≥30 Gy (V30) was the strongest predictor of hepatotoxicity (10.14% ± 12.75% vs 0.84% ± 3.24%; P=.006). All patients with V30 >13% experienced hepatotoxicity. Fatal REILD (n=2) occurred at the 2 highest EBRT mean liver doses (20.9 Gy and 23.1 Gy) but also at the highest cumulative liver doses (91.8 Gy and 149 Gy). CONCLUSIONS: Prior exposure of the liver to EBRT may lead to increased liver toxicity after RE treatment, depending on fractional liver exposure and dose level. The V30 was the strongest predictor of toxicity. RE appears to be safe for the treatment of hepatic malignancies only in patients who have had limited hepatic exposure to prior EBRT.

Migration of implanted markers for image-guided lung tumor stereotactic ablative radiotherapy.

The purpose of this study was to quantify postimplantation migration of percutaneously implanted cylindrical gold seeds ("seeds") and platinum endovascular embolization coils ("coils") for tumor tracking in pulmonary stereotactic ablative radiotherapy (SABR). We retrospectively analyzed the migration of markers in 32 consecutive patients with computed tomography scans postimplantation and at simulation. We implanted 147 markers (59 seeds, 88 coils) in or around 34 pulmonary tumors over 32 procedures, with one lesion implanted twice. Marker coordinates were rigidly aligned by minimizing fiducial registration error (FRE), the root mean square of the differences in marker locations for each tumor between scans. To also evaluate whether single markers were responsible for most migration, we aligned with and without the outlier causing the largest FRE increase per tumor. We applied the resultant transformation to all markers. We evaluated migration of individual markers and FRE of each group. Median scan interval was 8 days. Median individual marker migration was 1.28 mm (interquartile range [IQR] 0.78-2.63 mm). Median lesion FRE was 1.56 mm (IQR 0.92-2.95 mm). Outlier identification yielded 1.03 mm median migration (IQR 0.52-2.21 mm) and 1.97 mm median FRE (IQR 1.44-4.32 mm). Outliers caused a mean and median shift in the centroid of 1.22 and 0.80 mm (95th percentile 2.52 mm). Seeds and coils had no statistically significant difference. Univariate analysis suggested no correlation of migration with the number of markers, contact with the chest wall, or time elapsed. Marker migration between implantation and simulation is limited and unlikely to cause geometric miss during tracking.

Dosimetric analysis of organs at risk during expiratory gating in stereotactic body radiation therapy for pancreatic cancer.

PURPOSE: To determine how the respiratory phase impacts dose to normal organs during stereotactic body radiation therapy (SBRT) for pancreatic cancer. METHODS AND MATERIALS: Eighteen consecutive patients with locally advanced, unresectable pancreatic adenocarcinoma treated with SBRT were included in this study. On the treatment planning 4-dimensional computed tomography (CT) scan, the planning target volume (PTV), defined as the gross tumor volume plus 3-mm margin, the duodenum, and the stomach were contoured on the end-expiration (CTexp) and end-inspiration (CTinsp) phases for each patient. A separate treatment plan was constructed for both phases with the dose prescription of 33 Gy in 5 fractions with 95% coverage of the PTV by the 100% isodose line. The dose-volume histogram (DVH) endpoints, volume of duodenum that received 20 Gy (V20), V25, and V30 and maximum dose to 5 cc of contoured organ (D5cc), D1cc, and D0.1cc, were evaluated. RESULTS: Dosimetric parameters for the duodenum, including V25, V30, D1cc, and D0.1cc improved by planning on the CTexp compared to those on the CTinsp. There was a statistically significant overlap of the PTV with the duodenum but not the stomach during the CTinsp compared to the CTexp (0.38 ± 0.17 cc vs 0.01 ± 0.01 cc, P=.048). A larger expansion of the PTV, in accordance with a Danish phase 2 trial, showed even more overlapping volume of duodenum on the CTinsp compared to that on the CTexp (5.5 ± 0.9 cc vs 3.0 ± 0.8 cc, P=.0003) but no statistical difference for any stomach dosimetric DVH parameter. CONCLUSIONS: Dose to the duodenum was higher when treating on the inspiratory than on the expiratory phase. These data suggest that expiratory gating may be preferable to inspiratory breath-hold and free breathing strategies for minimizing risk of toxicity.

Clinical impact of dose overestimation by effective path length calculation in stereotactic ablative radiation therapy of lung tumors.

PURPOSE: To determine the clinical impact of calculated dose differences between effective path length (EPL) and Monte Carlo (MC) algorithms in stereotactic ablative radiation therapy (SABR) of lung tumors. METHODS AND MATERIALS: We retrospectively analyzed the treatment plans and clinical outcomes of 77 consecutive patients treated with SABR for 82 lung tumors between 2003 and 2009 at our institution. Sixty treatments were originally planned using EPL, and 22 using MC. All plans were recalculated for the same beam specifications using MC and EPL, respectively. The doses covering 95%, 50%, and 5% (D95, D50, D5, respectively) of the target volumes were compared between EPL and MC (assumed to be the actual delivered dose), both as physical dose and biologically effective dose. Time to local recurrence was correlated with dose by Cox regression analysis. The relationship between tumor control probability (TCP) and biologically effective dose was determined via logistic regression and used to estimate the TCP decrements due to prescribing by EPL calculations. RESULTS: EPL overestimated dose compared with MC in all tumor dose-volume histogram parameters in all plans. The difference was >10% of the MC D95 to the planning target volume and gross tumor volume in 60 of 82 (73%) and 52 of 82 plans (63%), respectively. Local recurrence occurred in 13 of 82 tumors. Controlling for gross tumor volume, higher physical and biologically effective planning target volume D95 correlated significantly with local control (P = .007 and P = .045, respectively). Compared with MC, prescribing based on EPL translated to a median TCP decrement of 4.3% (range, 1.2%-37%) and a >5% decrement in 46% of tumors. CONCLUSIONS: Clinical follow-up for local lung tumor control in a sizable cohort of patients treated with SABR demonstrates that EPL overestimates dose by amounts that substantially decrease TCP in a large proportion. EPL algorithms should be avoided for lung tumor SABR.

Results from a single institution phase II trial of concurrent docetaxel/carboplatin/radiotherapy followed by surgical resection and consolidation docetaxel/carboplatin in stage III non-small-cell lung cancer.

BACKGROUND: The optimal treatment of locally advanced non-small-cell lung cancer (NSCLC) remains controversial. We hypothesized that using a trimodality approach in selected patients with stage IIIA/IIIB disease would be both feasible and efficacious with reasonable toxicity. PATIENTS/METHODS: We enrolled 13 patients with resectable stage III NSCLC on a prospective phase II trial of trimodality therapy. Induction treatment consisted of weekly docetaxel 20 mg/m(2) and weekly carboplatin at an area under curve (AUC) of 2 concurrent with 45 Gy thoracic radiotherapy. Resection was performed unless felt to be unsafe or if patients had progressive disease. Postoperative consolidation consisted of docetaxel 75 mg/m(2) and carboplatin at an AUC of 6 every 3 weeks for 3 cycles with growth factor support. RESULTS: All patients responded to induction chemoradiotherapy as measured by total gross tumor volume reductions of 43% on average (range, 27%-64%). Twelve patients underwent resection of the tumor and involved nodes, yielding a resectability rate of 92%. The primary endpoint of 2-year overall survival (OS) was 72% (95% confidence interval [CI], 36%-90%), and 2-year progression-free survival (PFS) was 36% (95% CI, 9%-64%). The maximal toxicity observed per patient was grade II in 5 patients (38%); grade III in 7 patients (54%); grade IV in 1 patient (8%); and grade V in none. CONCLUSION: This trimodality approach resulted in promising outcomes with reasonable toxicity in carefully selected patients with stage III NSCLC at a single institution.