Using machine learning to predict risk of incident opioid use disorder among fee-for-service Medicare beneficiaries: A prognostic study.

OBJECTIVE: To develop and validate a machine-learning algorithm to improve prediction of incident OUD diagnosis among Medicare beneficiaries with ≥1 opioid prescriptions. METHODS: This prognostic study included 361,527 fee-for-service Medicare beneficiaries, without cancer, filling ≥1 opioid prescriptions from 2011-2016. We randomly divided beneficiaries into training, testing, and validation samples. We measured 269 potential predictors including socio-demographics, health status, patterns of opioid use, and provider-level and regional-level factors in 3-month periods, starting from three months before initiating opioids until development of OUD, loss of follow-up or end of 2016. The primary outcome was a recorded OUD diagnosis or initiating methadone or buprenorphine for OUD as proxy of incident OUD. We applied elastic net, random forests, gradient boosting machine, and deep neural network to predict OUD in the subsequent three months. We assessed prediction performance using C-statistics and other metrics (e.g., number needed to evaluate to identify an individual with OUD [NNE]). Beneficiaries were stratified into subgroups by risk-score decile. RESULTS: The training (n = 120,474), testing (n = 120,556), and validation (n = 120,497) samples had similar characteristics (age ≥65 years = 81.1%; female = 61.3%; white = 83.5%; with disability eligibility = 25.5%; 1.5% had incident OUD). In the validation sample, the four approaches had similar prediction performances (C-statistic ranged from 0.874 to 0.882); elastic net required the fewest predictors (n = 48). Using the elastic net algorithm, individuals in the top decile of risk (15.8% [n = 19,047] of validation cohort) had a positive predictive value of 0.96%, negative predictive value of 99.7%, and NNE of 104. Nearly 70% of individuals with incident OUD were in the top two deciles (n = 37,078), having highest incident OUD (36 to 301 per 10,000 beneficiaries). Individuals in the bottom eight deciles (n = 83,419) had minimal incident OUD (3 to 28 per 10,000). CONCLUSIONS: Machine-learning algorithms improve risk prediction and risk stratification of incident OUD in Medicare beneficiaries.

Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions.

Importance: Current approaches to identifying individuals at high risk for opioid overdose target many patients who are not truly at high risk. Objective: To develop and validate a machine-learning algorithm to predict opioid overdose risk among Medicare beneficiaries with at least 1 opioid prescription. Design, Setting, and Participants: A prognostic study was conducted between September 1, 2017, and December 31, 2018. Participants (n = 560 057) included fee-for-service Medicare beneficiaries without cancer who filled 1 or more opioid prescriptions from January 1, 2011, to December 31, 2015. Beneficiaries were randomly and equally divided into training, testing, and validation samples. Exposures: Potential predictors (n = 268), including sociodemographics, health status, patterns of opioid use, and practitioner-level and regional-level factors, were measured in 3-month windows, starting 3 months before initiating opioids until loss of follow-up or the end of observation. Main Outcomes and Measures: Opioid overdose episodes from inpatient and emergency department claims were identified. Multivariate logistic regression (MLR), least absolute shrinkage and selection operator-type regression (LASSO), random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN) were applied to predict overdose risk in the subsequent 3 months after initiation of treatment with prescription opioids. Prediction performance was assessed using the C statistic and other metrics (eg, sensitivity, specificity, and number needed to evaluate [NNE] to identify one overdose). The Youden index was used to identify the optimized threshold of predicted score that balanced sensitivity and specificity. Results: Beneficiaries in the training (n = 186 686), testing (n = 186 685), and validation (n = 186 686) samples had similar characteristics (mean [SD] age of 68.0 [14.5] years, and approximately 63% were female, 82% were white, 35% had disabilities, 41% were dual eligible, and 0.60% had at least 1 overdose episode). In the validation sample, the DNN (C statistic = 0.91; 95% CI, 0.88-0.93) and GBM (C statistic = 0.90; 95% CI, 0.87-0.94) algorithms outperformed the LASSO (C statistic = 0.84; 95% CI, 0.80-0.89), RF (C statistic = 0.80; 95% CI, 0.75-0.84), and MLR (C statistic = 0.75; 95% CI, 0.69-0.80) methods for predicting opioid overdose. At the optimized sensitivity and specificity, DNN had a sensitivity of 92.3%, specificity of 75.7%, NNE of 542, positive predictive value of 0.18%, and negative predictive value of 99.9%. The DNN classified patients into low-risk (76.2% [142 180] of the cohort), medium-risk (18.6% [34 579] of the cohort), and high-risk (5.2% [9747] of the cohort) subgroups, with only 1 in 10 000 in the low-risk subgroup having an overdose episode. More than 90% of overdose episodes occurred in the high-risk and medium-risk subgroups, although positive predictive values were low, given the rare overdose outcome. Conclusions and Relevance: Machine-learning algorithms appear to perform well for risk prediction and stratification of opioid overdose, especially in identifying low-risk subgroups that have minimal risk of overdose.

On using 3D γ-analysis for IMRT and VMAT pretreatment plan QA.

PURPOSE: To investigate using 3D γ analysis for IMRT and VMAT QA. METHODS: We explored and studied 3D γ-analysis by comparing TPS computed and EPID back-projection reconstructed doses in patient's CT images. Two 3D γ quantities, γ(PTV) and γ(10), were proposed and studied for evaluating the QA results, and compared to 2D γ (MapCheck composite: γ(MC)). RESULTS: It was found that when 3%(global)/3 mm criteria was used, all IMRT and 90% of VMAT plans passed QA with a γ pass rate ≥90%. A significant statistical correlation was observed between 3D and 2D γ-analysis results for IMRT QA if γ(10) and γ(MC) are concerned, but no significant relation is found between γ(PTV) and γ(MC). CONCLUSIONS: 3D γ analysis based on EPID dose back-projection may provide a feasible tool for IMRT and VMAT pretreatment plan QA.

The minimum knowledge base for predicting organ-at-risk dose-volume levels and plan-related complications in IMRT planning.

IMRT treatment planning requires consideration of two competing objectives: achieving the required amount of radiation for the planning target volume and minimizing the amount of radiation delivered to all other tissues. It is important for planners to understand the tradeoff between competing factors so that the time-consuming human interaction loop (plan-evaluate-modify) can be eliminated. Treatment-plan-surface models have been proposed as a decision support tool to aid treatment planners and clinicians in choosing between rival treatment plans in a multi-plan environment. In this paper, an empirical approach is introduced to determine the minimum number of treatment plans (minimum knowledge base) required to build accurate representations of the IMRT plan surface in order to predict organ-at-risk (OAR) dose-volume (DV) levels and complications as a function of input DV constraint settings corresponding to all involved OARs in the plan. We have tested our approach on five head and neck patients and five whole pelvis/prostate patients. Our results suggest that approximately 30 plans were sufficient to predict DV levels with less than 3% relative error in both head and neck and whole pelvis/prostate cases. In addition, approximately 30-60 plans were sufficient to predict saliva flow rate with less than 2% relative error and to classify rectal bleeding with an accuracy of 90%.

A two-stage sequential linear programming approach to IMRT dose optimization.

The conventional IMRT planning process involves two stages in which the first stage consists of fast but approximate idealized pencil beam dose calculations and dose optimization and the second stage consists of discretization of the intensity maps followed by intensity map segmentation and a more accurate final dose calculation corresponding to physical beam apertures. Consequently, there can be differences between the presumed dose distribution corresponding to pencil beam calculations and optimization and a more accurately computed dose distribution corresponding to beam segments that takes into account collimator-specific effects. IMRT optimization is computationally expensive and has therefore led to the use of heuristic (e.g., simulated annealing and genetic algorithms) approaches that do not encompass a global view of the solution space. We modify the traditional two-stage IMRT optimization process by augmenting the second stage via an accurate Monte Carlo-based kernel-superposition dose calculations corresponding to beam apertures combined with an exact mathematical programming-based sequential optimization approach that uses linear programming (SLP). Our approach was tested on three challenging clinical test cases with multileaf collimator constraints corresponding to two vendors. We compared our approach to the conventional IMRT planning approach, a direct-aperture approach and a segment weight optimization approach. Our results in all three cases indicate that the SLP approach outperformed the other approaches, achieving superior critical structure sparing. Convergence of our approach is also demonstrated. Finally, our approach has also been integrated with a commercial treatment planning system and may be utilized clinically.

Moderate predictive value of demographic and behavioral characteristics for a diagnosis of HPV16-positive and HPV16-negative head and neck cancer.

Patients with HPV-positive and HPV-negative head and neck squamous cell carcinoma (HNSCC) are significantly different with regard to sociodemographic and behavioral characteristics that clinicians may use to assume tumor HPV status. Machine learning methods were used to evaluate the predictive value of patient characteristics and laboratory biomarkers of HPV exposure for a diagnosis of HPV16-positive HNSCC compared to in situ hybridization, the current gold-standard. Models that used a combination of demographic characteristics such as age, tobacco use, gender, and race had only moderate predictive value for tumor HPV status among all patients with HNSCC (positive predictive value [PPV]=75%, negative predictive value [NPV]=68%) or when limited to oropharynx cancer patients (PPV=55%, NPV=65%) and thus included a sizeable number of false positive and false negative predictions. Prediction was not improved by the addition of other demographic or behavioral factors (sexual behavior, income, education) or biomarkers of HPV16 exposure (L1, E6/7 antibodies or DNA in oral exfoliated cells). Patient demographic and behavioral characteristics as well as HPV biomarkers are not an accurate substitute for clinical testing of tumor HPV status.

Modeling plan-related clinical complications using machine learning tools in a multiplan IMRT framework.

PURPOSE: To predict organ-at-risk (OAR) complications as a function of dose-volume (DV) constraint settings without explicit plan computation in a multiplan intensity-modulated radiotherapy (IMRT) framework. METHODS AND MATERIALS: Several plans were generated by varying the DV constraints (input features) on the OARs (multiplan framework), and the DV levels achieved by the OARs in the plans (plan properties) were modeled as a function of the imposed DV constraint settings. OAR complications were then predicted for each of the plans by using the imposed DV constraints alone (features) or in combination with modeled DV levels (plan properties) as input to machine learning (ML) algorithms. These ML approaches were used to model two OAR complications after head-and-neck and prostate IMRT: xerostomia, and Grade 2 rectal bleeding. Two-fold cross-validation was used for model verification and mean errors are reported. RESULTS: Errors for modeling the achieved DV values as a function of constraint settings were 0-6%. In the head-and-neck case, the mean absolute prediction error of the saliva flow rate normalized to the pretreatment saliva flow rate was 0.42% with a 95% confidence interval of (0.41-0.43%). In the prostate case, an average prediction accuracy of 97.04% with a 95% confidence interval of (96.67-97.41%) was achieved for Grade 2 rectal bleeding complications. CONCLUSIONS: ML can be used for predicting OAR complications during treatment planning allowing for alternative DV constraint settings to be assessed within the planning framework.

A nested partitions framework for beam angle optimization in intensity-modulated radiation therapy.

Coupling beam angle optimization with dose optimization in intensity-modulated radiation therapy (IMRT) increases the size and complexity of an already large-scale combinatorial optimization problem. We have developed a novel algorithm, nested partitions (NP), that is capable of finding suitable beam angle sets by guiding the dose optimization process. NP is a metaheuristic that is flexible enough to guide the search of a heuristic or deterministic dose optimization algorithm. The NP method adaptively samples from the entire feasible region, or search space, and coordinates the sampling effort with a systematic partitioning of the feasible region at successive iterations, concentrating the search in promising subsets. We used a 'warm-start' approach by initiating NP with beam angle samples derived from an integer programming (IP) model. In this study, we describe our implementation of the NP framework with a commercial optimization algorithm. We compared the NP framework with equi-spaced beam angle selection, the IP method, greedy heuristic and random sampling heuristic methods. The results of the NP approach were evaluated using two clinical cases (head and neck and whole pelvis) involving the primary tumor and nodal volumes. Our results show that NP produces better quality solutions than the alternative considered methods.

A multiplan treatment-planning framework: a paradigm shift for intensity-modulated radiotherapy.

PURPOSE: To describe a multiplan intensity-modulated radiotherapy (IMRT) planning framework, and to describe a decision support system (DSS) for ranking multiple plans and modeling the planning surface. METHODS AND MATERIALS: One hundred twenty-five plans were generated sequentially for a head-and-neck case and a pelvic case by varying the dose-volume constraints on each of the organs at risk (OARs). A DSS was used to rank plans according to dose-volume histogram (DVH) values, as well as equivalent uniform dose (EUD) values. Two methods for ranking treatment plans were evaluated: composite criteria and pre-emptive selection. The planning surface determined by the results was modeled using quadratic functions. RESULTS: The DSS provided an easy-to-use interface for the comparison of multiple plan features. Plan ranking resulted in the identification of one to three "optimal" plans. The planning surface models had good predictive capability with respect to both DVH values and EUD values and generally, errors of <6%. Models generated by minimizing the maximum relative error had significantly lower relative errors than models obtained by minimizing the sum of squared errors. Using the quadratic model, plan properties for one OAR were determined as a function of the other OAR constraint settings. The modeled plan surface can then be used to understand the interdependence of competing planning objectives. CONCLUSION: The DSS can be used to aid the planner in the selection of the most desirable plan. The collection of quadratic models constructed from the plan data to predict DVH and EUD values generally showed excellent agreement with the actual plan values.