A Machine-Learning Algorithm to Predict the Likelihood of Prolonged Opioid Use Following Arthroscopic Hip Surgery.

PURPOSE: To develop a machine-learning algorithm and clinician-friendly tool predicting the likelihood of prolonged opioid use (>90 days) following hip arthroscopy. METHODS: The Military Data Repository was queried for all adult patients undergoing arthroscopic hip surgery between 2012 and 2017. Demographic, health history, and prescription records were extracted for all included patients. Opioid use was divided into preoperative use (30-365 days before surgery), perioperative use (30 days before surgery through 14 days after surgery), postoperative use (14-90 days after surgery), and prolonged postoperative use (90-365 days after surgery). Six machine-learning algorithms (Naïve Bayes, Gradient Boosting Machine, Extreme Gradient Boosting, Random Forest, Elastic Net Regularization, and artificial neural network) were developed. Area under the receiver operating curve and Brier scores were calculated for each model. Decision curve analysis was applied to assess clinical utility. Local-Interpretable Model-Agnostic Explanations were used to demonstrate factor weights within the selected model. RESULTS: A total of 6,760 patients were included, of whom 2,762 (40.9%) filled at least 1 opioid prescription >90 days after surgery. The artificial neural network model showed superior discrimination and calibration with area under the receiver operating curve = 0.71 (95% confidence interval 0.68-0.74) and Brier score = 0.21 (95% confidence interval 0.20-0.22). Postsurgical opioid use, age, and preoperative opioid use had the most influence on model outcome. Lesser factors included the presence of a psychological comorbidity and strong history of a substance use disorder. CONCLUSIONS: The artificial neural network model shows sufficient validity and discrimination for use in clinical practice. The 5 identified factors (age, preoperative opioid use, postoperative opioid use, presence of a mental health comorbidity, and presence of a preoperative substance use disorder) accurately predict the likelihood of prolonged opioid use following hip arthroscopy. LEVEL OF EVIDENCE: III, retrospective comparative prognostic trial.

Can Predictive Modeling Tools Identify Patients at High Risk of Prolonged Opioid Use After ACL Reconstruction?

BACKGROUND: Machine-learning methods such as the Bayesian belief network, random forest, gradient boosting machine, and decision trees have been used to develop decision-support tools in other clinical settings. Opioid abuse is a problem among civilians and military service members, and it is difficult to anticipate which patients are at risk for prolonged opioid use. QUESTIONS/PURPOSES: (1) To build a cross-validated model that predicts risk of prolonged opioid use after a specific orthopaedic procedure (ACL reconstruction), (2) To describe the relationships between prognostic and outcome variables, and (3) To determine the clinical utility of a predictive model using a decision curve analysis (as measured by our predictive system's ability to effectively identify high-risk patients and allow for preventative measures to be taken to ensure a successful procedure process). METHODS: We used the Military Analysis and Reporting Tool (M2) to search the Military Health System Data Repository for all patients undergoing arthroscopically assisted ACL reconstruction (Current Procedure Terminology code 29888) from January 2012 through December 2015 with a minimum of 90 days postoperative follow-up. In total, 10,919 patients met the inclusion criteria, most of whom were young men on active duty. We obtained complete opioid prescription filling histories from the Military Health System Data Repository's pharmacy records. We extracted data including patient demographics, military characteristics, and pharmacy data. A total of 3.3% of the data was missing. To curate and impute all missing variables, we used a random forest algorithm. We shuffled and split the data into 80% training and 20% hold-out sets, balanced by outcome variable (Outcome90Days). Next, the training set was further split into training and validation sets. Each model was built on the training data set, tuned with the validation set as applicable, and finally tested on the separate hold-out dataset. We chose four predictive models to develop, at the end choosing the best-fit model for implementation. Logistic regression, random forest, Bayesian belief network, and gradient boosting machine models were the four chosen models based on type of analysis (classification). Each were trained to estimate the likelihood of prolonged opioid use, defined as any opioid prescription filled more than 90 days after anterior cruciate reconstruction. After this, we tested the models on our holdout set and performed an area under the curve analysis concordance statistic, calculated the Brier score, and performed a decision curve analysis for validation. Then, we chose the method that produced the most suitable analysis results and, consequently, predictive power across the three calculations. Based on the calculations, the gradient boosting machine model was selected for future implementation. We systematically selected features and tuned the gradient boosting machine to produce a working predictive model. We performed area under the curve, Brier, and decision curve analysis calculations for the final model to test its viability and gain an understanding of whether it is possible to predict prolonged opioid use. RESULTS: Four predictive models were successfully developed using gradient boosting machine, logistic regression, Bayesian belief network, and random forest methods. After applying the Boruta algorithm for feature selection based on a 100-tree random forest algorithm, features were narrowed to a final seven features. The most influential features with a positive association with prolonged opioid use are preoperative morphine equivalents (yes), particular pharmacy ordering sites locations, shorter deployment time, and younger age. Those observed to have a negative association with prolonged opioid use are particular pharmacy ordering sites locations, preoperative morphine equivalents (no), longer deployment, race (American Indian or Alaskan native) and rank (junior enlisted).On internal validation, the models showed accuracy for predicting prolonged opioid use with AUC greater than our benchmark cutoff 0.70; random forest were 0.76 (95% confidence interval 0.73 to 0.79), 0.76 (95% CI 0.73 to 0.78), 0.73 (95% CI 0.71 to 0.76), and 0.72 (95% CI 0.69 to 0.75), respectively. Although the results from logistic regression and gradient boosting machines were very similar, only one model can be used in implementation. Based on our calculation of the Brier score, area under the curve, and decision curve analysis, we chose the gradient boosting machine as the final model. After selecting features and tuning the chosen gradient boosting machine, we saw an incremental improvement in our implementation model; the final model is accurate, with a Brier score of 0.10 (95% CI 0.09 to 0.11) and area under the curve of 0.77 (95% CI 0.75 to 0.80). It also shows the best clinical utility in a decision curve analysis. CONCLUSIONS: These scores support our claim that it is possible to predict which patients are at risk of prolonged opioid use, as seen by the appropriate range of hold-out analysis calculations. Current opioid guidelines recommend preoperative identification of at-risk patients, but available tools for this purpose are crude, largely focusing on identifying the presence (but not relative contributions) of various risk factors and screening for depression. The power of this model is that it will permit the development of a true clinical decision-support tool, which risk-stratifies individual patients with a single numerical score that is easily understandable to both patient and surgeon. Probabilistic models provide insight into how clinical factors are conditionally related. Not only will this gradient boosting machine be used to help understand factors contributing to opiate misuse after ACL reconstruction, but also it will allow orthopaedic surgeons to identify at-risk patients before surgery and offer increased support and monitoring to prevent opioid abuse and dependency. LEVEL OF EVIDENCE: Level III, therapeutic study.

Utilization of the Department of Defense Peer-Reviewed Orthopaedic Research Program (PRORP): Combating Musculoskeletal Disease With PRORP.

INTRODUCTION: Established in 2009, the Department of Defense (DoD) Peer-Reviewed Orthopaedic Research Program (PRORP) is an annual funding program for orthopaedic research that seeks to develop evidence for new clinical practice guidelines, procedures, technologies, and drugs. The aim was to help reduce the burden of injury for wounded Service members, Veterans, and civilians and to increase return-to-duty and return-to-work rates. Relative to its burden of disease, musculoskeletal injuries (MSKIs) are one of the most disproportionately underfunded conditions. The focus of the PRORP includes a broad spectrum of MSKI in areas related to unique aspect of combat- and some noncombat-related injuries. The PRORP may serve as an important avenue of research for nonmilitary communities by offering areas of shared interests for the advancement of military and civilian patient cohort MSKI care. The purpose of this study was to provide a descriptive analysis of the DoD PRORP, which is an underrecognized but high value source of research funding for a broad spectrum of both combat- and noncombat-related MSKIs. METHODS: The complete PRORP Funding Portfolio for FY2009-FY2017 was obtained from the Congressionally Directed Medical Research Programs (CDMRP), which includes 255 awarded grants. Information pulled from the CDMRP included awardee descriptors (sex, education level, affiliated institution type, research specialty, and previous award winner [yes/no]) and grant award descriptors (grant amount, year, primary and secondary awarded topics, research type awarded, and mechanism of award). Distribution statistics were broken down by principal investigator specialty, sex, degree, organization type, research type, mechanism, and research topics. Distribution and statistical analysis was applied using R software version 3.6.3. RESULTS: From FY2009 to 2017, $285 million was allocated for 255 PRORP-funded research studies. The seven major orthopaedic subspecialties (foot and ankle, hand, musculoskeletal oncology, pediatrics, spine, sports medicine, and trauma) were represented. Trauma and hand subspecialists received the largest amount of funding, approximately $28 (9.6%) and $22 million (7.1%), respectively. However, only 22 (8.6%) and 26 (10.2%) of the primary investigators were trauma and hand subspecialists, respectively. The primary research categories were diverse with the top five funded PRORP topics being rehabilitation ($53 million), consortia ($39 million), surgery ($37 million), device development ($30 million), and pharmacology ($10 million). DISCUSSION: The CDMRP funding represents an excellent resource for orthopaedic medical research support that includes trauma and nontrauma orthopaedic conditions. This study serves to promote and communicate the missions of the PRORP both within and beyond the DoD to raise awareness and expand access of available funding for orthopaedic focused research. SIGNIFICANCE/CLINICAL RELEVANCE: A likelihood exists that this project will provide sustained and powerful influence on future research by promoting awareness of orthopaedic funding sources. LEVEL OF EVIDENCE: Level III.

Development and comparison of 1-year survival models in patients with primary bone sarcomas: External validation of a Bayesian belief network model and creation and external validation of a new gradient boosting machine model.

BACKGROUND: Bone sarcomas often present late with advanced stage at diagnosis and an according, varying short-term survival. In 2016, Nandra et al. generated a Bayesian belief network model for 1-year survival in patients with bone sarcomas. The purpose of this study is: (1) to externally validate the prior 1-year Bayesian belief network prediction model for survival in patients with bone sarcomas and (2) to develop a gradient boosting machine model using Nandra et al.'s cohort and evaluate whether the gradient boosting machine model outperforms the Bayesian belief network model when externally validated in an independent Danish population cohort. MATERIAL AND METHODS: The training cohort comprised 3493 patients newly diagnosed with bone sarcoma from the institutional prospectively maintained database at the Royal Orthopaedic Hospital, Birmingham, UK. The validation cohort comprised 771 patients with newly diagnosed bone sarcoma included from the Danish Sarcoma Registry during January 1, 2000-June 22, 2016. We performed area under receiver operator characteristic curve analysis, Brier score and decision curve analysis to evaluate the predictive performance of the models. RESULTS: External validation of the Bayesian belief network 1-year prediction model demonstrated an area under receiver operator characteristic curve of 68% (95% confidence interval, 62%-73%). Area under receiver operator characteristic curve of the gradient boosting machine model demonstrated: 75% (95% confidence interval: 70%-80%), overall model performance by the Brier score was 0.09 (95% confidence interval: 0.077-0.11) and decision curve analysis demonstrated a positive net benefit for threshold probabilities above 0.5. External validation of the developed gradient boosting machine model demonstrated an area under receiver operator characteristic curve of 63% (95% confidence interval: 57%-68%), and the Brier score was 0.14 (95% confidence interval: 0.12-0.16). CONCLUSION: External validation of the 1-year Bayesian belief network survival model yielded a poor outcome based on a Danish population cohort validation. We successfully developed a gradient boosting machine 1-year survival model. The gradient boosting machine did not outperform the Bayesian belief network model based on external validation in a Danish population-based cohort.