Improving Clinical Utility of Real-World Prediction Models: Updating Through Recalibration.

ABSTRACT: Bullock, GS, Shanley, E, Thigpen, CA, Arden, NK, Noonan, TK, Kissenberth, MJ, Wyland, DJ, and Collins, GS. Improving clinical utility of real-world prediction models: updating through recalibration. J Strength Cond Res 37(5): 1057-1063, 2023-Prediction models can aid clinicians in identifying at-risk athletes. However, sport and clinical practice patterns continue to change, causing predictive drift and potential suboptimal prediction model performance. Thus, there is a need to temporally recalibrate previously developed baseball arm injury models. The purpose of this study was to perform temporal recalibration on a previously developed injury prediction model and assess model performance in professional baseball pitchers. An arm injury prediction model was developed on data from a prospective cohort from 2009 to 2019 on minor league pitchers. Data for the 2015-2019 seasons were used for temporal recalibration and model performance assessment. Temporal recalibration constituted intercept-only and full model redevelopment. Model performance was investigated by assessing Nagelkerke's R-square, calibration in the large, calibration, and discrimination. Decision curves compared the original model, temporal recalibrated model, and current best evidence-based practice. One hundred seventy-eight pitchers participated in the 2015-2019 seasons with 1.63 arm injuries per 1,000 athlete exposures. The temporal recalibrated intercept model demonstrated the best discrimination (0.81 [95% confidence interval [CI]: 0.73, 0.88]) and R-square (0.32) compared with original model (0.74 [95% CI: 0.69, 0.80]; R-square: 0.32) and the redeveloped model (0.80 [95% CI: 0.73, 0.87]; R-square: 0.30). The temporal recalibrated intercept model demonstrated an improved net benefit of 0.34 compared with current best evidence-based practice. The temporal recalibrated intercept model demonstrated the best model performance and clinical utility. Updating prediction models can account for changes in sport training over time and improve professional baseball arm injury outcomes.

Initial kinematic chain injuries increase hazard of subsequent arm injuries in professional baseball pitchers.

BACKGROUND: Currently, there are few studies that have evaluated the relationship between a lower extremity or trunk injury (kinematic chain) and subsequent arm injury. The purpose of this study was (1) to investigate the relationship between initial kinematic chain (lower extremity or trunk) injury and subsequent arm injury; and (2) to investigate the relationship between initial shoulder or elbow injury and subsequent arm injury. METHODS: A 7-year prospective injury risk study was conducted with Minor League Baseball pitchers. Pitches, pitching appearances, athlete exposures (AEs), and arm injuries (≥1-day time loss) were documented throughout the season. Cox survival analyses with 95% confidence intervals (95% CIs) were performed. Confounders controlled for included age, body mass index, arm dominance, pitching role, previous arm injury, number of pitching appearances, and seasonal pitch load. RESULTS: A total of 297 pitchers participated (total player days = 85,270). Arm injury incidence was 11.4 arm injuries/10,000 AEs, and kinematic chain incidence was 5.2 injuries/10,000 AEs. Pitchers who sustained a kinematic chain injury demonstrated a greater hazard (2.6 [95% CI: 1.2, 5.6], P = .019) of sustaining an arm injury. Pitchers who sustained an initial shoulder injury demonstrated a greater hazard (9.3 [95% CI: 1.1, 83], P = .047) of sustaining a subsequent shoulder or elbow injury compared with pitchers who sustained an initial elbow injury. CONCLUSIONS: Pitchers who sustained an initial lower extremity or trunk injury demonstrated an increased subsequent arm injury hazard compared with pitchers who did not. Pitchers who sustained an initial shoulder injury demonstrated a greater hazard of sustaining a subsequent arm injury compared with pitchers who sustained an initial elbow injury. However, this secondary analysis should be interpreted with caution. Clinicians should monitor risk with workload accumulation, which may be related to pitching compensatory strategies in a fatigued state. Pitchers who sustain a shoulder injury should be evaluated and perform both shoulder and elbow rehabilitation strategies before return to sport.

Machine Learning Does Not Improve Humeral Torsion Prediction Compared to Regression in Baseball Pitchers.

Background: Humeral torsion is an important osseous adaptation in throwing athletes that can contribute to arm injuries. Currently there are no cheap and easy to use clinical tools to measure humeral torsion, inhibiting clinical assessment. Models with low error and "good" calibration slope may be helpful for prediction. Hypothesis/Purpose: To develop prediction models using a range of machine learning methods to predict humeral torsion in professional baseball pitchers and compare these models to a previously developed regression-based prediction model. Study Design: Prospective cohort. Methods: An eleven-year professional baseball cohort was recruited from 2009-2019. Age, arm dominance, injury history, and continent of origin were collected as well as preseason shoulder external and internal rotation, horizontal adduction passive range of motion, and humeral torsion were collected each season. Regression and machine learning models were developed to predict humeral torsion followed by internal validation with 10-fold cross validation. Root mean square error (RMSE), which is reported in degrees (°) and calibration slope (agreement of predicted and actual outcome; best = 1.00) were assessed. Results: Four hundred and seven pitchers (Age: 23.2 +/-2.4 years, body mass index: 25.1 +/-2.3 km/m2, Left-Handed: 17%) participated. Regression model RMSE was 12° and calibration was 1.00 (95% CI: 0.94, 1.06). Random Forest RMSE was 9° and calibration was 1.33 (95% CI: 1.29, 1.37). Gradient boosting machine RMSE was 9° and calibration was 1.09 (95% CI: 1.04, 1.14). Support vector machine RMSE was 10° and calibration was 1.13 (95% CI: 1.08, 1.18). Artificial neural network RMSE was 15° and calibration was 1.03 (95% CI: 0.97, 1.09). Conclusion: This is the first study to show that machine learning models do not improve baseball humeral torsion prediction compared to a traditional regression model. While machine learning models demonstrated improved RMSE compared to the regression, the machine learning models displayed poorer calibration compared to regression. Based on these results it is recommended to use a simple equation from a statistical model which can be quickly and efficiently integrated within a clinical setting. Levels of Evidence: 2.

Hazard of Arm Injury in Professional Starting and Relief Pitchers.

CONTEXT: How different pitching roles affect the risk of arm injury in professional pitchers is currently unclear. OBJECTIVES: To investigate differences between professional baseball starting and relief pitchers in the hazard of (1) arm injury and (2) elbow and shoulder injury. DESIGN: Prospective cohort study. SETTING: Minor League Baseball (MiLB) from 2013 to 2019. PATIENTS OR OTHER PARTICIPANTS: Pitchers in MiLB. MAIN OUTCOME MEASURE(S): Pitchers were followed for the entire MiLB season, and athlete-exposures and injuries were recorded. Risk ratios and risk difference were calculated between starting and relieving MiLB pitchers. A Cox survival analysis was then performed in relation to time to arm injury between starting and relieving MiLB pitchers. Subgroup analyses were conducted for elbow and shoulder injuries. RESULTS: A total of 297 pitchers were included, with 85 270 player-days recorded. The incidence of arm injuries was 11.4 per 10 000 athlete-exposures. Starting pitchers demonstrated a greater risk ratio (1.2 [95% CI = 1.1, 1.3]), risk difference (13.6 [95% CI = 5.6, 21.6]), and hazard of arm injury (2.4 [95% CI = 1.5, 4.0]) than relief pitchers. No differences were observed for the hazard of elbow injury between starting and relief pitchers (1.9; 95% CI = 0.8, 4.2). Starting pitchers had a greater hazard of shoulder injury than relief pitchers (3.8 [95% CI = 2.0, 7.1]). CONCLUSIONS: Starting pitchers displayed a 2.4 times greater hazard of arm injury than relief pitchers. Subgroup analyses indicated that starters exhibited a greater hazard of shoulder injury than relievers, but no differences occurred for the hazard of elbow injury. However, due to the wide CIs, these subgroup analyses should be interpreted with caution. Clinicians may need to consider cumulative exposure and fatigue and how these factors relate to different pitching roles when assessing the risk of pitching arm injury.

Development and internal validation of a humeral torsion prediction model in professional baseball pitchers.

BACKGROUND: Humeral torsion (HT) has been linked to pitching arm injury risk after controlling for shoulder range of motion. Currently measuring HT uses expensive equipment, which inhibits clinical assessment. Developing an HT predictive model can aid clinical baseball arm injury risk examination. Therefore, the purpose of this study was to develop and internally validate an HT prediction model using standard clinical tests and measures in professional baseball pitchers. METHODS: An 11-year (2009-2019) prospective professional baseball cohort was used for this study. Participants were included if they were able to participate in all practices and competitions and were under a Minor League Baseball contract. Preseason shoulder range of motion (external rotation [ER], internal rotation [IR], horizontal adduction [HA]) and HT were collected each season. Player age, arm dominance, arm injury history, and continent of origin were also collected. Examiners were blinded to arm dominance. An a priori power analysis determined that 244 players were needed for accurate prediction models. Missing data was low (<3%); thus, a complete case analysis was performed. Model development followed the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) recommendations. Regression models with restricted cubic splines were performed. Following primary model development, bootstrapping with 2000 iterations were performed to reduce overfitting and assess optimism shrinkage. Prediction model performance was assessed through root mean square error (RMSE), R2, and calibration slope with 95% confidence intervals (CIs). Sensitivity analyses included dominant and nondominant HT. RESULTS: A total of 407 professional pitchers (age: 23.2 [standard deviation 2.4] years, left-handed: 17%; arm history prevalence: 21%) participated. Predictors with the highest influence within the model include IR (0.4, 95% CI 0.3, 0.5; P < .001), ER (-0.3, 95% CI -0.4, -0.2; P < .001), HA (0.3, 95% CI 0.2, 0.4; P < .001), and arm dominance (right-handed: -1.9, 95% CI -3.6, -0.1; P = .034). Final model RMSE was 12, R2 was 0.41, and calibration was 1.00 (95% CI 0.94, 1.06). Sensitivity analyses demonstrated similar model performance. CONCLUSIONS: Every 3° of IR explained 1° of HT. Every 3° of ER explained 1° less of HT, and every 7° of HA explained 1° of HT. Right-handers had 2° less HT. Models demonstrated good predictive performance. This predictive model can be used by clinicians to infer HT using standard clinical test and measures. These data can be used to enhance professional baseball arm injury examination.