In Search of the Truth: Choice of Ground-Truth for Predictive Modeling of Trauma Team Activation in Pediatric Trauma.

BACKGROUND: Assigning trauma team activation levels for trauma patients is a classification task that machine learning models can help optimize. However, performance is dependent upon the "ground-truth" labels used for training. Our purpose was to investigate two ground-truths, the Cribari matrix and the Need for Trauma Intervention (NFTI), for labeling training data. STUDY DESIGN: Data was retrospectively collected from the institutional trauma registry and electronic medical record, including all pediatric patients (age <18 y) who triggered a trauma team activation (1/2014 - 12/2021). Three ground-truths were used to label training data: 1) Cribari (Injury Severity Score >15 = full activation), 2) NFTI (positive for any of 6 criteria = full activation), and 3) the union of Cribari+NFTI (either positive = full activation). RESULTS: Of 1,366 patients triaged by trained staff, 143 (10.47%) were considered under-triaged using Cribari, 210 (15.37%) using NFTI, and 273 (19.99%) using Cribari+NFTI. NFTI and Cribari+NFTI were more sensitive to under-triage in patients with penetrating mechanisms of injury (p = 0.006), specifically stab wounds (p = 0.014), compared to Cribari, but Cribari indicated over-triage in more patients who required prehospital airway management (p < 0.001), CPR (p = 0.017), and who had mean lower GCS scores on presentation (p < 0.001). The mortality rate was higher in the Cribari over-triage group (7.14%, n = 9) compared to NFTI and Cribari+NFTI (0.00%, n = 0, p = 0.005). CONCLUSION: To prioritize patient safety, Cribari+NFTI appears best for training a machine learning algorithm to predict trauma team activation level.

Machine Learning Improves the Accuracy of Trauma Team Activation Level Assignments in Pediatric Patients.

BACKGROUND: The assignment of trauma team activation levels can be conceptualized as a classification task. Machine learning models can be used to optimize classification predictions. Our purpose was to demonstrate proof-of-concept for a machine learning tool for predicting trauma team activation levels in pediatric patients with traumatic injuries. METHODS: Following IRB approval, we retrospectively collected data from the institutional trauma registry and electronic medical record at our Pediatric Trauma Center for all patients (age <18 y) who triggered a trauma team activation (1/2014-12/2021), including: demographics, mechanisms of injury, comorbidities, pre-hospital interventions, numeric variables, and the six "Need for Trauma Intervention (NFTI)" criteria. Three machine learning models (Logistic Regression, Random Forest, Support Vector Machine) were tested 1000 times in separate trials using the union of the Cribari and NFTI metrics as ground-truth (Injury Severity Score >15 or positive for any of 6 NFTI criteria = full activation). Model performance was quantified and compared to emergency department (ED) staff. RESULTS: ED staff had 75% accuracy, an area under the curve (AUC) of 0.73 ± 0.04, and an F1 score of 0.49. The best performing of all machine learning models, the support vector machine, had 80% accuracy, AUC 0.81 ± 4.1e-5, F1 Score 0.80, with less variance compared to other models and ED staff. CONCLUSIONS: All machine learning models outperformed ED staff in all performance metrics. These results suggest that data-driven methods can optimize trauma team activations in the ED, with potential improvements in both patient safety and hospital resource utilization. TYPE OF STUDY: Economic/Decision Analysis or Modeling Studies. LEVEL OF EVIDENCE: II.