Validity of the ACS NSQIP surgical risk calculator as a tool to predict postoperative outcomes in subacute orthopedic trauma diagnoses.

ABSTRACT

Purpose: This retrospective study aimed to validate the ACS NSQIP Surgical Risk Calculator (SCR) to predict 30-day postoperative outcomes in patients with one of the following subacute orthopedic trauma diagnoses; multiple rib fractures, pelvic ring/acetabular fracture, or unilateral femoral fracture. Methods: Data of patients with these diagnoses treated between January 1, 2015 and September 19, 2020 were extracted from the patients' medical files. Diagnostic performance, discrimination, calibration, and accuracy of the ACS NSQIP SRC to predict specific outcomes developing within 30 days after surgery was determined. Results: The total cohort of the three diagnoses consisted of 435 patients. ACS NSQIP SRC underestimated the risk for serious complications, especially in patients with multiple rib fractures (8.3% predicted vs 17.2% observed) or pelvic ring/acetabular fracture (6.1% vs 19.8%). Underestimation was more pronounced for the composite outcome 'any complication'. Sensitivity ranged from 16.7% to 100% and specificity from 41.1% to 97.1%. Specificity exceeded sensitivity for pelvic ring/acetabular and femoral fractures. Discrimination was good for predicting death (femoral fracture), fair for readmission (femoral fracture), serious complication (multiple rib fractures), and any complication (multiple rib fractures), but poor in all other outcomes and diagnoses. Calibration and accuracy were adequate for all three diagnoses (p-value for Hosmer-Lemeshow test >0.05 and Brier scores <0.25). Conclusion: Performance of the ACS NSQIP SRC in the studied cohort was variable for all three diagnoses. Although it underestimated the risk of most outcomes, calibration and accuracy seemed generally adequate. For most outcomes, adequate diagnostic performance and discrimination could not be confirmed.