Adding Patient-Reported Outcomes to the American College of Surgeons National Surgical Quality Improvement Program: Results of the First 33,842 Patients from 65 Hospitals.

OBJECTIVES: To investigate the initial set of patient-reported outcomes (PROs) in the ACS NSQIP and their associations with 30-day surgical outcomes. BACKGROUND: PROs provide important information that can be used to improve routine care and facilitate quality improvement. The ACS conducted a demonstration project to capture PROs into the NSQIP to complement clinical data. METHODS: From 2/2020-3/2023, 65 hospitals collected PROMIS measures assessing global health, pain interference, fatigue, and physical function from patients accrued into the NSQIP. Using multivariable mixed regression, we compared the scores of patients with and without 30-day complications and further analyzed scores exceeding one standard deviation (1-SD) worse than national benchmarks. RESULTS: Overall, 33842 patients completed the PROMIS measures a median 58 days (IQR 47-72) postoperatively. Among patients without complications (n=31210), 33.9% had PRO scores 1-SD worse than national benchmarks. Patients with complications were 1.7-times more likely to report worse PROs (95% CI 1.6-1.8). Patients with complications had lower scores for global physical health (adjusted mean difference [AMD] 2.6, 95% CI 2.2-3.0), lower for global mental health (AMD 1.8, 95% CI 1.4-2.2), higher for pain interference (AMD 2.4, 95% CI 2.0-2.8), higher fatigue (AMD 2.7, 95% CI 2.3-3.1), and lower physical function (AMD 3.2, 95% CI 2.8-3.5). CONCLUSIONS: Postoperative complications negatively affect multiple key dimensions of patients' health-related quality of life. PROs were well below national benchmarks for many patients, even among those without complications. Identifying solutions to improve PROs after surgery thus remains a tremendous quality opportunity.

Patient-Reported Outcome Measures Within a National Multispecialty Surgical Quality Improvement Program.

Importance: Patient-reported outcome measures (PROMs) are increasingly recognized for their ability to promote patient-centered care, but concerted health information technology (HIT)-enabled PROM implementations have yet to be achieved for national surgical quality improvement. Objective: To evaluate the feasibility of collecting PROMs within a national surgical quality improvement program. Design, Setting, and Participants: This was a pragmatic implementation cohort study conducted from February 2020 to March 2023. Hospitals in the US participating in the American College of Surgeons National Surgical Quality Improvement Program and their patients were included in this analysis. Exposures: Strategies to increase PROM collection rates were identified using the Institute for Healthcare Improvement (IHI) Framework for Spread and the Consolidated Framework for Implementation Research and operationalized with the IHI Model for Improvement's Plan-Do-Study-Act (PDSA) cycles. Main Outcomes and Measures: The primary goal was to accrue more than 30 hospitals and achieve collection rates of 30% or greater in the first 3 years. Logistic regression was used to identify hospital-level factors associated with achieving collection rates of 30% or greater and to identify patient-level factors associated with response to PROMs. Results: At project close, 65 hospitals administered PROMs to 130 365 patients (median [IQR] age, 60.1 [46.2-70.0] years; 77 369 female [59.4%]). Fifteen PDSA cycles were conducted to facilitate implementation, primarily targeting the Consolidated Framework for Implementation Research domains of Inner Setting (ie, HIT platform) and Individuals (ie, patients). The target collection rate was exceeded in quarter 3 (2022). Fifty-eight hospitals (89.2%) achieved collection rates of 30% or greater, and 9 (13.8%) achieved collection rates of 50% or greater. The median (IQR) maximum hospital-level collection rate was 40.7% (34.6%-46.7%). The greatest increases in collection rates occurred when both email and short-message service text messaging were used, communications to patients were personalized with their surgeon's and hospital's information, and the number of reminders increased from 2 to 5. No identifiable hospital characteristic was associated with achieving the target collection rate. Patient age and insurance status contributed to nonresponse. Conclusions and Relevance: Results of this cohort study suggest that the large-scale electronic collection of PROMs into a national multispecialty surgical registry was feasible. Findings suggest that HIT platform functionality and earning patient trust were the keys to success; although, iterative opportunities to increase collection rates and address nonresponse remain. Future work to drive continuous surgical quality improvement with PROMs are ongoing.

NSQIP Quality Benchmarking and Evaluation of Potential Bias Associated with Higher- or Lower-Risk Operation Case Mix.

BACKGROUND: To ensure validity and acceptance of NSQIP risk-adjusted benchmarking, it is important that adjustments adequately control for hospitals that vary in their proportions of lower- or higher-risk operations (combined risk for procedure and patient). This issue was addressed in separate empirical and simulation studies. STUDY DESIGN: For the empirical study, potential miscalibration bias favoring hospitals that do lower-risk operations or disfavoring hospitals that do higher-risk operations was evaluated for 14 modeled outcomes using NSQIP data. A determination was also made as to whether there was a relationship between mean hospital operation risk and benchmarking results (log odds ratio). In the simulation study of the same 14 outcomes, hospital benchmarked performance was evaluated when sampled cases were reconstituted to include either a larger proportion of lower-risk operations or a larger proportion of higher-risk operations. RESULTS: Miscalibration favoring either lower- or higher-risk operations was absent, as were important associations between operative risk and hospital log odds ratios (most model R 2 less than 0.01). In the simulation, there were no substantial changes in log odds ratios when greater percentages of either lower- or higher-risk operations were included in a hospital's sample (nonsignificant p values and effect sizes less than 0.1). CONCLUSIONS: These results should enhance NSQIP participants' confidence in the adequacy of NSQIP patient and procedure risk-adjustment methods. NSQIP participants may rely on benchmarking findings, and implement quality improvement efforts based on them, without concern that they are biased by a preponderance of lower or higher risk operations.