Attitudes about use of preoperative risk assessment tools: a survey of surgeons and surgical residents in an academic health system.

BACKGROUND: Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk. METHODS: Surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey via REDCap (Research Electronic Data Capture) between 8/2019-6/2020. Reminder emails were sent once weekly for three weeks. All completed surveys by surgical residents and attendings were included; incomplete surveys were excluded. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher's exact test and unpaired t-tests comparing responses by surgical attendings vs. residents. RESULTS: A total of 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons were emailed the survey. Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy. CONCLUSIONS: A small percentage of surgeons use surgical risk calculators as part of their routine practice. Time, inaccessibility, and trust in accuracy were the most significant barriers to use.

Associations between preoperative risks of postoperative complications: Results of an analysis of 4.8 Million ACS-NSQIP patients.

BACKGROUND: Surgical Risk Preoperative Assessment System (SURPAS) estimates patient's preoperative risk of 12 postoperative complications, yet little is known about associations between these probabilities- We sought to examine relationships between predicted probabilities. METHODS: Risk of 12 postoperative complications was calculated using SURPAS and the 2012-2018 ACS-NSQIP database. Pearson correlation coefficients (r) were computed to examine relationships between predicted outcomes. "High-risk" was predicted risk in the 10th decile. RESULTS: 4,777,267 patients were included. 71.1% were not high risk, 10.7% were high risk for 1, and 18.2% were high risk for ≥2 complications. High mortality risk was associated with high risk for pulmonary (r = 0.94), cardiac (r = 0.98), renal (r = 0.93), and stroke (0.96) complications. Patients high-risk for ≥2 complications had the most comorbidities and actual adverse outcomes. CONCLUSIONS: High preoperative risk for certain postoperative complications had strong correlations. 18.2% of patients were high-risk for ≥2 complications and could be targeted for risk reduction interventions.

Using the Surgical Risk Preoperative Assessment System to Define the "High Risk" Surgical Patient.

BACKGROUND: Defining a "high risk" surgical population remains challenging. Using the Surgical Risk Preoperative Assessment System (SURPAS), we sought to define "high risk" groups for adverse postoperative outcomes. MATERIALS AND METHODS: We retrospectively analyzed the 2009-2018 American College of Surgeons National Surgical Quality Improvement Program database. SURPAS calculated probabilities of 12 postoperative adverse events. The Hosmer Lemeshow graphs of deciles of risk and maximum Youden index were compared to define "high risk." RESULTS: Hosmer-Lemeshow plots suggested the "high risk" patient could be defined by the 10th decile of risk. Maximum Youden index found lower cutoff points for defining "high risk" patients and included more patients with events. This resulted in more patients classified as "high risk" and higher number needed to treat to prevent one complication. Some specialties (thoracic, vascular, general) had more "high risk" patients, while others (otolaryngology, plastic) had lower proportions. CONCLUSIONS: SURPAS can define the "high risk" surgical population that may benefit from risk-mitigating interventions.

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons vs a Standardized Surgical Risk Prediction Tool.

Considerable variability exists between surgeons' assessments of a patient's individual preoperative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes vs a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs 5.1%, P ≤ 0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs 69.8%, P < 0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs 44.3%, P < 0.001). C-indices for SURPAS vs surgeons were 0.84 vs 0.76 (P = 0.3) for morbidity and 0.98 vs 0.85 (P = 0.001) for mortality. Brier scores for SURPAS vs surgeons were 0.1579 vs 0.1986 for morbidity (P = 0.03) and 0.0409 vs 0.0543 for mortality (P = 0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC = 0.654) and mortality (ICC = 0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.

The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery.

BACKGROUND: The Surgical Risk Preoperative Assessment System (SURPAS) uses eight variables to accurately predict postoperative complications but has not been sufficiently studied in emergency surgery. We evaluated SURPAS in emergency surgery, comparing it to the Emergency Surgery Score (ESS). METHODS: SURPAS and ESS estimates of 30-day mortality and overall morbidity were calculated for emergency operations in the 2009-2018 ACS-NSQIP database and compared using observed-to-expected plots and rates, c-indices, and Brier scores. Cases with incomplete data were excluded. RESULTS: In 205,318 emergency patients, SURPAS underestimated (8.1%; 35.9%) while ESS overestimated (10.1%; 43.8%) observed mortality and morbidity (8.9%; 38.8%). Each showed good calibration on observed-to-expected plots. SURPAS had better c-indices (0.855 vs 0.848 mortality; 0.802 vs 0.755 morbidity), while the Brier score was better for ESS for mortality (0.0666 vs. 0.0684) and for SURPAS for morbidity (0.1772 vs. 0.1950). CONCLUSIONS: SURPAS accurately predicted mortality and morbidity in emergency surgery using eight predictor variables.

The value of the "Surgical Risk Preoperative Assessment System" (SURPAS) in preoperative consultation for elective surgery: a pilot study.

BACKGROUND: Risk assessment is essential to informed decision making in surgery. Preoperative use of the Surgical Risk Preoperative Assessment System (SURPAS) providing individualized risk assessment, may enhance informed consent. We assessed patient and provider perceptions of SURPAS as a risk assessment tool. METHODS: A convergent mixed-methods study assessed SURPAS's trial implementation, concurrently collecting quantitative and qualitative data, separately analyzing it, and integrating the results. Patients and providers were surveyed and interviewed on their opinion of how SURPAS impacted the preoperative encounter. Relationships between patient risk and patient and provider assessment of SURPAS were examined. RESULTS: A total of 197 patients were provided their SURPAS postoperative risk estimates in nine surgeon's clinics. Of the total patients, 98.8% reported they understood their surgical risks very or quite well after exposure to SURPAS; 92.7% reported SURPAS was very helpful or helpful. Providers shared that 83.4% of the time they reported SURPAS was very or somewhat helpful; 44.7% of the time the providers reported it changed their interaction with the patient and this change was beneficial 94.3% of the time. As patient risk increased, providers reported that SURPAS was increasingly helpful (p < 0.0001). CONCLUSIONS: Patients and providers reported the use of SURPAS helpful and informative during the preoperative risk assessment of patients, thus improving the surgical decision making process. Patients thought that SURPAS was helpful regardless of their risk level, whereas providers thought that SURPAS was more helpful in higher risk patients.

Comparison of accuracy of prediction of postoperative mortality and morbidity between a new, parsimonious risk calculator (SURPAS) and the ACS Surgical Risk Calculator.

BACKGROUND: The novel Surgical Risk Preoperative Assessment System (SURPAS) requires entry of five predictor variables (the other three variables of the eight-variable model are automatically obtained from the electronic health record or a table look-up), provides patient risk estimates compared to national averages, is integrated into the electronic health record, and provides a graphical handout of risks for patients. The accuracy of the SURPAS tool was compared to that of the American College of Surgeons Surgical Risk Calculator (ACS-SRC). METHODS: Predicted risk of postoperative mortality and morbidity was calculated using both SURPAS and ACS-SRC for 1,006 randomly selected 2007-2016 ACS National Surgical Quality Improvement Program (NSQIP) patients with known outcomes. C-indexes, Hosmer-Lemeshow graphs, and Brier scores were compared between SURPAS and ACS-SRC. RESULTS: ACS-SRC risk estimates for overall morbidity underestimated risk compared to observed postoperative overall morbidity, particularly for the highest risk patients. SURPAS accurately estimates morbidity risk compared to observed morbidity. CONCLUSIONS: SURPAS risk predictions were more accurate than ACS-SRC's for overall morbidity, particularly for high risk patients. SUMMARY: The accuracy of the SURPAS tool was compared to that of the American College of Surgeons Surgical Risk Calculator (ACS-SRC). SURPAS risk predictions were more accurate than those of the ACS-SRC for overall morbidity, particularly for high risk patients.