Formative evaluation of the development and implementation of the automated surveillance of postoperative infections tool.

BACKGROUND: The gold standard for detecting postoperative complications uses databases like the American College of Surgeons National Surgical Quality Improvement Program, a multi-centered database based on manual chart review. However, their limitations and costs have led many centers to discontinue participation. Novel techniques to detect postoperative complications must be developed and implemented with surgeon involvement, which is paramount to their adoption. We sought to assess surgeons' opinions of a newly developed postoperative complication detection tool, the Automated Surveillance of Postoperative Infections, within the contextual clinical environment. METHODS: This was a multi-site qualitative formative evaluation of surgeon perceptions of the Automated Surveillance of Postoperative Infections. We conducted semi-structured interviews and focus groups with surgeons and presented the Automated Surveillance of Postoperative Infections concept. Important domains and constructs, as categorized by Consolidated Framework for Implementation Research, were identified to support the successful adoption and implementation of the Automated Surveillance of Postoperative Infections. RESULTS: Twenty-four surgeons with 10 surgical subspecialties were interviewed. The following 4 main themes were found: (1) perception of the Automated Surveillance of Postoperative Infections tool-to provide important data that can improve and support clinical outcomes; (2) environment for implementation-description of factors to support or impede implementation; (3) adaptability of the Automated Surveillance of Postoperative Infections-to work with the complexity of surgical cases; and (4) the Automated Surveillance of Postoperative Infections report format and details. CONCLUSIONS: We successfully captured the perspectives and suggestions of surgeons to improve the Automated Surveillance of Postoperative Infections and potential barriers during the initial development phase. Barriers included fear of punitive action from reports and complex surgical cases. Facilitators identified were the need to improve clinical outcomes and organizational support. The results of this formative evaluation will be used to further develop Automated Surveillance of Postoperative Infections, starting with a prototype, the Automated Surveillance of Postoperative Infections 1.0.

Development and validation of a multivariable preoperative prediction model for postoperative length of stay in a broad inpatient surgical population.

BACKGROUND: Postoperative length of stay is a meaningful patient-centered outcome and an important determinant of healthcare costs. The Surgical Risk Preoperative Assessment System preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict postoperative length of stay has not been assessed. We aimed to determine whether the Surgical Risk Preoperative Assessment System variables could accurately predict postoperative length of stay up to 30 days in a broad inpatient surgical population. METHODS: This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database from 2012 to 2018. A model using the Surgical Risk Preoperative Assessment System variables and a 28-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables, were fit to the analytical cohort (2012-2018) using multiple linear regression and compared using model performance metrics. Internal chronological validation of the Surgical Risk Preoperative Assessment System model was conducted using training (2012-2017) and test (2018) datasets. RESULTS: We analyzed 3,295,028 procedures. The adjusted R2 for the Surgical Risk Preoperative Assessment System model fit to this cohort was 93.3% of that for the full model (0.347 vs 0.372). In the internal chronological validation of the Surgical Risk Preoperative Assessment System model, the adjusted R2 for the test dataset was 97.1% of that for the training dataset (0.3389 vs 0.3489). CONCLUSION: The parsimonious Surgical Risk Preoperative Assessment System model can preoperatively predict postoperative length of stay up to 30 days for inpatient surgical procedures almost as accurately as a model using all 28 American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables and has shown acceptable internal chronological validation.

The association between obesity and postoperative outcomes in a broad surgical population: A 7-year American College of Surgeons National Surgical Quality Improvement analysis.

BACKGROUND: The number of obese surgical patients continues to grow, and yet obesity's association with surgical outcomes is not totally clear. This study examined the association between obesity and surgical outcomes across a broad surgical population using a very large sample size. METHODS: This was an analysis of the 2012 to 2018 American College of Surgeons National Surgical Quality Improvement database, including all patients from 9 surgical specialties (general, gynecology, neurosurgery, orthopedics, otolaryngology, plastics, thoracic, urology, and vascular). Preoperative characteristics and postoperative outcomes were compared by body mass index class (normal weight 18.5-24.9 kg/m2, overweight 25.0-29.9, obese class I 30.0-34.9, obese II 35.0-39.9, obese III ≥40). Adjusted odds ratios were computed for adverse outcomes by body mass index class. RESULTS: A total of 5,572,019 patients were included; 44.6% were obese. Median operative times were marginally higher for obese patients (89 vs 83 minutes, P < .001). Compared to normal weight patients, overweight and obese patients in classes I, II, and III all had higher adjusted odds of developing infection, venous thromboembolism, and renal complications, but they did not exhibit elevated odds of other postoperative complications (mortality, overall morbidity, pulmonary, urinary tract infection, cardiac, bleeding, stroke, unplanned readmission, or discharge not home (except for class III patients). CONCLUSION: Obesity was associated with increased odds of postoperative infection, venous thromboembolism, and renal but not the other American College of Surgeons National Surgical Quality Improvement complications. Obese patients need to be carefully managed for these complications.

Lessons learned by thoracic surgeons during the COVID-19 pandemic.

Background: The scale of the coronavirus disease 2019 (COVID-19) pandemic has necessitated healthcare systems to adapt and evolve, altering physician roles and expectations. Thoracic surgeons have seen practice changes from new COVID-19 consults to necessary delay and triage of elective care. The goal of this study was to understand the impact of COVID-19 on thoracic surgeon experiences in order to anticipate roles and changes in practice in future such circumstances. Methods: Semi-structured, qualitative individual telephone interviews were conducted with thoracic surgeons. Interviews were structured to understand how surgeons were impacted by the COVID-19 pandemic and to record lessons learned. Interviews were conducted until thematic saturation was achieved. Data were analyzed using matrix analysis. Results: Eleven board-certified general thoracic surgeons from nine institutions were interviewed. Thoracic surgeon roles in COVID-19 care included critical care delivery, performing tracheostomies and establishing related protocols, and interventions for long-term airway complications. Attention was called to the impact of the pandemic on thoracic cancer: patients avoided hospitals because of concern over COVID-19, delaying care. Conclusions: Thoracic surgeons played a critical role in the COVID-19 pandemic response in both technical patient care and administrative capacities. Primary care responsibilities included the development, administration and delivery of tracheostomy protocols, and the care of down-stream airway complications. Thoracic surgeons were critical in triage decisions to minimize the impact of COVID-19 on thoracic cancer care. Lessons learned during the COVID-19 pandemic may provide insight into opportunities to promote collaboration in thoracic surgery and facilitate improved care delivery in future settings of resource limitation.

Does Work Relative Value Unit Measure Surgical Complexity for Risk Adjustment of Surgical Outcomes?

INTRODUCTION: The purpose of this study was to determine whether the work relative value unit (workRVU) of a patient's operation can be useful as a measure of surgical complexity for the risk adjustment of surgical outcomes. METHODS: We retrospectively analyzed the American College of Surgeon's National Surgical Quality Improvement Program database (2005-2018). We examined the associations of workRVU of the patient's primary operation with preoperative patient characteristics and associations with postoperative complications. We performed forward selection multiple logistic regression analysis to determine the predictive importance of workRVU. We then generated prediction models using patient characteristics with and without workRVU and compared c-indexes to assess workRVU's additive predictive value. RESULTS: 7,507,991 operations were included. Patients who were underweight, functionally dependent, transferred from an acute care hospital, had higher American Society of Anesthesiologists class or who had medical comorbidities had operations with higher workRVU (all P < 0.0001). The subspecialties with the highest workRVU were neurosurgery (mean = 22.2), thoracic surgery (mean = 21.1), and vascular surgery (mean = 18.8) (P < 0.0001). For all postoperative complications, mean workRVU was higher for patients with the complication than those without (all P < 0.0001). For eight of 12 postoperative complications, workRVU entered the logistic regression models as a predictor variable in the 1st to 4th steps. Addition of workRVU as a preoperative predictive variable improved the c-index of the prediction models. CONCLUSIONS: WorkRVU was associated with sicker patients and patients experiencing postoperative complications and was an important predictor of postoperative complications. When added to a prediction model including patient characteristics, it only marginally improved prediction. This is possibly because workRVU is associated with patient characteristics.

Preoperative Prediction of Unplanned Reoperation in a Broad Surgical Population.

INTRODUCTION: Unplanned reoperation is an undesirable outcome with considerable risks and an increasingly assessed quality of care metric. There are no preoperative prediction models for reoperation after an index surgery in a broad surgical population in the literature. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict unplanned reoperation has not been assessed. This study's objective was to determine whether the SURPAS model could accurately predict unplanned reoperation. METHODS: This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database, 2012-2018. An unplanned reoperation was defined as any unintended operation within 30 d of an initial scheduled operation. The 8-variable SURPAS model and a 29-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program nonlaboratory preoperative variables, were developed using multiple logistic regression and compared using discrimination and calibration metrics: C-indices (C), Hosmer-Lemeshow observed-to-expected plots, and Brier scores (BSs). The internal chronological validation of the SURPAS model was conducted using "training" (2012-2017) and "test" (2018) datasets. RESULTS: Of 5,777,108 patients, 162,387 (2.81%) underwent an unplanned reoperation. The SURPAS model's C-index of 0.748 was 99.20% of that for the full model (C = 0.754). Hosmer-Lemeshow plots showed good calibration for both models and BSs were similar (BS = 0.0264, full; BS = 0.0265, SURPAS). Internal chronological validation results were similar for the training (C = 0.749, BS = 0.0268) and test (C = 0.748, BS = 0.0250) datasets. CONCLUSIONS: The SURPAS model accurately predicted unplanned reoperation and was internally validated. Unplanned reoperation can be integrated into the SURPAS tool to provide preoperative risk assessment of this outcome, which could aid patient risk education.

Effect of Present at Time of Surgery on Unadjusted and Risk-Adjusted Postoperative Complication Rate.

BACKGROUND: Present at the time of surgery (PATOS) is an important measure to collect in postoperative complication surveillance systems because it may affect a patient's risk of a subsequent complication and the estimation of postoperative complication rates attributed to the healthcare system. The American College of Surgeons (ACS) NSQIP started collecting PATOS data for 8 postoperative complications in 2011, but no one has used these data to quantify how this may affect unadjusted and risk-adjusted postoperative complication rates. STUDY DESIGN: This study was a retrospective observational study of the ACS NSQIP database from 2012 to 2018. PATOS data were analyzed for the 8 postoperative complications of superficial, deep, and organ space surgical site infection; pneumonia; urinary tract infection; ventilator dependence; sepsis; and septic shock. Unadjusted postoperative complication rates were compared ignoring PATOS vs taking PATOS into account. Observed to expected ratios over time were also compared by calculating expected values using multiple logistic regression analyses with complication as the dependent variable and the 28 nonlaboratory preoperative variables in the ACS NSQIP database as the independent variables. RESULTS: In 5,777,108 patients, observed event rates for each outcome were reduced by between 6.1% (superficial surgical site infection) and 52.5% (sepsis) when PATOS was taken into account. The observed to expected ratios were similar each year for all outcomes, except for sepsis and septic shock in the early years. CONCLUSIONS: Taking PATOS into account is important for reporting unadjusted event rates. The effect varied by type of complication-lowest for superficial surgical site infection and highest for sepsis and septic shock. Taking PATOS into account was less important for risk-adjusted outcomes (observed to expected ratios), except for sepsis and septic shock.

Barriers and Facilitators in Implementation of an Esophagectomy Care Pathway: a Qualitative Analysis.

INTRODUCTION: A new postoperative esophagectomy care pathway was recently implemented at our institution. Practice pattern change among provider teams can prove challenging; therefore, we sought to study the barriers and facilitators toward pathway implementation at the provider level. METHODS: This qualitative study was guided by the Theoretical Domains Framework (TDF) to study the adoption and implementation of a post-esophagectomy care pathway. Sixteen in-depth interviews were conducted with providers involved with the pathway. Matrix analysis was used to analyze the data. RESULTS: Providers included attending surgeons (n = 6), advanced practice providers (n = 8), registered dietitian (n = 1), and clinic staff (n = 1). TDF domains that were salient across our findings included knowledge, beliefs about consequences, social influences, and environmental context and resources. Identified facilitators included were electronic health record tools, such as note templates including pathway components and a pathway-specific order set, patient satisfaction, and preliminary data indicating clinical benefits such as a reduced anastomotic leak rate. The major barrier reported was a hesitance to abandon previous practice patterns, most prevalent at the attending surgeon level. CONCLUSION: The TDF enabled us to identify and understand the individuals' perceived barriers and facilitators toward adoption and implementation of a postoperative esophagectomy pathway. This analysis can help guide and improve adoption of surgical patient care pathways among providers.

Inpatient Versus Outpatient Surgery: A Comparison of Postoperative Mortality and Morbidity in Elective Operations.

BACKGROUND: Operations performed outpatient offer several benefits. The prevalence of outpatient operations is growing. Consequently, the proportion of patients with multiple comorbidities undergoing outpatient surgery is increasing. We compared 30-day mortality and overall morbidity between outpatient and inpatient elective operations. METHODS: Using the 2005-2018 ACS-NSQIP database, we evaluated trends in percent of hospital outpatient operations performed over time, and the percent of operations done outpatient versus inpatient by CPT code. Patient characteristics were compared for outpatient versus inpatient operations. We compared unadjusted and risk-adjusted 30-day mortality and morbidity for inpatient and outpatient operations. RESULTS: A total of 6,494,298 patients were included. The proportion of outpatient operations increased over time, from 37.8% in 2005 to 48.2% in 2018. We analyzed the 50 most frequent operations performed outpatient versus inpatient 25-75% of the time (n = 1,743,097). Patients having outpatient operations were younger (51.6 vs 54.6 years), female (70.3% vs 67.3%), had fewer comorbidities, and lower ASA class (I-II, 69.3% vs. 59.9%). On both unadjusted and risk-adjusted analysis, 30-day mortality and overall morbidity were less likely in outpatient versus inpatient operations. CONCLUSION: In this large multi-specialty analysis, we found that patients undergoing outpatient surgery had lower risk of 30-day morbidity and mortality than those undergoing the same inpatient operation. Patients having outpatient surgery were generally healthier, suggesting careful patient selection occurred even with increasing outpatient operation frequency. Patients and providers can feel reassured that outpatient operations are a safe, reasonable option for selected patients.

Development and validation of models for detection of postoperative infections using structured electronic health records data and machine learning.

BACKGROUND: Postoperative infections constitute more than half of all postoperative complications. Surveillance of these complications is primarily done through manual chart review, which is time consuming, expensive, and typically only covers 10% to 15% of all operations. Automated surveillance would permit the timely evaluation of and reporting of all operations. METHODS: The goal of this study was to develop and validate parsimonious, interpretable models for conducting surveillance of postoperative infections using structured electronic health records data. This was a retrospective study using 30,639 unique operations from 5 major hospitals between 2013 and 2019. Structured electronic health records data were linked to postoperative outcomes data from the American College of Surgeons National Surgical Quality Improvement Program. Predictors from the electronic health records included diagnoses, procedures, and medications. Infectious complications included surgical site infection, urinary tract infection, sepsis, and pneumonia within 30 days of surgery. The knockoff filter, a penalized regression technique that controls type I error, was applied for variable selection. Models were validated in a chronological held-out dataset. RESULTS: Seven percent of patients experienced at least one type of postoperative infection. Models selected contained between 4 and 8 variables and achieved >0.91 area under the receiver operating characteristic curve, >81% specificity, >87% sensitivity, >99% negative predictive value, and 10% to 15% positive predictive value in a held-out test dataset. CONCLUSION: Surveillance and reporting of postoperative infection rates can be implemented for all operations with high accuracy using electronic health records data and simple linear regression models.

Development of a universal thoracic enhanced recover after surgery protocol for implementation across a diverse multi-hospital health system.

Background: Implementation of enhanced recovery after surgery (ERAS) pathways for patients undergoing anatomic lung resection have been reported at individual institutions. We hypothesized that an ERAS pathway can be successfully implemented across a large healthcare system including different types of hospital settings (academic, academic-affiliated, community). Methods: An expert panel with representation from each hospital within a healthcare system was convened to establish a thoracic ERAS pathway for patients undergoing anatomic lung resection and to develop tools and analytics to ensure consistent application. The protocol was translated into an order set and pathway within the electronic health record (EHR). Iterative implementation was performed with recording of the processes involved. Barriers and facilitators to implementation were recorded. Results: Development and implementation of the protocol took 13 months from conception to rollout. Considerable change management was needed for consensus and incorporation into practice. Facilitators of change included peer accountability, incorporating ERAS care elements into the EHR, and conducting case reviews with timely feedback on protocol deviations. Barriers included institutional cultural differences, agreement in defining mindful deviation from the ERAS protocol, lack of access to specific coded data, and resource scarcity caused by the COVID-19 pandemic. Support from the hospital system's executive leadership and institutional commitment to quality improvement helped overcome barriers and maintain momentum. Conclusions: Development and implementation of a health-system wide thoracic ERAS protocol for anatomic lung resections across a six-hospital health system requires a multidisciplinary team approach. Barriers can be overcome though multidisciplinary team engagement and executive leadership support.

Associations Between Preoperative Risk, Postoperative Complications, and 30-Day Mortality.

BACKGROUND: Comorbidities and postoperative complications increase mortality, making early recognition and management critical. It is useful to understand how they are associated with one another. This study assesses associations between comorbidities, complications, and mortality. METHODS: We calculated associations between comorbidities, complications, and 30-day mortality using the 2012-2018 ACS-NSQIP database. We examined the association between mortality and number of complications which complications were most associated with mortality. RESULTS: 5,777,108 patients were included. 30-day mortality was 0.95%. For most comorbidities or postoperative complications, patients with these had higher mortality than patients without. Having ≥ 1 complication increased mortality risk by 32.5-fold (6.5% vs. 0.2%). Mortality rate significantly increased with increasing number of complications, particularly after two or more complications. Bleeding and sepsis were associated with the most deaths. CONCLUSION: The 30-day mortality rate was < 1% but was 32-fold higher in patients with complications and increased rapidly for patients with ≥ 2 complications. Bleeding and sepsis were the most prominent complications associated with mortality.

Attitudes about use of preoperative risk assessment tools: a survey of surgeons and surgical residents in an academic health system.

BACKGROUND: Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk. METHODS: Surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey via REDCap (Research Electronic Data Capture) between 8/2019-6/2020. Reminder emails were sent once weekly for three weeks. All completed surveys by surgical residents and attendings were included; incomplete surveys were excluded. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher's exact test and unpaired t-tests comparing responses by surgical attendings vs. residents. RESULTS: A total of 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons were emailed the survey. Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy. CONCLUSIONS: A small percentage of surgeons use surgical risk calculators as part of their routine practice. Time, inaccessibility, and trust in accuracy were the most significant barriers to use.

Development and Validation of a Multivariable Prediction Model for Postoperative Intensive Care Unit Stay in a Broad Surgical Population.

IMPORTANCE: Despite limited capacity and expensive cost, there are minimal objective data to guide postoperative allocation of intensive care unit (ICU) beds. The Surgical Risk Preoperative Assessment System (SURPAS) uses 8 preoperative variables to predict many common postoperative complications, but it has not yet been evaluated in predicting postoperative ICU admission. OBJECTIVE: To determine if the SURPAS model could accurately predict postoperative ICU admission in a broad surgical population. DESIGN, SETTING, AND PARTICIPANTS: This decision analytical model was a retrospective, observational analysis of prospectively collected patient data from the 2012 to 2018 American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database, which were merged with individual patients' electronic health record data to capture postoperative ICU use. Multivariable logistic regression modeling was used to determine how the 8 preoperative variables of the SURPAS model predicted ICU use compared with a model inputting all 28 preoperatively available NSQIP variables. Data included in the analysis were collected for the ACS NSQIP at 5 hospitals (1 tertiary academic center, 4 academic affiliated hospitals) within the University of Colorado Health System between January 1, 2012, and December 31, 2018. Included patients were those undergoing surgery in 9 surgical specialties during the 2012 to 2018 period. Data were analyzed from May 29 to July 30, 2021. EXPOSURE: Surgery in 9 surgical specialties, including general, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery. MAIN OUTCOMES AND MEASURES: Use of ICU care up to 30 days after surgery. RESULTS: A total of 34 568 patients were included in the analytical data set: 32 032 (92.7%) in the cohort without postoperative ICU use and 2545 (7.4%) in the cohort with postoperative ICU use (no ICU use: mean [SD] age, 54.9 [16.6] years; 18 188 women [56.8%]; ICU use: mean [SD] age, 60.3 [15.3] years; 1333 men [52.4%]). For the internal chronologic validation of the 7-variable SURPAS model, data from 2012 to 2016 were used as the training data set (n = 24 250, 70.2% of the total sample size of 34 568) and data from 2017 to 2018 were used as the test data set (n = 10 318, 29.8% of the total sample size of 34 568). The C statistic improved in the test data set compared with the training data set (0.933; 95% CI, 0.924-0.941 vs 0.922; 95% CI, 0.917-0.928), whereas the Brier score was slightly worse in the test data set compared with the training data set (0.045; 95% CI, 0.042-0.048 vs 0.045; 95% CI, 0.043-0.047). The SURPAS model compared favorably with the model inputting all 28 NSQIP variables, with both having good calibration between observed and expected outcomes in the Hosmer-Lemeshow graphs and similar Brier scores (model inputting all variables, 0.044; 95% CI, 0.043-0.048; SURPAS model, 0.045; 95% CI, 0.042-0.046) and C statistics (model inputting all variables, 0.929; 95% CI, 0.925-0.934; SURPAS model, 0.925; 95% CI, 0.921-0.930). CONCLUSIONS AND RELEVANCE: Results of this decision analytical model study revealed that the SURPAS prediction model accurately predicted postoperative ICU use across a diverse surgical population. These results suggest that the SURPAS prediction model can be used to help with preoperative planning and resource allocation of limited ICU beds.

Development and validation of a prediction model for conversion of outpatient to inpatient surgery.

BACKGROUND: Unplanned hospital admission after intended outpatient surgery is an undesirable outcome. We aimed to develop a prediction model that estimates a patient's risk of conversion from outpatient surgery to inpatient hospitalization. METHODS: This was a retrospective analysis using the American College of Surgeons National Surgical Quality Improvement Program database, 2005 to 2018. Conversion from outpatient to inpatient surgery was defined as having outpatient surgery and >1 day hospital stay. The Surgical Risk Preoperative Assessment System was developed using multiple logistic regression on a training dataset (2005-2016) and compared to a model using the 26 relevant variables in the American College of Surgeons National Surgical Quality Improvement Program. The Surgical Risk Preoperative Assessment System was validated using a testing dataset (2017-2018). Performance statistics and Hosmer-Lemeshow plots were compared. Two high-risk definitions were compared: (1) the maximum Youden index, and (2) the cohort above the tenth decile of risk on the Hosmer-Lemeshow plot. The sensitivities, specificities, positive predictive values, negative predictive values, and accuracies were compared. RESULTS: In all, 2,822,379 patients were included; 3.6% of patients unexpectedly converted to inpatient. The 6-variable Surgical Risk Preoperative Assessment System model performed comparably to the 26-variable American College of Surgeons National Surgical Quality Improvement Program model (c-indices = 0.818 vs. 0.823; Brier scores = 0.0308 vs 0.0306, respectively). The Surgical Risk Preoperative Assessment System performed well on internal validation (c-index = 0.818, Brier score = 0.0341). The tenth decile of risk definition had higher specificity, positive predictive values, and accuracy than the maximum Youden index definition, while having lower sensitivity. CONCLUSION: The Surgical Risk Preoperative Assessment System accurately predicted a patient's risk of unplanned outpatient-to-inpatient conversion. Patients at higher risk should be considered for inpatient surgery, while lower risk patients could safely undergo operations at ambulatory surgery centers.

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons vs a Standardized Surgical Risk Prediction Tool.

Considerable variability exists between surgeons' assessments of a patient's individual preoperative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes vs a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs 5.1%, P ≤ 0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs 69.8%, P < 0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs 44.3%, P < 0.001). C-indices for SURPAS vs surgeons were 0.84 vs 0.76 (P = 0.3) for morbidity and 0.98 vs 0.85 (P = 0.001) for mortality. Brier scores for SURPAS vs surgeons were 0.1579 vs 0.1986 for morbidity (P = 0.03) and 0.0409 vs 0.0543 for mortality (P = 0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC = 0.654) and mortality (ICC = 0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.

Using the Surgical Risk Preoperative Assessment System to Define the "High Risk" Surgical Patient.

BACKGROUND: Defining a "high risk" surgical population remains challenging. Using the Surgical Risk Preoperative Assessment System (SURPAS), we sought to define "high risk" groups for adverse postoperative outcomes. MATERIALS AND METHODS: We retrospectively analyzed the 2009-2018 American College of Surgeons National Surgical Quality Improvement Program database. SURPAS calculated probabilities of 12 postoperative adverse events. The Hosmer Lemeshow graphs of deciles of risk and maximum Youden index were compared to define "high risk." RESULTS: Hosmer-Lemeshow plots suggested the "high risk" patient could be defined by the 10th decile of risk. Maximum Youden index found lower cutoff points for defining "high risk" patients and included more patients with events. This resulted in more patients classified as "high risk" and higher number needed to treat to prevent one complication. Some specialties (thoracic, vascular, general) had more "high risk" patients, while others (otolaryngology, plastic) had lower proportions. CONCLUSIONS: SURPAS can define the "high risk" surgical population that may benefit from risk-mitigating interventions.

Associations between preoperative risks of postoperative complications: Results of an analysis of 4.8 Million ACS-NSQIP patients.

BACKGROUND: Surgical Risk Preoperative Assessment System (SURPAS) estimates patient's preoperative risk of 12 postoperative complications, yet little is known about associations between these probabilities- We sought to examine relationships between predicted probabilities. METHODS: Risk of 12 postoperative complications was calculated using SURPAS and the 2012-2018 ACS-NSQIP database. Pearson correlation coefficients (r) were computed to examine relationships between predicted outcomes. "High-risk" was predicted risk in the 10th decile. RESULTS: 4,777,267 patients were included. 71.1% were not high risk, 10.7% were high risk for 1, and 18.2% were high risk for ≥2 complications. High mortality risk was associated with high risk for pulmonary (r = 0.94), cardiac (r = 0.98), renal (r = 0.93), and stroke (0.96) complications. Patients high-risk for ≥2 complications had the most comorbidities and actual adverse outcomes. CONCLUSIONS: High preoperative risk for certain postoperative complications had strong correlations. 18.2% of patients were high-risk for ≥2 complications and could be targeted for risk reduction interventions.

Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas.

BACKGROUND: The universal Surgical Risk Preoperative Assessment System (SURPAS) prediction models for postoperative adverse outcomes have good accuracy for estimating risk in broad surgical populations and for surgical specialties. The accuracy in individual operations has not yet been assessed. The objective of this study was to evaluate the Surgical Risk Preoperative Assessment System in predicting adverse outcomes for selected individual operations. METHODS: The SURPAS models were applied to the top 2 most frequent common procedural terminology codes in 9 surgical specialties and 5 additional common general surgical operations in the 2009 to 2018 database of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). Goodness of fit statistics were estimated, including c-indices for discrimination, Hosmer-Lemeshow graphs and P values for calibration, overall observed versus expected event rates, and Brier scores. RESULTS: The total sample size was 2,020,172, which represented 29% of the 6.9 million operations in the ACS NSQIP database. Average c-indices across 12 outcomes were acceptable (≥0.70) for 13 (56.5%) of the 23 operations. Overall observed-to-expected rates were similar for mortality and overall morbidity across the 23 operations. Hosmer-Lemeshow graphs over quintiles of risk comparing observed-to-expected rates of mortality and overall morbidity were similar for 52% and 70% of operations, respectively. Model performance was better in less complex operations and those done in patients with lower preoperative risk. CONCLUSION: SURPAS displayed accuracy in estimating postoperative adverse events for some of the 23 operations studied, but not all. In the procedures where SURPAS was not accurate, developing disease or operation-specific risk models might be appropriate.