Attitudes about use of preoperative risk assessment tools: a survey of surgeons and surgical residents in an academic health system.

BACKGROUND: Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk. METHODS: Surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey via REDCap (Research Electronic Data Capture) between 8/2019-6/2020. Reminder emails were sent once weekly for three weeks. All completed surveys by surgical residents and attendings were included; incomplete surveys were excluded. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher's exact test and unpaired t-tests comparing responses by surgical attendings vs. residents. RESULTS: A total of 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons were emailed the survey. Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy. CONCLUSIONS: A small percentage of surgeons use surgical risk calculators as part of their routine practice. Time, inaccessibility, and trust in accuracy were the most significant barriers to use.

Using the Surgical Risk Preoperative Assessment System to Define the "High Risk" Surgical Patient.

BACKGROUND: Defining a "high risk" surgical population remains challenging. Using the Surgical Risk Preoperative Assessment System (SURPAS), we sought to define "high risk" groups for adverse postoperative outcomes. MATERIALS AND METHODS: We retrospectively analyzed the 2009-2018 American College of Surgeons National Surgical Quality Improvement Program database. SURPAS calculated probabilities of 12 postoperative adverse events. The Hosmer Lemeshow graphs of deciles of risk and maximum Youden index were compared to define "high risk." RESULTS: Hosmer-Lemeshow plots suggested the "high risk" patient could be defined by the 10th decile of risk. Maximum Youden index found lower cutoff points for defining "high risk" patients and included more patients with events. This resulted in more patients classified as "high risk" and higher number needed to treat to prevent one complication. Some specialties (thoracic, vascular, general) had more "high risk" patients, while others (otolaryngology, plastic) had lower proportions. CONCLUSIONS: SURPAS can define the "high risk" surgical population that may benefit from risk-mitigating interventions.

Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas.

BACKGROUND: The universal Surgical Risk Preoperative Assessment System (SURPAS) prediction models for postoperative adverse outcomes have good accuracy for estimating risk in broad surgical populations and for surgical specialties. The accuracy in individual operations has not yet been assessed. The objective of this study was to evaluate the Surgical Risk Preoperative Assessment System in predicting adverse outcomes for selected individual operations. METHODS: The SURPAS models were applied to the top 2 most frequent common procedural terminology codes in 9 surgical specialties and 5 additional common general surgical operations in the 2009 to 2018 database of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). Goodness of fit statistics were estimated, including c-indices for discrimination, Hosmer-Lemeshow graphs and P values for calibration, overall observed versus expected event rates, and Brier scores. RESULTS: The total sample size was 2,020,172, which represented 29% of the 6.9 million operations in the ACS NSQIP database. Average c-indices across 12 outcomes were acceptable (≥0.70) for 13 (56.5%) of the 23 operations. Overall observed-to-expected rates were similar for mortality and overall morbidity across the 23 operations. Hosmer-Lemeshow graphs over quintiles of risk comparing observed-to-expected rates of mortality and overall morbidity were similar for 52% and 70% of operations, respectively. Model performance was better in less complex operations and those done in patients with lower preoperative risk. CONCLUSION: SURPAS displayed accuracy in estimating postoperative adverse events for some of the 23 operations studied, but not all. In the procedures where SURPAS was not accurate, developing disease or operation-specific risk models might be appropriate.

Mapping the organizational readiness to change assessment to the Consolidated Framework for Implementation Research.

BACKGROUND: Implementation researchers recognize the influential role of organizational factors and, thus, seek to assess these factors using quantitative measurement instruments. However, researchers are hindered by instruments that measure similar constructs but rely on different nomenclature and/or definitions. The Consolidated Framework for Implementation Research (CFIR) provides a taxonomy of constructs derived from prior frameworks and empirical studies of implementation-related constructs. The CFIR includes constructs based on the original Promoting Action on Research Implementation in Health Services (PARiHS) framework which highlights the key roles of strength of evidence for a specific evidence-based intervention (EBI), favorability of organizational context for change, and capacities to facilitate implementation of the EBI. Although the CFIR is among the most frequently used implementation frameworks, it does not include quantitative measures. The Organizational Resource and Context Assessment (ORCA) is a quantitative measurement instrument that was developed based on PARiHS, assessing its three domains. Factors within these three domains are conceptually similar to constructs in the CFIR but do not match directly. The aim of this work was to map ORCA survey items to CFIR constructs to enable direct comparisons and syntheses of findings across studies using the CFIR and/or ORCA. METHODS: Two distinct, independent research teams, each used rigorous constant comparative techniques with deliberation and consensus to map individual items from the ORCA to the five domains and 39 constructs of CFIR. RESULTS: ORCA items were mapped primarily to three of five CFIR domains: Inner Setting, Process, and Intervention Characteristics. The two research teams agreed on 88% of mappings at the higher domain level; at the lower construct level, their mappings aligned for 62.2% of the ORCA items. CONCLUSIONS: Mapping results reveal that the ORCA focuses measurement prominently on Inner Setting, Process, and Intervention Characteristics. This mapping guide can help improve consistency in measurement and reporting, enabling more efficient comparison and synthesis of findings that use either the ORCA instrument or the CFIR framework. The guide helps advance implementation science utilizing mixed methods by providing CFIR users with quantitative measures for selected constructs and enables ORCA users to map their findings to CFIR constructs.

A comparison of the new, parsimonious tool Surgical Risk Preoperative Assessment System (SURPAS) to the American College of Surgeons (ACS) risk calculator in emergency surgery.

BACKGROUND: The Surgical Risk Preoperative Assessment System accurately predicts postoperative complications in elective surgery using only 8 preoperative variables, but its performance in emergency surgery has not been evaluated. We hypothesized that the Surgical Risk Preoperative Assessment System accurately predicts postoperative complications in emergency surgery and compared its performance to that of the American College of Surgeons Surgical Risk Calculator. METHODS: We calculated the Surgical Risk Preoperative Assessment System and the American College of Surgeons Surgical Risk Calculator risk estimates in a random sample of 1,010 emergency surgery cases from the American College of Surgeons National Surgical Quality Improvement Program 2014 to 2017 database. Risk estimates were compared with known outcomes. Analyses included the Hosmer-Lemeshow goodness of fit graphs and P values, c-indexes, and Brier scores. RESULTS: Results were similar between the Surgical Risk Preoperative Assessment System and the American College of Surgeons Surgical Risk Calculator for mortality, urinary tract infection, cardiac, venous thromboembolism, and renal complications. The American College of Surgeons Surgical Risk Calculator underestimated morbidity compared with the Surgical Risk Preoperative Assessment System (16.04% American College of Surgeons Surgical Risk Calculator vs 24.88% Surgical Risk Preoperative Assessment System vs 24.3% observed). Both calculators overestimated readmission (7.67% American College of Surgeons Surgical Risk Calculator vs 5.18% Surgical Risk Preoperative Assessment System vs 4.1% observed). CONCLUSION: Both calculators predicted mortality, urinary tract infection, cardiac, venous thromboembolism, and renal complications well, but readmissions relatively poorly. The Surgical Risk Preoperative Assessment System estimated overall morbidity accurately, while the American College of Surgeons Surgical Risk Calculator underestimated this risk.

The value of the "Surgical Risk Preoperative Assessment System" (SURPAS) in preoperative consultation for elective surgery: a pilot study.

BACKGROUND: Risk assessment is essential to informed decision making in surgery. Preoperative use of the Surgical Risk Preoperative Assessment System (SURPAS) providing individualized risk assessment, may enhance informed consent. We assessed patient and provider perceptions of SURPAS as a risk assessment tool. METHODS: A convergent mixed-methods study assessed SURPAS's trial implementation, concurrently collecting quantitative and qualitative data, separately analyzing it, and integrating the results. Patients and providers were surveyed and interviewed on their opinion of how SURPAS impacted the preoperative encounter. Relationships between patient risk and patient and provider assessment of SURPAS were examined. RESULTS: A total of 197 patients were provided their SURPAS postoperative risk estimates in nine surgeon's clinics. Of the total patients, 98.8% reported they understood their surgical risks very or quite well after exposure to SURPAS; 92.7% reported SURPAS was very helpful or helpful. Providers shared that 83.4% of the time they reported SURPAS was very or somewhat helpful; 44.7% of the time the providers reported it changed their interaction with the patient and this change was beneficial 94.3% of the time. As patient risk increased, providers reported that SURPAS was increasingly helpful (p < 0.0001). CONCLUSIONS: Patients and providers reported the use of SURPAS helpful and informative during the preoperative risk assessment of patients, thus improving the surgical decision making process. Patients thought that SURPAS was helpful regardless of their risk level, whereas providers thought that SURPAS was more helpful in higher risk patients.

Use of Surgical Risk Preoperative Assessment System (SURPAS) and Patient Satisfaction During Informed Consent for Surgery.

BACKGROUND: The objective of this study was to determine the effects of using the Surgical Risk Preoperative Assessment System (SURPAS) on patient satisfaction and surgeon efficiency in the surgical informed consent process, as compared to surgeons' "usual" consent process. STUDY DESIGN: Patient perception of the consent process was assessed via survey in 2 cohorts: 10 surgeons in different specialties used their "usual" consent process for 10 patients; these surgeons were then taught to use SURPAS, and they used it during the informed consent process of 10 additional patients. The data were compared using Fisher's exact test and the Cochran-Mantel-Haenszel test. RESULTS: One hundred patients underwent the "usual" consent process (USUAL), and 93 underwent SURPAS-guided consent (SURPAS). Eighty-two percent of SURPAS were "very satisfied" and 18% were "satisfied" with risk discussion vs 16% and 72% of USUAL, respectively. Of those who used SURPAS, 75.3% reported the risk discussion made them "more comfortable" with surgery vs 19% of USUAL, and 90.3% of SURPAS users reported "somewhat" or "greatly decreased" anxiety vs 20% of USUAL. All p values were <0.0001. Among SURPAS patients, 97.9% reported "enough time spent discussing risks" vs 72.0% of USUAL patients. CONCLUSIONS: The SURPAS tool improved the informed consent process for patients compared with the "usual" consent process, in terms of patient satisfaction, ie making patients feel more comfortable and less anxious about their impending operations. Providers should consider integrating the SURPAS tool into their preoperative consent process.

Refining the predictive variables in the "Surgical Risk Preoperative Assessment System" (SURPAS): a descriptive analysis.

BACKGROUND: The Surgical Risk Preoperative Assessment System (SURPAS) is a parsimonious set of models providing accurate preoperative prediction of common adverse outcomes for individual patients. However, focus groups with surgeons and patients have developed a list of questions about and recommendations for how to further improve SURPAS's usability and usefulness. Eight issues were systematically evaluated to improve SURPAS. METHODS: The eight issues were divided into three groups: concerns to be addressed through further analysis of the database; addition of features to the SURPAS tool; and the collection of additional outcomes. Standard multiple logistic regression analysis was performed using the 2005-2015 American College of Surgeons National Surgical Quality Improvement Participant Use File (ACS NSQIP PUF) to refine models: substitution of the preoperative sepsis variable with a procedure-related risk variable; testing of an indicator variable for multiple concurrent procedure codes in complex operations; and addition of outcomes to increase clinical applicability. Automated risk documentation in the electronic health record and a patient handout and supporting documentation were developed. Long term functional outcomes were considered. RESULTS: Model discrimination and calibration improved when preoperative sepsis was replaced with a procedure-related risk variable. Addition of an indicator variable for multiple concurrent procedures did not significantly improve the models. Models were developed for a revised set of eleven adverse postoperative outcomes that separated bleeding/transfusion from the cardiac outcomes, UTI from the other infection outcomes, and added a predictive model for unplanned readmission. Automated documentation of risk assessment in the electronic health record, visual displays of risk for providers and patients and an "About" section describing the development of the tool were developed and implemented. Long term functional outcomes were considered to be beyond the scope of the current SURPAS tool. CONCLUSION: Refinements to SURPAS were successful in improving the accuracy of the models, while reducing manual entry to five of the eight variables. Adding a predictor variable to indicate a complex operation with multiple current procedure codes did not improve the accuracy of the models. We developed graphical displays of risk for providers and patients, including a take-home handout and automated documentation of risk in the electronic health record. These improvements should facilitate easier implementation of SURPAS.

Improving anti-platelet therapy adherence in the Veterans Health Administration: A randomized multi-site hybrid effectiveness-implementation study protocol.

BACKGROUND: P2Y12 inhibitors are critical following percutaneous coronary intervention (PCI) with stent placement; they reduce the risk of stent thrombosis and myocardial infarction. Despite the importance of the therapy, non-adherence is common among Veterans. METHODS AND RESULTS: Our main objective is to conduct a multi-site randomized stepped wedge trial to test the effectiveness of a multi-faceted intervention to improve adherence to P2Y12 inhibitors and PCI outcomes as well as formatively evaluate and refine the intervention implementation process. The primary outcomes of the study are the proportion of patients whose P2Y12 inhibitor prescription was filled at the time of hospital discharge following PCI with stent placement as well as the proportion of patients who were adherent based on the pharmacy refill data in the year after PCI hospital discharge. We will also assess the secondary outcomes such as bleeding, myocardial infarction, stroke, and mortality among these patients, and the cost-effectiveness of the intervention. The study was conducted at Veterans Health Administration (VA) PCI sites. At each site, we enrolled patients over a 6-month period and followed them for 12 months after PCI. Additionally, we collected qualitative data to identify contextual factors and to assess barriers and facilitators to the implementation and maintenance of the intervention. CONCLUSIONS: The study will add to the current state of knowledge on improving medication adherence in patients receiving PCI with stent implantation. Moreover, the study includes an extensive examination of the implementation process and will contribute to the field of implementation science. CLINICAL TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT01609842 https://clinicaltrials.gov/ct2/show/NCT01609842?term=clopidogrel+adherence&cntry1=NA%3AUS&rank=1.

Assessment of barriers and facilitators in the implementation of appropriate use criteria for elective percutaneous coronary interventions: a qualitative study.

BACKGROUND: The use of inappropriate elective Percutaneous Coronary Intervention (PCI) has decreased over time, but hospital-level variation in the use of inappropriate PCI persists. Understanding the barriers and facilitators to the implementation of Appropriate Use Criteria (AUC) guidelines may inform efforts to improve elective PCI appropriateness. METHODS: All hospitals performing PCI in Washington State were categorized by their use of inappropriate elective PCI in 2010 to 2013. Semi-structured, qualitative telephone interviews were then conducted with 17 individual interviews at 13 sites in Washington State to identify barriers and facilitators to the implementation of the AUC guidelines. An inductive and deductive, team-based analytical approach, drawing primarily on Matrix analysis was performed to identify factors affecting implementation of the AUC. RESULTS: Specific facilitators were identified that supported successful implementation of the AUC. These included collaborative catheterization laboratory environments that allow all staff to participate with questions and opinions; ongoing AUC education with catheterization laboratory teams and referring providers; internal AUC peer review processes; interventional cardiologist be directly involved with the pre-procedural review process; checklist-based algorithms for pre-procedural documentation; systems redesign to include insurance companies; and AUC educational information with patients. Barriers to implementation of the AUC included external pressures, such as competition for patients, and the lack of shared medical records with sites that referred patients for coronary angiography. CONCLUSIONS: The identified facilitators enabled sites to successfully implement the AUC. Catheterization laboratories struggling to successfully implement the AUC may consider utilizing these strategies to improve their processes to improve patient selection for elective PCI.

Assessment of attitudes towards future implementation of the "Surgical Risk Preoperative Assessment System" (SURPAS) tool: a pilot survey among patients, surgeons, and hospital administrators.

BACKGROUND: Risk assessment in surgery is essential to guide treatment decisions but is highly variable in practice. Providing formal preoperative risk assessment to surgical teams and patients may optimize understanding of risk. Implementation of the Surgical Risk Preoperative Assessment System (SURPAS), an innovative real time, universal, preoperative tool providing individualized risk assessment, may enhance informed consent and reduce adverse outcomes. To ensure optimal development and implementation of SURPAS we performed an in-depth pre-implementation evaluation of SURPAS at an academic tertiary referral center in Colorado. METHODS: Four focus groups with 24 patients, three focus groups with 29 surgical providers and clinic administrators, and five individual interviews with administrative officials were conducted to elicit their perspectives about the development and implementation of SURPAS. Qualitative data collection and analyses, utilizing a Matrix Analysis approach were used to explore insights regarding SURPAS. RESULTS: Participants were positive about SURPAS and provided suggestions to improve and address concerns regarding it. For healthcare personnel three major themes emerged: 1) The SURPAS tool - Important work especially for high risk patients, yet not a substitute for clinical judgment; 2) Benefits of SURPAS to the risk assessment process - Improves the processes, enhances patients' participation in shared decision-making process, and creates a permanent record; and 3) Facilitators and barriers of implementation of SURPAS - Easy to incorporate into clinical practice in spite of surgical providers' resistance to adoption of new technology. For patients three major themes emerged: 1) Past experience of preoperative risk assessment discussions - Patients were not made aware of possible complications that occurred; 2) The SURPAS tool - All patients liked SURPAS and believed having printed material would be useful to guide discussions and facilitate remembering conversations with the providers; and 3) Potential concerns with having risk assessment information - Patients were mixed in deciding to have an operation with high risks. CONCLUSIONS: Systematically capturing data from the beginning of the implementation process from key stakeholders (patients, surgical providers, clinical staff, and administrators) that includes adaptations to the tool and implementation process will help to inform pragmatic approaches for implementing the SURPAS tool in various settings, scaling-up, and sustaining it.

Bringing it home: expanding the local reach of dissemination and implementation training via a university-based workshop.

BACKGROUND: Currently, national training programs do not have the capacity to meet the growing demand for dissemination and implementation (D&I) workforce education and development. The Colorado Research in Implementation Science Program (CRISP) developed and delivered an introductory D&I workshop adapted from national programs to extend training reach and foster a local learning community for D&I. METHODS: To gauge interest and assess learning needs, a pre-registration survey was administered. Based on feedback, a 1.5-day workshop was designed. Day 1 introduced D&I frameworks, strategies, and evaluation principles. Local and national D&I experts provided ignite-style talks on key lessons followed by panel discussion. Breakout sessions discussed community engagement and applying for D&I grants. A workbook was developed to enhance the training and provided exercises for application to an individual's projects. Day 2 offered expert-led mentoring sessions with selected participants who desired advanced instruction. Two follow-up surveys (immediate post-workshop, 6 months) assessed knowledge gained from participation and utilization of workshop content. RESULTS: Ninety-three workshop registrants completed an assessment survey to inform workshop objectives and curriculum design; 43 % were new and 54 % reported a basic understanding of the D&I field. Pre-registrants intended to use the training to "apply for a D&I grant" (73 %); "incorporate D&I into existing projects" (76 %), and for quality improvement (51 %). Sixty-eight individuals attended Day 1; 11 also attended Day 2 mentoring sessions. In the 1-week post-workshop survey (n = 34), 100 % strongly agreed they were satisfied with the training; 97 % strongly agreed the workshop workbook was a valuable resource. All Day 2 participants strongly agreed that working closely with faculty and experts increased their overall confidence. In the 6-month follow-up evaluation (n = 23), evidence of new D&I-related manuscripts and grant proposals was found. Training materials were published online ( www.ucdenver.edu/implementation/workshops ) and disseminated via the National Institutes of Health (NIH) Clinical and Translational Science Awards Consortium. To sustain reach, CRISP adapted the materials into an interactive e-book ( www.CRISPebooks.org ) and launched a new graduate course. CONCLUSIONS: Local D&I training workshops can extend the reach of national training programs.

Temporal Trends in Percutaneous Coronary Intervention Appropriateness: Insights From the Clinical Outcomes Assessment Program.

BACKGROUND: It is unknown whether the appropriate use of percutaneous coronary intervention (PCI) has improved over time and whether trends in PCI appropriateness have been accompanied by changes in the use of PCI. METHODS AND RESULTS: We applied appropriate use criteria to determine the appropriateness of all 51 872 PCI performed in Washington State from 2010 through 2013. We evaluated the number of PCIs performed from 2006 through 2013 to provide a comparator period that preceded statewide appropriateness assessment beginning in 2010. Between 2010 and 2013, the overall number of PCI decreased by 6.8% (13 267 PCIs in 2010 to 12 193 in 2013) with a 43% decline in the number of PCIs for elective indications (3818 PCIs in 2010 to 2193 PCIs in 2013). The decline in the use of elective PCI was significantly larger after the onset of statewide PCI appropriateness assessment in 2010 (P=0.03). The proportion of elective PCIs classified as appropriate increased from 26% in 2010 to 38% in 2013, whereas the proportion of inappropriate PCIs decreased from 16% to 13% (P<0.001 for trends). Significant improvements in the proportion of inappropriate PCI were limited to the tertile of hospitals with the largest decline in PCIs classified as inappropriate (25% in 2010 to 12% in 2013; P=0.03). CONCLUSIONS: In Washington State, the use of PCI for elective indications has decreased over time with concurrent improvements in PCI appropriateness. However, improvements in PCI appropriateness were limited to a minority of hospitals. Understanding processes at these high-performing hospitals may inform efforts to improve PCI appropriateness.

Patient-centered adherence intervention after acute coronary syndrome hospitalization.

BACKGROUND: Adherence to cardioprotective medications in the year after acute coronary syndrome hospitalization is generally poor and is associated with increased risk of rehospitalization and mortality. Few interventions have specifically targeted this high-risk patient population to improve medication adherence. We hypothesize that a multifaceted patient-centered intervention could improve adherence to cardioprotective medications. METHODS AND RESULTS: To evaluate this intervention, we propose enrolling 280 patients with a recent acute coronary syndrome event into a multicenter randomized, controlled trial. The intervention comprises 4 main components: (1) pharmacist-led medication reconciliation and tailoring; (2) patient education; (3) collaborative care between pharmacist and primary care provider/cardiologist; and (4) 2 types of voice messaging (educational and medication refill reminder calls). Patients in the intervention arm will visit with the study pharmacist ≈1 week post-hospital discharge. The pharmacist will work with the patient and collaborate with providers to reconcile medication issues. Voice messages will augment the educational process and remind patients to refill their cardioprotective medications. The study will compare the intervention versus usual care for 12 months. The primary outcome of interest is adherence using the ReComp method. Secondary and tertiary outcomes include achievement of targets for blood pressure and low-density lipoprotein, and reduction in the combined cardiovascular end points of myocardial infarction hospitalization, coronary revascularization, and all-cause mortality. Finally, we will also evaluate the cost-effectiveness of the intervention compared with usual care. CONCLUSIONS: If the intervention is effective in improving medication adherence and demonstrating a lower cost, the intervention has the potential to improve cardiovascular outcomes in this high-risk patient population.