Decision-Making Confidence of Clinical Competency Committees for Entrustable Professional Activities.

Importance: A competency-based assessment framework using entrustable professional activities (EPAs) was endorsed by the American Board of Surgery following a 2-year feasibility pilot study. Pilot study programs' clinical competency committees (CCCs) rated residents on EPA entrustment semiannually using this newly developed assessment tool, but factors associated with their decision-making are not yet known. Objective: To identify factors associated with variation in decision-making confidence of CCCs in EPA summative entrustment decisions. Design, Setting, and Participants: This cohort study used deidentified data from the EPA Pilot Study, with participating sites at 28 general surgery residency programs, prospectively collected from July 1, 2018, to June 30, 2020. Data were analyzed from September 27, 2022, to February 15, 2023. Exposure: Microassessments of resident entrustment for pilot EPAs (gallbladder disease, inguinal hernia, right lower quadrant pain, trauma, and consultation) collected within the course of routine clinical care across four 6-month study cycles. Summative entrustment ratings were then determined by program CCCs for each study cycle. Main Outcomes and Measures: The primary outcome was CCC decision-making confidence rating (high, moderate, slight, or no confidence) for summative entrustment decisions, with a secondary outcome of number of EPA microassessments received per summative entrustment decision. Bivariate tests and mixed-effects regression modeling were used to evaluate factors associated with CCC confidence. Results: Among 565 residents receiving at least 1 EPA microassessment, 1765 summative entrustment decisions were reported. Overall, 72.5% (1279 of 1765) of summative entrustment decisions were made with moderate or high confidence. Confidence ratings increased with increasing mean number of EPA microassessments, with 1.7 (95% CI, 1.4-2.0) at no confidence, 1.9 (95% CI, 1.7-2.1) at slight confidence, 2.9 (95% CI, 2.6-3.2) at moderate confidence, and 4.1 (95% CI, 3.8-4.4) at high confidence. Increasing number of EPA microassessments was associated with increased likelihood of higher CCC confidence for all except 1 EPA phase after controlling for program effects (odds ratio range: 1.21 [95% CI, 1.07-1.37] for intraoperative EPA-4 to 2.93 [95% CI, 1.64-5.85] for postoperative EPA-2); for preoperative EPA-3, there was no association. Conclusions and Relevance: In this cohort study, the CCC confidence in EPA summative entrustment decisions increased as the number of EPA microassessments increased, and CCCs endorsed moderate to high confidence in most entrustment decisions. These findings provide early validity evidence for this novel assessment framework and may inform program practices as EPAs are implemented nationally.

Entrustable Professional Activities in Surgery: A Review.

Importance: Entrustable professional activities (EPAs) compose a competency-based education (CBE) assessment framework that has been increasingly adopted across medical specialties as a workplace-based assessment tool. EPAs focus on directly observed behaviors to determine the level of entrustment a trainee has for a given activity of that specialty. In this narrative review, we highlight the rationale for EPAs in general surgery, describe current evidence supporting their use, and outline some of the practical considerations for EPAs among residency programs, faculty, and trainees. Observations: An expanding evidence base for EPAs in general surgery has provided moderate validity evidence for their use as well as practical recommendations for implementation across residency programs. Challenges to EPA use include garnering buy-in from individual faculty and residents to complete EPA microassessments and engage in timely, specific feedback after a case or clinical encounter. When successfully integrated into a program's workflow, EPAs can provide a more accurate picture of residents' competence for a fundamental surgical task or activity compared with other assessment methods. Conclusions and Relevance: EPAs represent the next significant shift in the evaluation of general surgery residents as part of the overarching progression toward CBE among all US residency programs. While pragmatic challenges to the implementation of EPAs remain, the best practices from EPA and other CBE assessment literature summarized in this review may assist individuals and programs in implementing EPAs. As EPAs become more widely used in general surgery resident training, further analysis of barriers and facilitators to successful and sustainable EPA implementation will be needed to continue to optimize and advance this new assessment framework.

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs.

BACKGROUND: The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. STUDY DESIGN: Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman's correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. RESULTS: A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). CONCLUSIONS: Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.

Surgical Education.

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

Implementation of Entrustable Professional Activities in General Surgery: Results of a National Pilot Study.

OBJECTIVE: The ongoing complexity of general surgery training has led to an increased focus on ensuring the competence of graduating residents. Entrustable professional activities (EPAs) are units of professional practice that provide an assessment framework to drive competency-based education. The American Board of Surgery convened a group from the American College of Surgeons, Accreditation Council for Graduate Medical Education (ACGME) Surgery Review Committee, and Association of Program Directors in Surgery to develop and implement EPAs in a pilot group of residency programs across the country. The objective of this pilot study was to determine the feasibility and utility of EPAs in general surgery resident training. METHODS: 5 EPAs were chosen based on the most common procedures reported in ACGME case logs and by practicing general surgeons (right lower quadrant pain, biliary disease, inguinal hernia), along with common activities covering additional ACGME milestones (performing a consult, care of a trauma patient). Levels of entrustment assigned (1 to 5) were observation only, direct supervision, indirect supervision, unsupervised, and teaching others. Participating in site recruitment and faculty development occurred from 2017 to 2018. EPA implementation at individual residency programs began on July 1, 2018, and was completed on June 30, 2020. Each site was assigned 2 EPAs to implement and collected EPA microassessments on residents for those EPAs. The site clinical competency committees (CCC) used these microassessments to make summative entrustment decisions. Data submitted to the independent deidentified data repository every 6 months included the number of microassessments collected per resident per EPA and CCC summative entrustment decisions. RESULTS: Twenty-eight sites were selected to participate in the program and represented geographic and size variability, community, and university-based programs. Over the course of the 2-year pilot programs reported on 14 to 180 residents. Overall, 6,272 formative microassessments were collected (range, 0 to 1144 per site). Each resident had between 0 and 184 microassessments. The mean number of microassessments per resident was 5.6 (SD = 13.4) with a median of 1 [interquartile range (IQR) = 6]. There were 1,763 summative entrustment ratings assigned to 497 unique residents. The average number of observations for entrustment was 3.24 (SD 3.61) with a median of 2 (IQR 3). In general, PGY1 residents were entrusted at the level of direct supervision and PGY5 residents were entrusted at unsupervised practice or teaching others. For each EPA other than the consult EPA, the degree of entrustment reported by the CCC increased by resident level. CONCLUSIONS: These data provide evidence that widespread implementation of EPAs across general surgery programs is possible, but variable. They provide meaningful data that graduating chief residents are entrusted by their faculty to perform without supervision for several common general surgical procedures and highlight areas to target for the successful widespread implementation of EPAs.

Impact of Endocrinologist and Surgeon Density on Well-Differentiated Thyroid Cancer Survival.

BACKGROUND: The influence of provider density and access on well-differentiated thyroid cancer (DTC) survival is uncertain. METHODS: We used the SEER18 database to study DTC patients undergoing surgery from 2000-2012. County-level endocrinologist and surgeon density were calculated. We evaluated the relationship between provider density and cause-specific survival controlling for demographic, socioeconomic, and treatment characteristics. RESULTS: Median endocrinologist density was 1.4/100 000 residents, with 15.5% of patients living in a county with no endocrinologist. Survival increased by 11% for each endocrinologist/100,000 people (P = .007). Median surgeon density was 14.8/100 000 residents, with only 1.7% of patients living in a county with no general surgeon or otolaryngologist. No significant association between surgeon density and survival was identified (P = .06). Rural residence was independently associated with lower survival (P = .009). CONCLUSIONS: County-level endocrinologist density is associated with improved DTC survival. Results may reflect endocrinologist expertise, earlier diagnosis, or represent surrogacy for higher county-level access to specialized care.

The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments.

OBJECTIVE: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs. SUMMARY OF BACKGROUND DATA: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares. METHODS: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis. RESULTS: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001). CONCLUSIONS: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.

Career Advancement for Surgeon-Educators:Findings from a Modified Delphi Process.

OBJECTIVES: Reward and recognition of surgical education as an academic activity remains a highly variable process between institutions. The goal of this study is to provide expert consensus definition of an academic surgical educator, with focus on criteria for academic promotion. STUDY DESIGN AND SETTING: Following IRB approval, a Web-based modified Delphi process was used to generate prioritized academic promotion criteria for surgical educators. PARTICIPANTS AND SETTING: Participants were recruited nationally from a pool of senior academic surgeons who are members of the Society of University Surgeons and the Society of Surgical Chairs. RESULTS: Following a three-round modified Delphi process, the top domains of educational activity for promotion to associate professor and professor were scholarship, teaching, and administration; mentorship was also a priority category for promotion to professor. The top three activities described for promotion to Associate Professor were active participation in conferences/ departmental educational activities for medical students and residents; educational portfolio demonstrating commitment to activities as an educator; and clinical teaching excellence at their home institution. The three activities most highly scored items for promotion to Professor were mentorship of junior surgical educators; active participation in conferences/ departmental educational activities for medical students and residents; and a record of teaching excellence at the medical student and resident levels. CONCLUSIONS: These findings demonstrate a progression from teacher to scholar to leader across a surgical educator's career, with each level incorporating and building upon the prior activities. Identification of categories and criteria may meaningfully inform best practices to be incorporated into the career development and promotion processes for surgeons on an educator academic pathway.

Association of socioeconomic status with 30- and 90-day readmission following open and laparoscopic hernia repair: a nationwide readmissions database analysis.

BACKGROUND: Socioeconomic disparities have been associated with outcomes in many medical conditions. The association of socioeconomic status (SES) with readmissions after ventral and inguinal hernia repair has not been well studied on a national level. This study aims to evaluate the association of SES with readmission as a significant outcome in patients undergoing ventral and inguinal hernia repair. METHODS: A retrospective cohort study was performed evaluating patients undergoing ventral hernia and inguinal hernia repair with 1:1 propensity score matching using the Nationwide Readmissions Database (2016-2017). Both 30- and 90-day readmissions were examined. After matching, a multivariate logistic regression analysis was performed using confounding variables including hospital setting, comorbidities, urgency of repair, sociodemographic status, and payer. Likelihood of readmission was reported in odds ratio form. RESULTS: Readmission rates were 11.56% (24,323 out of 210,381) and 17.94% (30,893 out of 172,210) for 30- and 90-day readmissions, respectively. Patients with Medicaid and in the lower income quartile were more likely to present in an emergent fashion for hernia repair. After matching, a multivariate logistic regression analysis showed socioeconomic status (OR 1.250 and 1.229) was a statistically significant independent predictor of readmission at 30 and 90 days, respectively. Inversely, factors associated with the least likely chance of readmission were a laparoscopic approach (OR 0.646 and 0.641), elective admission (OR 0.824 and 0.779), and care in a teaching hospital (OR 0.784 and 0.798). CONCLUSION: SES is an independent predictor of readmission at 30 and 90 days following open and laparoscopic ventral and inguinal hernia repair. Patients with a lower socioeconomic status were more likely to undergo hernia repair in the emergent setting. Efforts toward mitigating SES disparities by potentially promoting MIS techniques, enhancing access to elective cases, and systematic approaches to perioperative care for this disadvantaged population can potentially enhance overall hernia outcomes.

On pandemics and pivots: a COVID-19 reflection on envisioning the future of medical education.

The required adjustments precipitated by the coronavirus disease 2019 crisis have been challenging, but also represent a critical opportunity for the evolution and potential disruptive and constructive change of medical education. Given that the format of medical education is not fixed, but malleable and in fact must be adaptable to societal needs through ongoing reflexivity, we find ourselves in a potentially transformative learning phase for the field. An Association for Medical Education in Europe ASPIRE Academy group of 18 medical educators from seven countries was formed to consider this opportunity, and identified critical questions for collective reflection on current medical education practices and assumptions, with the attendant challenge to envision the future of medical education. This was achieved through online discussion as well as asynchronous collective reflections by group members. Four major themes and related conclusions arose from this conversation: Why we teach: the humanitarian mission of medicine should be reinforced; what we teach: disaster management, social accountability and embracing an environment of complexity and uncertainty should be the core; how we teach: open pathways to lean medical education and learning by developing learners embedded in a community context; and whom we teach: those willing to take professional responsibility. These collective reflections provide neither fully matured digests of the challenges of our field, nor comprehensive solutions; rather they are offered as a starting point for medical schools to consider as we seek to harness the learning opportunities stimulated by the pandemic.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

OBJECTIVE: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees. SUMMARY BACKGROUND DATA: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession. Current methods of operative skill assessment for surgeons rely heavily on global assessment strategies across a very broad domain of procedures. There is no mechanism to assure technical competence for individual procedures. The science and scalability of operative skill assessment has progressed significantly in recent decades, and can inform a much more meaningful strategy for competency-based assessment of operative skill than has been previously achieved. METHODS: The present article reviews the current status and science of operative skill assessment and proposes a template for competency-based assessment which could be used to update training, accreditation, and certification processes. The proposal is made in reference to general surgery but is more generally applicable to other procedural specialties. RESULTS: Streamlined, routine assessment of every procedure performed by surgical trainees is feasible and would enable a more competency-based educational paradigm. In light of the constraints imposed by both clinical volume and assessment bias, trainees should be expected to become proficient and be measured against a mastery learning standard only for the most important and highest-frequency procedures. For less frequently observed procedures, performance can be compared to a norm-referenced standard and, to provide an overall trajectory of performance, analyzed in aggregate. Key factors in implementing this approach are the number of evaluations, the number of raters, the timeliness of evaluation, and evaluation items. CONCLUSIONS: A competency-based operative skill assessment can be incorporated into surgical training, assessment, and certification. The time has come to develop a systematic approach to this issue as a means of demonstrating professional standards worthy of the public trust.

The COVID-19 reset: lessons from the pandemic on Burnout and the Practice of Surgery.

BACKGROUND: Burnout among physicians is an increasing concern, and surgeons are not immune to this threat. The ongoing COVID-19 pandemic has caused dramatic changes to surgeon workflow, often leading to redeployment to other clinical areas, slowdown and shutdown of elective surgery practices, and an uncertain future of surgical practice in the post-pandemic setting. Paradoxically, for many surgeons who had to prepare for but not immediately care for a major surge, the crisis did allow for reflective opportunities and a resetting of priorities that could serve to mitigate chronic patterns contributory to Burnout. METHODS: SAGES Reimagining the Practice of Surgery task force convened a webinar to discuss lessons learned from the COVID pandemic that may address burnout. RESULTS: Burnout is multifactorial and may vary in cause among different generation/experience groups. Those that report burnout symptoms often complain of lacking purpose or meaning in their work. Although many mechanisms to address Burnout are from a defensive standpoint-including coping mechanisms, problem solving, and identification of a physician having wellness difficulties-offensive mechanisms such as pursuing purpose and meaning and finding joy in one's work can serve as reset points that promote thriving and fulfillment. Understanding what motivates physicians will help physician leaders to develop and sustain effective teams. Reinvigorating the surgical workforce around themes of meaning and joy in the service rendered via our surgical skills may diminish Burnout through generative and aspirational strategies, as opposed to merely reactive ones. Fostering an educational environment free of discriminatory or demeaning behavior may produce a new workforce conducive to enhanced and resilient wellbeing at the start of careers. CONCLUSION: Surgeon wellness and self-care must be considered an important factor in the future of all healthcare delivery systems, a need reaffirmed by the COVID-19 pandemic.