Implementation of Entrustable Professional Activities in General Surgery: Results of a National Pilot Study.

OBJECTIVE: The ongoing complexity of general surgery training has led to an increased focus on ensuring the competence of graduating residents. Entrustable professional activities (EPAs) are units of professional practice that provide an assessment framework to drive competency-based education. The American Board of Surgery convened a group from the American College of Surgeons, Accreditation Council for Graduate Medical Education (ACGME) Surgery Review Committee, and Association of Program Directors in Surgery to develop and implement EPAs in a pilot group of residency programs across the country. The objective of this pilot study was to determine the feasibility and utility of EPAs in general surgery resident training. METHODS: 5 EPAs were chosen based on the most common procedures reported in ACGME case logs and by practicing general surgeons (right lower quadrant pain, biliary disease, inguinal hernia), along with common activities covering additional ACGME milestones (performing a consult, care of a trauma patient). Levels of entrustment assigned (1 to 5) were observation only, direct supervision, indirect supervision, unsupervised, and teaching others. Participating in site recruitment and faculty development occurred from 2017 to 2018. EPA implementation at individual residency programs began on July 1, 2018, and was completed on June 30, 2020. Each site was assigned 2 EPAs to implement and collected EPA microassessments on residents for those EPAs. The site clinical competency committees (CCC) used these microassessments to make summative entrustment decisions. Data submitted to the independent deidentified data repository every 6 months included the number of microassessments collected per resident per EPA and CCC summative entrustment decisions. RESULTS: Twenty-eight sites were selected to participate in the program and represented geographic and size variability, community, and university-based programs. Over the course of the 2-year pilot programs reported on 14 to 180 residents. Overall, 6,272 formative microassessments were collected (range, 0 to 1144 per site). Each resident had between 0 and 184 microassessments. The mean number of microassessments per resident was 5.6 (SD = 13.4) with a median of 1 [interquartile range (IQR) = 6]. There were 1,763 summative entrustment ratings assigned to 497 unique residents. The average number of observations for entrustment was 3.24 (SD 3.61) with a median of 2 (IQR 3). In general, PGY1 residents were entrusted at the level of direct supervision and PGY5 residents were entrusted at unsupervised practice or teaching others. For each EPA other than the consult EPA, the degree of entrustment reported by the CCC increased by resident level. CONCLUSIONS: These data provide evidence that widespread implementation of EPAs across general surgery programs is possible, but variable. They provide meaningful data that graduating chief residents are entrusted by their faculty to perform without supervision for several common general surgical procedures and highlight areas to target for the successful widespread implementation of EPAs.

Objective predictors of intern performance.

BACKGROUND: Residency programs select medical students for interviews and employment using metrics such as the United States Medical Licensing Examination (USMLE) scores, grade-point average (GPA), and class rank/quartile. It is unclear whether these metrics predict performance as an intern. This study tested the hypothesis that performance on these metrics would predict intern performance. METHODS: This single institution, retrospective cohort analysis included 244 graduates from four classes (2015-2018) who completed an Accreditation Council for Graduate Medical Education (ACGME) certified internship and were evaluated by program directors (PDs) at the end of the year. PDs provided a global assessment rating and ratings addressing ACGME competencies (response rate = 47%) with five response options: excellent = 5, very good = 4, acceptable = 3, marginal = 2, unacceptable = 1. PDs also classified interns as outstanding = 4, above average = 3, average = 2, and below average = 1 relative to other interns from the same residency program. Mean USMLE scores (Step 1 and Step 2CK), third-year GPA, class rank, and core competency ratings were compared using Welch's ANOVA and follow-up pairwise t-tests. RESULTS: Better performance on PD evaluations at the end of intern year was associated with higher USMLE Step 1 (p = 0.006), Step 2CK (p = 0.030), medical school GPA (p = 0.020) and class rank (p = 0.016). Interns rated as average had lower USMLE scores, GPA, and class rank than those rated as above average or outstanding; there were no significant differences between above average and outstanding interns. Higher rating in each of the ACGME core competencies was associated with better intern performance (p < 0.01). CONCLUSIONS: Better performance as an intern was associated with higher USMLE scores, medical school GPA and class rank. When USMLE Step 1 reporting changes from numeric scores to pass/fail, residency programs can use other metrics to select medical students for interviews and employment.

Proton Pump Inhibitor Use After Hiatal Hernia Repair: Inhibitor of Recurrent Symptoms and Potential Revisional Surgery.

BACKGROUND: Hiatal hernia recurrence after hiatal hernia repair (HHR) is often underdiagnosed and underreported but may present with recurrent gastroesophageal reflux disease (GERD) symptoms. Because of their availability, proton pump inhibitor (PPI) use is common and may mask patients who would benefit from revisional surgery, which has been shown to improve symptoms and quality of life. METHODS: A retrospective analysis was performed to evaluate recurrence patterns of patients who underwent HHR, specifically for the indication of GERD, from 2007 to 2015 at a single Veterans Administration Medical Center. Clinicopathologic parameters were reviewed for association with hiatal hernia recurrence, including postoperative PPI use. RESULTS: Sixty-four patients were identified with a median follow-up time of 57.8 mo. Thirty-eight patients developed an anatomic recurrence, which did not demonstrate any associated factors on univariate analysis. Seventy percent of patients remained or were restarted on PPI after their initial surgery. For patients with a documented recurrence, the median time to start a PPI was 224 d, but the time to identify recurrence on imaging or endoscopy was 712.5 d. Eleven (39.3%) patients had a reintervention for anatomic recurrence, of which all had developed recurrent symptoms of GERD. CONCLUSIONS: Most patients who developed recurrent hiatal hernia were restarted on PPI without workup for their symptoms. The time of initiation of PPI was much earlier than the time of identification of a recurrent hiatal hernia. The use of PPIs in patients whom have undergone HHR may delay proper workup to identify recurrent hiatal hernia amenable to surgical repair and should be reserved until patients develop recurrent symptoms and have at least begun a diagnostic workup to rule out an anatomic cause for the recurrent symptoms.

Implementing Survey-Based Changes and Analyzing Usage in an Online Curriculum for Surgical Interns.

BACKGROUND: To improve the online curriculum at the authors' program, study habits and learning resources of surgical interns were evaluated. Based on the results, changes were implemented to align with their teaching conferences. We sought to determine utilization and satisfaction with the new materials. METHODS: At the end of the 2017 academic year, surgical interns at a single institution voluntarily responded to an Institutional Review Board-approved survey regarding new and established learning material use and study habits. Responses were deidentified. Descriptive statistics were performed on demographics. Likert responses underwent Mann-Whitney analysis (α = 0.05). RESULTS: The response rate was 52.9% (n = 9). The internet was the most-used resource (P < 0.05). All respondents used the internet to varying degrees. Textbooks and the internet were always used by 22.2%. 33.3% never used textbooks to study. There was a statistically significant increase in internet access but not with other materials (P < 0.05). Regarding new material organized from the pilot, none was consistently accessed. 55.6% sometimes used new weekly reading links associated with a question bank. 66.7% were somewhat satisfied with these links. 44.4% were somewhat satisfied with new video links, organized in parallel with This Week in Surgical Council on Resident Education. Limited sample size did not allow for meaningful statistical analysis of material use with American Board of Surgery In-Training Examination scores. CONCLUSIONS: Overall, materials organized based on pilot study feedback were not widely used, and satisfaction was limited. The statistically significant increase in use of internet resources warrants attention. Focusing changes in future curricular design may help create a more effective learning environment.

Associations Between National Board Exam Performance and Residency Program Emphasis on Patient Safety and Interprofessional Teamwork.

OBJECTIVE: Several aspects of medical training may contribute to the ultimate goal of producing excellent physicians whose patients will have the best possible outcomes. However, the relative importance of education, evaluation and feedback, duty hours, practice structure, and program culture in achieving this goal is unclear. This study assessed associations among in-training exam performance, Accreditation Council for Graduate Medical Education (ACGME) Resident Survey responses, and American Board of Medical Specialties (ABMS) national board exam performance. METHODS: Residency training programs at a university teaching hospital were classified as having 5-year first-time ABMS pass rates above (n=12) or below (n=3) the national average for their specialty. These groups were compared by ACGME Resident Survey data and in-training exam performance. RESULTS: Surveys were collected from 484/543 eligible residents (89%), including 177 surveys from programs with below-average board pass rates and 307 surveys from programs with aboveaverage board pass rates. In-training exam performance was similar between groups. Aboveaverage programs had stronger agreement with statements that their culture reinforced patient safety (4.72 vs. 4.30, p=0.006) and that information was not lost during transitions of care (4.14 vs. 3.63, p=0.001). Although the occurrence of interprofessional teamwork was similar between groups, above-average programs had stronger agreement with the statement that interprofessional teamwork was effective (4.60 vs. 4.17, p=0.003). CONCLUSION: Residency programs emphasizing patient safety and effective interprofessional teamwork had above-average first-time national board pass rates.

Postoperative surgical trainee opioid prescribing practices (POST OPP): an institutional study.

BACKGROUND: Increasing mortality from opioid overdoses has prompted increased focus on prescribing practices of physicians. Unfortunately, resident physicians rarely receive formal education in effective opioid prescribing practices or postoperative pain management. Data to inform surgical training programs regarding the utility and feasibility of formal training are lacking. METHODS: Following Institutional Review Board approval, a single institution's resident physicians who had completed at least one surgical rotation were surveyed to assess knowledge of pain management and evaluate opioid prescribing practices. RESULTS: Fifty-three respondents (68% males and 32% females) completed the survey. Most respondents denied receiving formal instruction in opioid pain medication prescribing practices during either medical school (62.3%) or residency (56.6%); however, nearly all respondents stated they were aware of the side effects of opioid pain medications, and a majority felt confident in their knowledge of opioid pharmacokinetics and pharmacodynamics. Of the respondents, 47% either "agreed" or "strongly agreed" that they prescribed more opioid medications than necessary to patients being discharged following a surgical procedure. Individual case scenario responses demonstrated variability in the number of morphine milligram equivalents prescribed across scenarios (P < 0.001). Male and nonsurgical specialty respondents reported prescribing significantly fewer overall morphine milligram equivalents in these scenarios. CONCLUSIONS: This pilot study shows wide variability in opioid prescribing practices and attitudes toward pain management among surgical trainees, illustrating the potential utility of formal education in pain management and effective prescribing of these medications. A broader assessment of surgical trainees' knowledge and perception of opioid prescribing practices is warranted to facilitate the development of such a program.

Percutaneous cholecystostomy: prognostic factors and comparison to cholecystectomy.

BACKGROUND: Data regarding long-term outcomes following percutaneous cholecystostomy (PC) are limited, and comparisons to cholecystectomy (CCY) are lacking. We hypothesized that chronic disease burden would predict 1-year mortality following PC, and that outcomes following PC and CCY would be similar when controlling for preprocedural risk factors. METHODS: We performed a 10-year retrospective cohort analysis of patients with acute cholecystitis managed by PC (n = 114) or CCY (n = 234). Treatment response was assessed by systemic inflammatory response syndrome (SIRS) criteria at PC/CCY and 72 h later. Logistic regression identified predictors of 30-day and 1-year mortality following PC. PC and CCY patients were matched by age, Tokyo Guidelines (TG13) cholecystitis severity grade, and VASQIP calculator predicted mortality (n = 42/group). RESULTS: The presence of SIRS at 72 h following PC was associated with 30-day mortality [OR 8.9 (95% CI 2.6-30)]. SIRS at 72 h was present in and 21.4% of all PC patients, significantly higher than unmatched CCY patients (4.7%, p = 0.048). Independent predictors of 1-year mortality following PC were DNR status [19.7 (2.1-186)], disseminated cancer [7.5 (2.1-26)], and congestive heart failure [3.9 (1.4-11)]. PC patients with none of these risk factors had 17.9% 90-day mortality and no deaths after 90 days; late deaths continued to occur among patients with DNR, CHF, or disseminated cancer. At baseline, PC patients had greater acute and chronic disease burden than CCY patients. After matching, PC and CCY patients had similar age (69 vs. 70 years), TG13 grade (2.4 vs. 2.4), and predicted 30-day mortality (5.5 vs. 6.8%). Matched PC patients had higher 30-day mortality (14.3 vs. 2.4%, p = 0.109) and 180-day mortality (28.6 vs. 7.1%, p = 0.048). CONCLUSIONS: Treatment response to PC predicted 30-day mortality; DNR status, and chronic diseases predicted 1-year mortality. Although the matching procedure did not eliminate selection bias, PC was associated with persistent systemic inflammation and higher long-term mortality than CCY.

Antibiotics May be Safely Discontinued Within One Week of Percutaneous Cholecystostomy.

BACKGROUND: For patients with acute cholecystitis managed with percutaneous cholecystostomy (PC), the optimal duration of post-procedural antibiotic therapy is unknown. Our objective was to compare short versus long courses of antibiotics with the hypothesis that patients with persistent signs of systemic inflammation 72 h following PC would receive prolonged antibiotic therapy and that antibiotic duration would not affect outcomes. METHODS: We performed a retrospective cohort analysis of 81 patients who underwent PC for acute cholecystitis at two hospitals during a 41-month period ending November 2014. Patients who received short (≤7 day) courses of post-procedural antibiotics were compared to patients who received long (>7 day) courses. Treatment response to PC was evaluated by systemic inflammatory response syndrome (SIRS) criteria. Logistic and linear regressions were used to evaluate associations between antibiotic duration and outcomes. RESULTS: Patients who received short (n = 30) and long courses (n = 51) of antibiotics had similar age, comorbidities, severity of cholecystitis, pre-procedural vital signs, treatment response, and culture results. There were no differences in recurrent cholecystitis (13 vs. 12%), requirement for open/converted to open cholecystectomy (23 vs. 22%), or 1-year mortality (20 vs. 18%). On logistic and linear regressions, antibiotic duration as a continuous variable was not predictive of any salient outcomes. CONCLUSIONS: Patients who received short and long courses of post-PC antibiotics had similar baseline characteristics and outcomes. Antibiotic duration did not predict recurrent cholecystitis, interval open cholecystectomy, or mortality. These findings suggest that antibiotics may be safely discontinued within one week of uncomplicated PC.

Patient-derived xenograft models for pancreatic adenocarcinoma demonstrate retention of tumor morphology through incorporation of murine stromal elements.

Direct implantation of viable surgical specimens provides a representative preclinical platform in pancreatic adenocarcinoma. Patient-derived xenografts consistently demonstrate retained tumor morphology and genetic stability. However, the evolution of the tumor microenvironment over time remains poorly characterized in these models. This work specifically addresses the recruitment and incorporation of murine stromal elements into expanding patient-derived pancreatic adenocarcinoma xenografts, establishing the integration of murine cells into networks of invading cancer cells. In addition, we provide methods and observations in the establishment and maintenance of a patient-derived pancreatic adenocarcinoma xenograft model. A total of 25 histologically confirmed pancreatic adenocarcinoma specimens were implanted subcutaneously into nonobese diabetic severe combined immunodeficiency mice. Patient demographics, staging, pathological analysis, and outcomes were analyzed. After successful engraftment of tumors, histological and immunofluorescence analyses were performed on explanted tumors. Pancreatic adenocarcinoma specimens were successfully engrafted in 15 (60%) of 25 attempts. Successful engraftment does not appear to correlate with clinicopathologic factors or patient survival. Tumor morphology is conserved through multiple passages, and tumors retain metastatic potential. Interestingly, despite morphological similarity between passages, human stromal elements do not appear to expand with invading cancer cells. Rather, desmoplastic murine stroma dominates the xenograft microenvironment after the initial implantation. Recruitment of stromal elements in this manner to support and maintain tumor growth represents a novel avenue for investigation into tumor-stromal interactions.

Seroprevalence of histoplasmosis in Kampala, Uganda.

Histoplasmosis is endemic to the Midwestern United States, but cases have been reported nearly worldwide. A 1970 study found 3.8% skin test sensitivity to Histoplasma capsulatum in Uganda but no systemic study of histoplasmosis exposure has occurred since the onset of the human immunodeficiency virus (HIV) pandemic. This study investigated the seroprevalence of H. capsulatum and sought previously undetected cases of histoplasmosis in Kampala, Uganda. Serum, cerebrospinal fluid (CSF) and/or urine specimens were obtained from HIV-infected persons with suspected meningitis. Specimens were tested for H. capsulatum IgG and IgM by enzyme immune assay and Histoplasma antigen. 147 of the 257 subjects who were enrolled had cryptococcal meningitis. Overall, 1.3% (2/151) of subjects were serum Histoplasma IgG positive, and zero of 151 were IgM positive. Antigen was not detected in any serum (n = 57), urine (n = 37, or CSF (n = 63) samples. Both subjects with serum Histoplasma IgG positivity had cryptococcal meningitis. Histoplasma capsulatum IgG was detected at low levels in persons with HIV/AIDS in Kampala, Uganda. Histoplasmosis is not widespread in Uganda but microfoci do exist. There appears to be no cross-reactivity between Cryptococcus neoformans and Histoplasma antigen testing, and cryptococcosis appears to be at most, a rare cause of positive Histoplasma IgG.

Primary outgrowth cultures are a reliable source of human pancreatic stellate cells.

Recent advances demonstrate a critical yet poorly understood role for the pancreatic stellate cell (PSC) in the pathogenesis of chronic pancreatitis (CP) and pancreatic cancer (PC). Progress in this area has been hampered by the availability, fidelity, and/or reliability of in vitro models of PSCs. We examined whether outgrowth cultures from human surgical specimens exhibited reproducible phenotypic and functional characteristics of PSCs. PSCs were cultured from surgical specimens of healthy pancreas, CP and PC. Growth dynamics, phenotypic characteristics, soluble mediator secretion profiles and co-culture with PC cells both in vitro and in vivo were assessed. Forty-seven primary cultures were established from 52 attempts, demonstrating universal α-smooth muscle actin and glial fibrillary acidic protein but negligible epithelial surface antigen expression. Modification of culture conditions consistently led to cytoplasmic lipid accumulation, suggesting induction of a quiescent phenotype. Secretion of growth factors, chemokines and cytokines did not significantly differ between donor pathologies, but did evolve over time in culture. Co-culture of PSCs with established PC cell lines resulted in significant changes in levels of multiple secreted mediators. Primary PSCs co-inoculated with PC cells in a xenograft model led to augmented tumor growth and metastasis. Therefore, regardless of donor pathology, outgrowth cultures produce PSCs that demonstrate consistent growth and protein secretion properties. Primary cultures from pancreatic surgical specimens, including malignancies, may represent a reliable source of human PSCs.

The inflammatory milieu within the pancreatic cancer microenvironment correlates with clinicopathologic parameters, chemoresistance and survival.

BACKGROUND: The tumor microenvironment impacts pancreatic cancer (PC) development, progression and metastasis. How intratumoral inflammatory mediators modulate this biology remains poorly understood. We hypothesized that the inflammatory milieu within the PC microenvironment would correlate with clinicopathologic findings and survival. METHODS: Pancreatic specimens from normal pancreas (n = 6), chronic pancreatitis (n = 9) and pancreatic adenocarcinoma (n = 36) were homogenized immediately upon resection. Homogenates were subjected to multiplex analysis of 41 inflammatory mediators. RESULTS: Twenty-three mediators were significantly elevated in adenocarcinoma specimens compared to nonmalignant controls. Increased intratumoral IL-8 concentrations associated with larger tumors (P = .045) and poor differentiation (P = .038); the administration of neoadjuvant chemotherapy associated with reduced IL-8 concentrations (P = .003). Neoadjuvant therapy was also associated with elevated concentrations of Flt-3 L (P = .005). Elevated levels of pro-inflammatory cytokines IL-1ß (P = .017) and TNFα (P = .033) were associated with a poor histopathologic response to neoadjuvant therapy. Elevated concentrations of G-CSF (P = .016) and PDGF-AA (P = .012) correlated with reduced overall survival. Conversely, elevated concentrations of FGF-2 (P = .038), TNFα (P = .031) and MIP-1α (P = .036) were associated with prolonged survival. CONCLUSION: The pancreatic cancer microenvironment harbors a unique inflammatory milieu with potential diagnostic and prognostic value.

Resident and Fellow Performance and Supervision in Surgical Oncology Procedures.

BACKGROUND: Previous research has highlighted concerns among trainees and attendings that general surgery training and fellowship are inadequately preparing trainees for practice. Providing trainees with supervision that matches their proficiency may help bridge this gap. We sought to benchmark operative performance and supervision levels among senior surgery residents (post-graduate year 4 or 5) and fellows performing general surgical oncology procedures. STUDY DESIGN: Observational data were obtained from the Society for Improving Medical Procedural Learning (SIMPL) OR application for core general surgical oncology procedures performed at 103 unique residency and fellowship programs. Procedures were divided into breast and soft tissue, endocrine, and hepatopancreatobiliary (HPB). Case evaluations completed by trainees and attendings were analyzed to benchmark trainee operative performance and level of supervision. RESULTS: There were 4,907 resident cases and 425 fellow cases. Practice-ready performance, as assessed by trainees and faculty, was achieved by relatively low proportions of residents and fellows for breast and soft tissue cases (residents: 38%, fellows: 48%), endocrine cases (residents: 22%, fellows: 41%), and HPB cases (residents: 10%, fellows: 40%). Among cases in which trainees did achieve practice-ready performance, supervision only was provided for low proportions of cases as rated by trainees (residents: 17%, fellows: 18%) and attendings (residents: 21%, fellows 25%). CONCLUSION: In a sample of 103 residency and fellowship programs, attending surgeons rarely provided senior residents and fellows with levels of supervision commensurate to performance for surgical oncology procedures, even for high performing trainees. These findings suggest a critical need for surgical training programs to prioritize providing greater levels of independence to trainees that have demonstrated excellent performance.

Operating Room Supply Cost and Value of Care after Implementing a Sustainable Quality Intervention.

BACKGROUND: Variability in operating room supply cost is a modifiable cause of suboptimal resource use and low value of care (outcomes vs cost). This study describes implementation of a quality improvement intervention to decrease operating room supply costs. STUDY DESIGN: An automated electronic health record data pipeline harmonized operating room supply cost data with patient and case characteristics and outcomes. For inpatient procedures, predicted mortality and length of stay were used to calculate observed-to-expected ratios and value of care using validated equations. For commonly performed (1 or more per week) procedures, the pipeline generated figures illustrating individual surgeon performance vs peers, costs for each surgeon performing each case type, and control charts identifying out-of-control cases and surgeons with more than 90th percentile costs, which were shared with surgeons and division chiefs alongside guidance for modifying case-specific supply instructions to operating room nurses and technicians. RESULTS: Preintervention control (1,064 cases for 7 months) and postintervention (307 cases for 2 months) cohorts had similar baseline characteristics across all 16 commonly performed procedures. Median costs per case were lower in the intervention cohort ($811 [$525 to $1,367] vs controls: $1,080 [$603 to $1,574], p < 0.001), as was the incidence of out-of-control cases (19 (6.2%) vs 110 (10.3%), p = 0.03). Duration of surgery, length of stay, discharge disposition, and 30-day mortality and readmission rates were similar between cohorts. Value of care was higher in the intervention cohort (1.1 [0.1 to 1.5] vs 1.0 [0.2 to 1.4], p = 0.04). Pipeline runtime was 16:07. CONCLUSIONS: An automated, sustainable quality improvement intervention was associated with decreased operating room supply costs and increased value of care.

Decision-Making Confidence of Clinical Competency Committees for Entrustable Professional Activities.

Importance: A competency-based assessment framework using entrustable professional activities (EPAs) was endorsed by the American Board of Surgery following a 2-year feasibility pilot study. Pilot study programs' clinical competency committees (CCCs) rated residents on EPA entrustment semiannually using this newly developed assessment tool, but factors associated with their decision-making are not yet known. Objective: To identify factors associated with variation in decision-making confidence of CCCs in EPA summative entrustment decisions. Design, Setting, and Participants: This cohort study used deidentified data from the EPA Pilot Study, with participating sites at 28 general surgery residency programs, prospectively collected from July 1, 2018, to June 30, 2020. Data were analyzed from September 27, 2022, to February 15, 2023. Exposure: Microassessments of resident entrustment for pilot EPAs (gallbladder disease, inguinal hernia, right lower quadrant pain, trauma, and consultation) collected within the course of routine clinical care across four 6-month study cycles. Summative entrustment ratings were then determined by program CCCs for each study cycle. Main Outcomes and Measures: The primary outcome was CCC decision-making confidence rating (high, moderate, slight, or no confidence) for summative entrustment decisions, with a secondary outcome of number of EPA microassessments received per summative entrustment decision. Bivariate tests and mixed-effects regression modeling were used to evaluate factors associated with CCC confidence. Results: Among 565 residents receiving at least 1 EPA microassessment, 1765 summative entrustment decisions were reported. Overall, 72.5% (1279 of 1765) of summative entrustment decisions were made with moderate or high confidence. Confidence ratings increased with increasing mean number of EPA microassessments, with 1.7 (95% CI, 1.4-2.0) at no confidence, 1.9 (95% CI, 1.7-2.1) at slight confidence, 2.9 (95% CI, 2.6-3.2) at moderate confidence, and 4.1 (95% CI, 3.8-4.4) at high confidence. Increasing number of EPA microassessments was associated with increased likelihood of higher CCC confidence for all except 1 EPA phase after controlling for program effects (odds ratio range: 1.21 [95% CI, 1.07-1.37] for intraoperative EPA-4 to 2.93 [95% CI, 1.64-5.85] for postoperative EPA-2); for preoperative EPA-3, there was no association. Conclusions and Relevance: In this cohort study, the CCC confidence in EPA summative entrustment decisions increased as the number of EPA microassessments increased, and CCCs endorsed moderate to high confidence in most entrustment decisions. These findings provide early validity evidence for this novel assessment framework and may inform program practices as EPAs are implemented nationally.

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs.

BACKGROUND: The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. STUDY DESIGN: Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman's correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. RESULTS: A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). CONCLUSIONS: Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.

Resident Operative Autonomy and Attending Verbal Feedback Differ by Resident and Attending Gender.

Objectives: This study tests the null hypotheses that overall sentiment and gendered words in verbal feedback and resident operative autonomy relative to performance are similar for female and male residents. Background: Female and male surgical residents may experience training differently, affecting the quality of learning and graduated autonomy. Methods: A longitudinal, observational study using a Society for Improving Medical Professional Learning collaborative dataset describing resident and attending evaluations of resident operative performance and autonomy and recordings of verbal feedback from attendings from surgical procedures performed at 54 US general surgery residency training programs from 2016 to 2021. Overall sentiment, adjectives, and gendered words in verbal feedback were quantified by natural language processing. Resident operative autonomy and performance, as evaluated by attendings, were reported on 5-point ordinal scales. Performance-adjusted autonomy was calculated as autonomy minus performance. Results: The final dataset included objective assessments and dictated feedback for 2683 surgical procedures. Sentiment scores were higher for female residents (95 [interquartile range (IQR), 4-100] vs 86 [IQR 2-100]; P < 0.001). Gendered words were present in a greater proportion of dictations for female residents (29% vs 25%; P = 0.04) due to male attendings disproportionately using male-associated words in feedback for female residents (28% vs 23%; P = 0.01). Overall, attendings reported that male residents received greater performance-adjusted autonomy compared with female residents (P < 0.001). Conclusions: Sentiment and gendered words in verbal feedback and performance-adjusted operative autonomy differed for female and male general surgery residents. These findings suggest a need to ensure that trainees are given appropriate and equitable operative autonomy and feedback.