Surgical endoscopy education research: how are we doing?

BACKGROUND: Surgical endoscopy (SE), the official journal of the Society of American Gastrointestinal and Endoscopic Surgeons and the European Association for Endoscopic Surgery, is an important source of new evidence pertaining to surgical education in the field. However, qualitative deficiencies in medical education research have prompted medical education leaders to advocate for increased methodological rigor. The purpose of this study is to review the quality of education-focused research published through SE. METHODS: A PubMed search examining all SE articles categorized as education-related research from 2010 to 2019 was conducted; studies not meeting inclusion criteria were excluded. Remaining publications were independently reviewed, classified, and scored by 7 raters using the medical education research study quality instrument (MERSQI). Intraclass correlation was calculated and data were examined with descriptive statistics. RESULTS: A total of 227 studies met inclusion criteria. There was no significant difference in number of publications by year (average 25.88 [SD 5.6]); 60% were conducted outside of the United States, and 47% (n = 106) were funded. The average MERSQI was 12.5 (SD 2). Most studies used two-group non-random (42%, n = 96) or post/cross-sectional designs (29%, n = 65). Thirty-six (16%) were randomized controlled trials. Multi-institutional studies comprised 24% (n = 54). Of the manuscripts, 96% (n = 217) reported at least one measure of validity evidence and 28% (n = 67) described three levels of validity evidence. Studies primarily reported changes in skills or knowledge (45%, n = 103) or satisfaction or general facts (44%, n = 99), while patient-related outcomes encompassed 3% (n = 6) of studies. ICC between raters was 0.93 (CI 0.90-0.93, p < 0.001). CONCLUSIONS: Based on publications to date, this journal's peer review process appears to facilitate the dissemination of education-related studies of moderate to good quality. However, there were uncovered deficits, ranging from validity evidence to study designs and level of outcomes. This journal's breadth of viewership offers a potential venue to advance education-related research.

The fundamentals of laparoscopic surgery in general surgery residency: fundamental for junior residents' self-efficacy.

BACKGROUND: Implementation of the Fundamentals of Laparoscopic Surgery (FLS) by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) has served a need for educational structure for laparoscopic skill within General Surgery training since 2004. This study looks at how FLS affects resident self-efficacy (SE) with laparoscopic procedures. METHODS: We conducted a national survey, linked to the 2020 American Board of Surgery In-Training Examination (ABSITE), in which 9275 residents from 325 US General Surgery Training Programs participated. The online survey included multimodal questions that analyzed whether participants felt they could perform the most commonly-logged laparoscopic operations among residents [Laparoscopic Appendectomy (LA), Laparoscopic Cholecystectomy (LC), Laparoscopic Right Hemicolectomy (LRH), Diagnostic Laparoscopy (DL)] without faculty assistance. This used a 5-point scaled assessment, ranging from "not able to" to "definitely able to." Multivariate analyses determined if completion of FLS made a difference for resident self-efficacy, stratified by post-graduate year (PGY). RESULTS: At the time of the survey, 2300 reported completion of FLS. The percentage of FLS completion increased from PGY1 to PGY5 (4.2% n = 59 vs 85.8% n = 893). PGY1 residents who completed FLS, from 48 diverse institutions, demonstrated the most significant increases in SE (p < 0.05) with significantly higher perceived self-efficacy in LA (p = 0.001) and LRH (p = 0.012). PGY2 and PGY3 residents indicated increased SE in DL (p = 0.037, p = 0.015, respectively), based on FLS completion. These FLS effects were less evident in the more senior classes. CONCLUSIONS: Completion of FLS arguably has the greatest benefits for more junior residents, as it establishes a foundation of laparoscopic knowledge and skill, upon which further residency training can build. Successful completion of the curriculum and assessment offered by the Fundamentals of Laparoscopic Surgery leads to greater sense of ability in early trainees.

COVID-19 Impact on Surgical Resident Education and Coping.

BACKGROUND: Healthcare systems and surgical residency training programs have been significantly affected by the novel coronavirus disease 2019 (COVID-19) pandemic. A shelter-in-place and social distancing mandate went into effect in our county on March 16, 2020, considerably altering clinical and educational operations. Along with the suspension of elective procedures, resident academic curricula transitioned to an entirely virtual platform. We aimed to evaluate the impact of these modifications on surgical training and resident concerns about COVID-19. MATERIALS AND METHODS: We surveyed residents and fellows from all eight surgical specialties at our institution regarding their COVID-19 experiences from March to May 2020. Residents completed the survey via a secure Qualtrics link. A total of 38 questions addressed demographic information and perspectives regarding the impact of the COVID-19 pandemic on surgical training, education, and general coping during the pandemic. RESULTS: Of 256 eligible participants across surgical specialties, 146 completed the survey (57.0%). Junior residents comprised 43.6% (n = 61), compared to seniors 37.1% (n = 52) and fellows 19.3% (n = 27). Most participants, 97.9% (n = 138), anticipated being able to complete their academic year on time, and 75.2% (n = 100) perceived virtual learning to be the same as or better than in-person didactic sessions. Participants were most concerned about their ability to have sufficient knowledge and skills to care for patients with COVID-19, and the possibility of exposure to COVID-19. CONCLUSIONS: Although COVID-19 impacted residents' overall teaching and clinical volume, residency programs may identify novel virtual opportunities to meet their educational and research milestones during these challenging times.

Diversity, Equity, and Inclusion: A Current Analysis of General Surgery Residency Programs.

BACKGROUND: Local, regional, and national diversity, equity, and inclusion (DEI) initiatives have been established to combat barriers to entry and promote retention in surgery residency programs. Our study evaluates changes in diversity in general surgery residency programs. We hypothesize that diversity trends have remained stable nationally and regionally. MATERIALS AND METHODS: General surgery residents in all postgraduate years were queried regarding their self-reported sex, race, and ethnicity following the 2020 ABSITE. Residents were then grouped into geographic regions. Data were analyzed utilizing descriptive statistics, Kruskal-Wallis test, and chi-square analyses. RESULTS: A total of 9276 residents responded. Nationally, increases in female residents were noted from 38.0 to 46.0% (P < .001) and in Hispanic or Latinx residents from 7.3 to 8.3% (P = .031). Across geographic regions, a significant increase in female residents was noted in the Northwest (51.9 to 58.3%, P = .039), Midwest (36.9 to 43.3%, P = .006), and Southwest (35.8 to 47.5%, P = .027). A significant increase in black residents was only noted in the Northwest (0 to 15.8%, P = .031). The proportion of white residents decreased nationally by 8.9% and in the Mid-Atlantic, Southeast, and Southwest between 5.5 and 15.9% (P < .05). DISCUSSION: In an increasingly diverse society, expanding the numbers of underrepresented surgeons in training, and ultimately in practice, is a necessity. This study shows that there are region-specific increases in diversity, despite minimal change on a national level. This finding may suggest the need for region-specific DEI strategies and initiatives. Future studies will seek to evaluate individual programs with DEI plans and determine if there is a correlation to changing demographics.

Individual and Institutional Factors Associated with PGY5 General Surgery Resident Self-Efficacy: A National Survey.

BACKGROUND: Variability in post-graduate year 5 (PGY5) residents' operative self-efficacy exists; yet the causes of variability have not been explored. Our study aims to determine resident-related and program-dependent factors associated with residents' perceptions of self-efficacy. STUDY DESIGN: Following the 2020 American Board of Surgery In-Training Examination, a national survey of self-efficacy in 10 of the most commonly performed Accreditation Council for Graduate Medical Education case-log procedures was completed. RESULTS: A total of 1,145 PGY5 residents completed the survey (response rate 83.8%), representing 296 surgical residency programs. Female sex (odds ratio [OR] 0.46 to 0.67; 95% CI 0.30 to 0.95; p < 0.05) was associated with decreased self-efficacy for 6 procedures. Residents from institutions with emphasis on autonomy were more likely to report higher self-efficacy for 8 of 10 procedures (OR 1.39 to 3.03; 95% CI 1.03 to 4.51; p < 0.05). In addition, increased socialization among residents and faculty also correlated with increased self-efficacy in 3 of 10 procedures (OR 1.41 to 2.37; 95% CI 1.03 to 4.69; p < 0.05). Procedures performed with higher levels of resident responsibility, based on Graduated Levels of Resident Responsibility (GLRR) and Teaching Assistant (TA) scores, were correlated with higher self-efficacy (p < 0.001). CONCLUSION: Ensuring that residents receive ample opportunities for GLRR and TA experiences, while implementing programmatic support for resident-dependent factors, may be crucial for building self-efficacy in PGY5 residents. Institutional support of resident "autonomy" and increasing methods of socialization may provide a means of building trust and improving perceptions of self-efficacy. In addition, reevaluating institutional policies that limit opportunities for graduated levels of responsibility, while maintaining patient safety, may lead to increased self-efficacy.

A 20-year review of surgical training case logs: Is general surgery still general?

BACKGROUND: Surgical training has undergone many facets of restructuring over the most recent decades, with critiques of the quality and variability of training as well as the competency of recent graduates. This study examines the changes in surgical training in operative volume and breadth in the past 2 decades. METHODS: The Accreditation Council for Graduate Medical Education Case Log Statistics Reports from 1999 to 2019 were reviewed. Case logs were grouped into defined case categories and group levels of postgraduate training. Descriptive analyses and multiple linear regressions were performed. RESULTS: General surgery residents are graduating with 10.7% more cases, owing to increases in mostly junior year cases (P < .001). The breadth of specialty cases has decreased, while there was an increase in alimentary and abdominal cases to 58.4% from 47.2% 20 years ago. A decrease in vascular surgery cases from 19.9% to 10.7% of all cases was noted. Analysis of the distribution of defined categories showed right skewness in many categories with mode being much lower than reported mean. CONCLUSION: Evaluation of trends, despite residents graduating with higher case volume than the minimum required, shows that the breadth and variety of cases has narrowed significantly in the past 20 years, providing a case for general surgery training restructuring.

Defining the Deficit in US Surgical Training: The Trainee's Perspective.

BACKGROUND: Self-efficacy is the personal judgment of how well one can successfully complete a task. The goal of this study was to assess self-efficacy of PGY5 residents for common general surgery operations. We hypothesized there are deficits in self-efficacy of PGY5 residents, and self-efficacy of a given operation correlates with experience performing the operation without attending assistance (independently) and teaching the operation from start to finish. METHODS: A survey was linked to the 2020 American Board of Surgery In-Training Examination. From the ACGME case log's 15 most commonly performed surgeon-chief operations and AHRQ's 15 most common operations, 10 operations were selected. Residents evaluated their ability to perform these operations independently using a 5-point self-efficacy scale. Residents were asked whether they had experience performing these operations independently or teaching the operation from start to finish. Descriptive statistics and Pearson correlation were used to examine the relationship between self-efficacy and operative experience. RESULTS: In all, 1,145 of 1,367 PGY5 residents (84%) responded. Highest self-efficacy was in performing wide-local excision (90.24%) and lowest was in performing open thyroidectomy (19.58%). Eighty-eight (7.7%) reported self-efficacy in all procedures. Statistically significant positive correlations were identified between experience and self-efficacy for cases performed without assistance (r = 0.98, p < 0.01) and cases taught (r = 0.91, p < 0.01). CONCLUSIONS: With 5 months left in training, 92.3% of residents report deficits in preparation for practice, as defined by self-efficacy to complete common procedures independently. Resident self-efficacy increased in direct relation to performed cases and cases taught.

Isolating steps instead of learners: Use of deliberate practice and validity evidence in coronavirus disease (COVID)-era procedural assessment.

BACKGROUND: In surgical training, assessment tools based on strong validity evidence allow for standardized evaluation despite changing external circumstances. At a large academic institution, surgical interns undergo a multimodal curriculum for central line placement that uses a 31-item binary assessment at the start of each academic year. This study evaluated this practice within increased in-person learning restrictions. We hypothesized that external constraints would not affect resident performance nor assessment due to a robust curriculum and assessment checklist. METHODS: From 2018 to 2020, 81 residents completed central line training and assessment. In 2020, this curriculum was modified to conform to in-person restrictions and social distancing guidelines. Resident score reports were analyzed using multivariate analyses to compare performance, objective scoring parameters, and subjective assessments among "precoronavirus disease" years (2018 and 2019) and 2020. RESULTS: There were no significant differences in average scores or objective pass rates over 3 years. Significant differences between 2020 and precoronavirus disease years occurred in subjective pass rates and in first-time success for 4 checklist items: patient positioning, draping, sterile ultrasound probe cover placement, and needle positioning before venipuncture. CONCLUSION: Modifications to procedural training within current restrictions did not adversely affect residents' overall performance. However, our data suggest that in 2020, expert trainers may not have ensured learner acquisition of automated procedural steps. Additionally, although 2020 raters could have been influenced by logistical barriers leading to more lenient grading, the assessment tool ensured training and assessment integrity.

A Novel Blended Curriculum for Communication of Informed Consent With Surgical Interns.

BACKGROUND: Interns often conduct procedural informed consent discussions (ICDs), identified as a core entrustable professional activity. Deficiencies in the training process for ICDs span across specialties. OBJECTIVE: We provide evidence for a curriculum and assessment designed to standardize the training process and ensure ICD competency in surgical interns. METHODS: In March 2019, PowerPoint educational materials were emailed to one academic institution's new surgical interns, who in June participated in an onsite 1-hour role-play "hot seat" group activity (GA) with an untrained simulated patient, and in October completed a single trained simulated patient (real-time raters) verification of proficiency (VOP) assessment. Curriculum evaluation was measured through intern pre-/post-confidence (5-point scale), and the VOP's Cronbach's alpha and test-retest were examined. Data were analyzed with descriptive statistics, paired t tests, and 2-way random effects models. RESULTS: Of 44 new interns, 40 (91%) participated in the remote teaching and live GA and were assessed by the VOP. Pre-/post-GA confidence increased a mean difference of 1.3 (SD = 0.63, P < .001). The VOP's Cronbach's alpha was 0.88 and test-retest was 0.84 (95% CI 0.67-0.93, P < .001), with a 95% pass rate. The 2 first-time fail students required remediation. Time commitment included 1 hour maximum for individual training and implementation and 30 minutes for assessment. The use of volunteers and donated space mitigated additional costs. CONCLUSIONS: Remote asynchronous and group skills teaching for new general surgical interns improved their confidence in conducting procedural ICDs. A patient-simulation verification process appeared feasible with preliminary evidence of retest and internal consistency.

American Board of Surgery Entrustable Professional Activities (EPAs): Assessing Graduating Residents' Perception of Preoperative Entrustment.

OBJECTIVE: To determine if graduating surgical residents are achieving entrustment of surgical entrustable professional activities (EPAs). We hypothesize that postgraduate year 5 (PGY5) residents are achieving evaluation and management entrustment in the selected EPAs. DESIGN: In January 2020, surgical residents completed a survey following the American Board of Surgery In-Training Examination (ABSITE) to measure their levels of entrustment in 4 of the 5 ABS-selected EPAs. A Resident Entrustability Index (REI) score was developed to ascertain PGY5 residents' levels of entrustment (range 1-5). Residents indicated how often their assessments and operative plans were modified in the prior 6 months for each EPA (1=Always, 2=Frequently, 3=Occasionally, 4=Rarely, 5=Never). An independent clinical decision-making score (ICDM) was developed with subsequent evaluation of its relationships to intrinsic, resident-related and extrinsic, program-dependent factors. SETTING: A national post-ABSITE survey. PARTICIPANTS: All general surgery residents participating in ABSITE were invited to participate. Of the 1367 PGY5 residents that completed the survey, 1049 residents (76.7%) responded to the surgical EPA items. RESULTS: Residents achieved an average REI of 4, indicating rare modification of assessments and operative plans for the 4 EPAs assessed. Complete entrustment was reported for inguinal hernias and penetrating abdominal trauma (Median REI = 5, IQR 4, 5) indicating assessments and operative plans were never modified. Lack of entrustment (REI ≤3) was reported by a minority of residents (ranging from 8.6% for operative plan of right lower quadrant pathology to 12.8% for operative plan of blunt abdominal trauma). Significant resident-related and program-dependent factors associated with achievement of expected ICDM was socializing with a co-resident (p = 0.001), while training in one's hometown (p < 0.001) and policies that mandate attendings be scrubbed in (p = 0.022) were associated with decreased achievement of expected ICDM. Overall, 89.2% and 90.3% of PGY5 residents are attaining appropriate levels of entrustment and ICDM abilities, respectively, within 6 months of graduating. CONCLUSIONS: Of the EPAs evaluated, PGY5 residents are achieving appropriate levels of entrustment in evaluation and management. Although this is the case for a vast majority of PGY5 residents, there is still work to be done to ensure that all PGY5 residents are attaining entrustment prior to graduation. Our study also provides content validity for the surgical EPAs in assessing levels of entrustment in PGY5 residents.

Validity Evidence for Vascular Skills Assessment: The Feasibility of Fundamentals of Vascular Surgery in General Surgery Residency.

OBJECTIVE: As the Fundamentals of Laparoscopic Surgery (FLS) and Fundamentals of Endoscopic Surgery (FES) have been used for general surgery assessment, the Fundamentals of Vascular Surgery (FVS) has recently been developed to evaluate core operative skills for vascular trainees. This study examines the 3 year implementation of FVS for general surgery residents and it gathers validity evidence using Messick's framework. We hypothesized that the curriculum and assessment tool enhance general surgery resident training and assessment. DESIGN: This is a retrospective review of FVS assessments of residents using descriptive and multivariate analyses. SETTING: This study was conducted at an academic institution, where simulation-based teaching sessions occur in coordination between the general surgery and the integrated vascular surgery residency programs. PARTICIPANTS: Seventeen general surgery residents were assessed in FVS skills by an expert rater from 2018 to 2020. RESULTS: Overall, 86 assessments were completed. CONTENT: Assessment focuses on 3 open vascular skills (End-to-Side Anastomosis, Patch Angioplasty and Clockface Suturing). Response Process: 7 items comprise a graded rating for a skills score. Additionally, a global summary score is designated. Internal Structure: The assessment tool has a Cronbach's alpha of 0.87, demonstrating good internal consistency. Addition of the second rater correlated with Cohen's kappa -0.69 (p < 0.001), indicating poor interrater reliability. Relationships to other variables: The most significant improvement occurred in total scores between PGY2s (17.4 ± 2.37) and PGY4s (23.2 ± 3.00), p < 0.001, indicating adequate level discernment. CONCLUSIONS: The validity evidence of FVS assessment in this study supports its use in general surgery residency at a time when opportunities for open vascular skills assessment may be decreasing due to case availability and shifting paradigms. Further study into quality rater training is needed to optimize national implementation of FVS and ensure consistency in grading.

A Generalizable Multimodal Scrub Training Curriculum in Surgical Sterile Technique.

Introduction: Recent endeavors from governing bodies such as the AAMC have formally recognized the importance of aseptic technique. AAMC guidelines include activities that all graduating physicians should be able to perform with minimum indirect supervision and were developed to recognize these needs. For example, the skills necessary for aseptic technique include daily safety habits and general physician procedures. Methods: We developed a scrub training curriculum and evaluated the program through a quasi-experimental study with a pre- and posttest design. Questions were developed to examine students' perceived knowledge and skills as related to the objectives of the course and to their anxieties, concerns, and future training needs. Results: Between February 2020 and March 2020, 44 students completed the curriculum. Students indicated that self-efficacy significantly increased in all aspects of the curricular goals following curriculum completion. Students identified understanding OR etiquette as the most anxiety-provoking element associated with scrub training. They felt that more time could be spent elucidating this etiquette. On the other hand, tasks such as surgical hand hygiene were the least anxiety-inducing. Discussion: We share this multimodal scrub training curriculum, mapped to the AAMC's guidelines, to reduce variability in teaching strategies and skills acquisition through a standardized curriculum. Also, we effectively imparted these skills and instilled a sense of confidence in learners as they worked to provide their best in patient care and safety.

Preclinical Surgical Preparatory Course and the NRMP Match: Early Exposure and Surgical Recruitment a 10-Year Follow-Up.

OBJECTIVE: Many medical students mentally commit to specialties prior to entering clerkships. This is why early preclinical interactions with surgical specialties, through mentorship and/or interest groups, increases the opportunity to nurture enthusiasm for surgery. In 2007, a course providing preclinical medical students with introductory surgical skills training and preparation for the surgical environment ("SURG205") was established at our institution. The course underwent a major revision in 2016, increasing intraoperative mentorship by matching students to surgical attendings and requiring students to scrub into operative cases together. We anticipate that the positive surgical experiences created by the course will lead to further development and enhancement of student interest in surgical specialties-interest that we hypothesized would reflect in their National Resident Matching Program (NRMP) Match outcomes. DESIGN: NRMP results from 2010 to 2019 were cross-referenced with a database of students who participated in the SURG205 course from 2007 to 2016. With this, we examined the correlation between student participation in SURG205 and surgical specialty match. Descriptive statistics were used to review the trends of the NRMP results, and Pearson's correlation was used to determine the relationship and its significance. SETTING: This study was conducted in a single private medical school in California. PARTICIPANTS: Specialties considered "surgical" included: General Surgery, integrated programs-such as Plastic, Thoracic, or Vascular surgery, Obstetrics and Gynecology, Orthopedic Surgery, Otolaryngology, Neurosurgery, and Urology. All other specialties were considered nonsurgical. Students identified as having participated in SURG205 and who then also took part in the NRMP. RESULTS: Seven hundred eighty students underwent the Match process from 2010 to 2019. 144 (18.5%) of these students participated in SURG205 between 2007 and 2016. Each Match class ranged in size from 62 to 91 (median = 77.5, IQR = 14.5) students. (Table 1) Two-hundred and nineteen students (28.1%) matched into a surgical specialty, of which 34 (15.5%) selected general surgery. From 2010 to 2019 the rate of students who matched into surgical specialties averaged 28.1% per year with a slight nonstatistically significantly increasing trend over that time period R2 = 0.30 (p = 0.09; Fig. 1). There was a significant increase in trend in proportion of students who took the course and matched into any specialty between 2010 and 2019 (R2 = 0.85, p = 0.0002; Fig. 2). And, there was a statistically significant positive relationship between students taking the course and matching into a surgical specialty (R2 = 0.63, p = 0.01; Fig. 3). CONCLUSION: Our results highlight the increasing tendency of students who pursue surgical specialties having previously participated in this early exposure courses. Not only is student interest created and encouraged through positive mentorship experiences, but that interest may be associated with increases in application rates and eventual match into the specialty. General surgery training programs might consider these trends when designing courses to ease transitions into first-year residency positions-such as fourth-year surgical boot camps, surgical procedure-based anatomy courses, and mentorship frameworks. This information further justifies the cost and time commitment required to administer these programs for students.