The Impact of Simulation Training on Operative Performance in General Surgery: Lessons Learned from a Prospective Randomized Trial.

BACKGROUND: Practice in the simulated environment can improve surgical skills. However, the transfer of open complex surgical skills to the operating room is unclear. This study evaluated the effect of resident operative performance following a simulation experience on a hand-sewn small bowel anastomosis and determined the impact of utilizing proficiency-based training. METHODS: Nine categorical interns performed a hand-sewn small bowel anastomosis in the operating room prior to (pre-test) and following (post-test) a 3-h simulation training session with an assessment at the end. Participants were randomly assigned to 1of 2 simulation training groups: proficiency-based or standard. Operative performance was videotaped. 2 independent, blinded faculty surgeons assessed performances by a global rating scale. Pre- and post-confidence levels were obtained on a 5-point Likert scale. RESULTS: Overall, pre-test and post-test operative performance was similar (3 [IQR, 2.5 -3.5] versus 3 [IQR, 3 -3], P = 0.59). Furthermore, no difference was observed in the post-test performance with proficiency-based or standard training (3 [IQR, 3 -3] versus 3 [IQR, 3 -3], P = 0.73). Self-reported confidence with the skills, however, significantly improved (median 1 versus 4, P = 0.007). CONCLUSIONS: In this prospective, randomized study, we did not observe an improvement in operative performance following simulation instruction and assessment, with both training groups. Overcoming barriers to skills transfer will be paramount in the future to optimize simulation training in general surgery. These findings highlight the importance of continued study for the ideal conditions and timing of technical skills training.

Correlation Between Medical School Performance and Receiving Awards in General Surgery Residency.

IMPORTANCE: It is uncertain whether current measures of achievement during medical school predict exceptional performance during surgical residency. One surrogate of excellence during residency may be awards, especially those given for teaching and annual overall accomplishment. OBJECTIVE: Determine whether markers of superior performance during medical school documented in the Electronic Residency Application Service (ERAS) application and student record correlated with receiving awards during residency. DESIGN: Data was analyzed from ERAS and student applications of 296 residency graduates. Spearman correlation coefficients were calculated. PARTICIPANTS: 296 residency graduates from 14 US general surgery residency programs. MAIN OUTCOMES AND MEASUREMENTS: The relationship between each residency and preresidency variable and the outcome of receiving any qualifying award. Secondary analysis included the correlation with winning a teaching or annual overall award. RESULTS: Although 140 (48%) residents received an award during residency, only 69 (23.3%) received a teaching award, while 50 (17.6%) received one for annual overall performance. Membership in Alpha Omega Alpha (AOA) and honors in pediatrics rotation correlated with receiving any award. USMLE step 1 score and AOA membership reached statistically significant positively correlation with receiving a teaching award, while completing a sub-internship at that institution was negatively associated. For annual overall awards, only completing a sub-internship at the same institution reached statistical significance. CONCLUSIONS: None of the traditional metrics in the ERAS or global student record consistently correlated with receiving an award during residency. Our findings suggest that preresidency factors available on the global application, including grades, test scores, and honor society membership, poorly correlate with exceptional general surgery residency performance as measured by receiving awards. Residency programs should shift away from using traditional ERAS metrics as the primary criteria for selection, but rather as 1 component of holistic applicant review.

The Patient's Voice in Assessing Resident Communication Skills.

OBJECTIVE: To generate validity evidence for using patient-reported satisfaction ratings of residents' communication skills to determine progress along the Interpersonal and Communication Skills (ICS) Milestones. DESIGN: A single-institution, retrospective study analyzed a CAHPS Clinician and Group Survey (CG-CAHPS) database which collects inpatients' ratings of residents' communication skills using 6 questions on a scale of 1 (very poor) to 5 (very good). CG-CAHPS results for each resident were averaged for each question as well as across the 6 questions. The averaged ratings were compared between low and high performer groups. SETTING: A large, academic, mid-western General Surgery residency program. PARTICIPANTS: General Surgery residents with 3 or more survey responses from July 2020 to June 2021 were included. Residents were dichotomized into low or high performer groups based on their end-of-year ICS1 sub-competency milestone within their post-graduate year (PGY) cohort. RESULTS: 543 CG-CAHPS responses across 44 residents were analyzed with a median of 9 (Interquartile range 6, 17) responses per resident. When residents were compared based on PGY, ratings for the question "resident's knowledge about your medical care of condition" demonstrated statistically significant differences with PGY5s receiving the lowest score (p = 0.05). PGY5s received the lowest averaged ratings across all questions (p = 0.08, η2 = 0.10). When residents were dichotomized into low (n = 21) and high performer groups (n = 23) based on ICS1 milestones, statistically significant differences were noted in ratings for the questions "concern the resident showed for your questions or worries" (4.81 vs. 5, p = 0.047) and "courtesy and respect of the resident" (4.75 vs. 5, p = 0.046). CONCLUSIONS: Analysis of patient ratings of surgery residents' communication skills demonstrated concordant findings between ICS1 milestone and 2 domains of CG-CAHPS responses. When low and high performer groups were compared, CG-CAHPS responses demonstrated a similar pattern. These findings provide validity evidence for CG-CAHPS data as a source of information for ICS1 sub-competency assessment.

Managing Up: Approaching Mentoring From a Mentee's Perspective.

OBJECTIVE: Discuss the evolution of mentorship models in surgical training and how educating the surgical trainees with the concepts from "manage up" theory can empower them to maximize the benefits afforded by mentoring relationships. METHODS: "Manage up" theory is derived from the business world where the subordinate takes ownership of the mentoring relationship with their superior by assessing the strengths and weaknesses of both parties and applying that information in managing a productive relationship. DISCUSSION: Surgery residency programs implement a variety of structured and unstructured mentorship programs to promote mentoring relationships and to encourage professional development. Mentees in successful mentoring relationships demonstrate characteristics and skills that residency programs can promote through formal training. Components of "manage up" theory can be applied by surgical trainees in approaching their mentors and in managing their mentor-mentee relationships. CONCLUSIONS: The benefits gained from a successful mentoring relationship for both the mentor and the mentee depend on active roles played by both parties. Strong evidence supports the need for educating mentees through formal curricula to empower them to assume an active role in their mentoring relationships.

Operative teaching takes "GUTS": Impact of Educational Time Out on trainee's cognitive load.

BACKGROUND: Educational Time Out (ETO) incorporates intraoperative goal setting, task delineation, and debriefing. ETO has been previously reported to generate positive learning experiences for residents. This study aimed to study the impact of ETO on surgery residents' cognitive load. METHODS: A cross-over study design utilized the ETO arm during the first half of the rotation followed by no ETO for the second half. Surgery residents completed a modified NASA Task Load Index (NASA TLX) questionnaire following each operative case to report their cognitive load. RESULTS: 141 modified NASA TLX questionnaire responses were obtained where ETO occurred in 73 responses and no ETO in 68 responses. Residents reported a higher performance (p = 0.004) and a lower frustration (p = 0.018) component scores when ETO occurred. CONCLUSIONS: This study identified improved cognitive load on the modified NASA TLX instrument with higher performance and lower frustration scores associated with trainees who underwent preoperative goal setting with an ETO using the GUTS method.

The Virtual Recruitment Onion: Peeling Back the Layers of the Interview Season During the COVID-era.

OBJECTIVE: We sought to characterize General Surgery residency program directors' (PDs) baseline perspective on how the COVID-19 mandated changes to the recruitment and interview processes impacted how the PDs evaluated and recruited the applicants. DESIGN: An anonymous cross-sectional questionnaire survey. SETTING: A large, mid-western academic general surgery residency program. PARTICIPANTS: 47 PDs of Accreditation Council for Graduate Medical Education (ACGME) accredited General Surgery residency programs. RESULTS: During the virtual-only interviews during the COVID-19 pandemic-era 2020-21 General Surgery residency application cycle, PDs shifted their focus to virtual outreach efforts and bolstered social media presences to recruit strong applicants. Also, our study found statistically significant changes to the increased value of letters of recommendation (LORs) for the PDs when assessing an applicant's commitment to surgery. These findings suggest that the necessity of adapting to the virtual-only interview format significantly altered how the PDs recruited and evaluated applicants for the General Surgery residency match. CONCLUSIONS: A complete replacement of the in-person interviews with virtual-only interviews may be challenging unless buy-in exists from key stakeholders in the surgical community. Our study highlights the PDs' hesitation in assessing candidates' commitment to surgery from virtual interviews alone. Incorporating virtual interviews as a part of the screening process for applicants may serve as an avenue to maximize the benefits of the virtual interview format. Furthermore, COVID-19 pandemic has normalized the growing social media presence of residency programs, adding to the changing landscape of recruiting and interviewing applicants for General Surgery residency match.

Characteristics Associated With Outstanding General Surgery Residency Graduate Performance, as Rated by Surgical Educators.

Importance: Characteristics of outstanding graduating surgical residents are currently undefined. Identifying these qualities may be important in guiding resident selection and resident education. Objective: To determine characteristics that are most strongly associated with being rated as an outstanding graduating surgical resident. Design, Setting, and Participants: The multi-institutional study had 3 phases. First, an expert panel developed a list of characteristics embodied by top graduating surgical residents. Second, groups of faculty from 14 US general-surgery residency programs ranked 2017 through 2020 graduates into quartiles of overall performance. Third, faculty evaluated their graduates on each characteristic using a 5-point Likert scale. Data were analyzed using Spearman rank-order correlation to identify which individual characteristics were associated with overall graduate performance. A least absolute shrinkage and selection operator (LASSO) ordinal regression was performed to select a parsimonious model to predict the outcome of overall performance rating from individual characteristic scores. Main Outcome and Measures: Surgical educators' rankings of general surgery residency graduates' overall performance. Results: Fifty faculty from 14 US residency programs with a median of 13 (range, 5-30) years of surgical education experience evaluated 297 general surgery residency graduates. Surgical educators identified 21 characteristics that they believed outstanding graduating surgical residents possessed. Two hundred ninety-seven surgical residency graduates were evaluated. Higher scores in every characteristic correlated with better overall performance. Characteristics most strongly associated with higher overall performance scores were surgical judgment (r = 0.728; P < .001), leadership (r = 0.726; P < .001), postoperative clinical skills (r = 0.715; P < .001), and preoperative clinical skills (r = 0.707; P < .001). The remainder of the characteristics were moderately associated with overall performance. The LASSO regression model identified 3 characteristics from which overall resident performance could be accurately predicted without measuring other qualities: surgical judgment (odds ratio [OR] per 1 level of 5-level Likert scale OR, 1.27; 95% CI, 1.03-1.51), leadership (OR, 1.27; 95% CI, 1.06-1.48), and medical knowledge (OR, 1.16; 95% CI, 1.01-1.33). Conclusions and Relevance: All individual characteristics identified by surgical educators as being qualities of outstanding graduating surgical residents were positively associated with overall graduate performance. Surgical judgment and leadership skills had the strongest individual associations. Assessment of only 3 qualities (surgical judgment, leadership, and medical knowledge) were required to predict overall resident performance ratings. These findings highlight the importance of developing specific surgical judgment and leadership skills curricula and assessments during surgical residency.

Don't Judge a Letter by its Title: Linguistic Analysis of Letters of Recommendation by Author's Academic Rank.

OBJECTIVE: This study analyzed the linguistic differences in letters of recommendation (LORs) for general surgery residency applicants written by authors of various academic ranks. Given that many general surgery residency programs require a LOR from the Chair of surgery, this study also examined whether LORs written by the Chair demonstrate linguistic differences to support this practice. DESIGN: A single institution, retrospective review analyzed LORs from two application cycles of general surgery residency applicants who were selected for interview at a large academic institution. Word count (WC) and linguistic characteristics of LORs were analyzed with a previously developed institution-specific dictionary using the Linguistic Inquiry and Word Count software (LIWC2015; Pennebaker Conglomerates, Inc., Austin, Texas). WC and linguistic characteristics of LORs reported as frequencies of terms within twenty-four categories were examined based on the letter authors' academic rank. Further examination compared LORs written by a Chair of surgery with those written by non-Chairs. SETTING: A single large, Midwestern academic general surgery residency program PARTICIPANTS: Four hundred and sixty-five letters of recommendation received during two interview cycles were included for analysis. RESULTS: A total of 465 LORs written by assistant (n = 82), associate (n = 94), and full professors (n = 289) were included in the study. No statistically significant difference was noted in the WC of LORs based on the letter writers' academic ranks (p = 0.95). Assistant professors utilized grindstone, communal, and technical skill terms with higher frequencies compared to associate professors and full professors. LORs written by assistant professors demonstrated the highest authentic variable score followed by associate professors then full professors (4.94, 3.92, 3.28, p < 0.01). LORs written by Chairs (n = 128) had lower authentic variable scores compared to LORs written by non-Chairs (n = 337; 2.71 vs. 3.91, p = 0.001). Only 50 (39%) LORs written by Chairs indicated working directly with the applicant, and sub-group analysis demonstrated a higher authentic variable score in this group compared with LORs written by Chairs who did not indicate having worked directly with the applicant (3.51 vs. 2.5, p = 0.01). CONCLUSIONS: Linguistic analysis of LORs for general surgery residency applicants demonstrated minor yet statistically significant differences based on the author's academic rank. If applicants can obtain linguistically similar LORs from surgeons of any academic rank, but less authentic LORs from writers with higher academic ranks, these LORs may be less valuable for the residency programs when evaluating applicants. Based on the subgroup analysis, less than 40% of Chair LORs indicated that the Chair worked directly with the applicant, calling into question the utility of the Chair LORs as meaningful evaluation of applicants. Further study to compare LORs of applicants selected and not selected for interview may add additional insight into linguistic differences in LORs written by authors of different academic ranks.