Validation of a Novel Inverted Peg Transfer Task: Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment.

OBJECTIVE: To evaluate the validity of a novel inverted peg transfer (iPT) task for assessing laparoscopic skills of novices and experts and compare iPT to the regular PT (rPT) task to ensure surgical trainee acquisition of an adequate advanced laparoscopic skills level for safe laparoscopic practice in the operating room. DESIGN: Prospective crossover study. SETTING: Multidisciplinary simulation center and motion analysis laboratory, Mayo Clinic. PARTICIPANTS: Novices were medical students and surgical interns without laparoscopic experience. Experts were surgeons with at least 3 years of experience in laparoscopic surgery. METHODS: This was the first exposure to iPT for both groups. Completion time and performance metrics were recorded. A scoring rubric was used to calculate a normalized performance score between 0 and 100. Wilcoxon rank sum and Mann-Whitney tests were performed with α = 0.05. Receiver-operating characteristic curves were graphed for the 2 task scores to assess the tasks' sensitivity and specificity in differentiating laparoscopic experience level. MAIN OUTCOME MEASURES: Performance measures of completion time, transferred triangles, dropped triangles (errors), and the overall performance score on both tasks between- and within-subjects (i.e., novices and experts). RESULTS: Thirty-six novices and eight experts participated. Both experts and novices had longer completion time and lower scores during iPT than rPT (p < 0.05). Within iPT, novice completion times were 144 seconds longer (p = 0.04), and performance score was 35 points lower than experts (p < 0.01). No differences between novices and experts were observed for completion time or performance scores (p > 0.05) for rPT. The iPT scores had a higher sensitivity and specificity than the rPT (area under the receiver-operating characteristic curve: iPT = 0.91; rPT = 0.69). CONCLUSIONS: iPT is a valid assessment of advanced laparoscopic skills for surgical trainees with higher specificity and sensitivity than rPT. As advanced minimally invasive surgery becomes more common, it is important that tasks such as iPT be included in surgical simulation curricula and training assessment.

A Comparison of Objective Assessment Data for the United States and International Medical Graduates in a General Surgery Residency.

OBJECTIVE: To compare objective assessment scores between international medical graduates (IMGs) and United States Medical Graduates. Scores of residents who completed a preliminary year, who later matched into a categorical position, were compared to those who matched directly into a categorical position at the Mayo Clinic, Rochester. DESIGN: Postgraduate year (PGY) 1 to 5 residents participate in a biannual multistation, OSCE-style assessment event as part of our surgical training program. Assessment data were, retrospectively, reviewed and analyzed from 2008 to 2016 for PGY-1 and from 2013 to 2016 for PGY 2 to 5 categorical residents. SETTING: Academic medical center. PARTICIPANTS: Categorical PGY 1 to 5 General Surgery (GS) residents at Mayo Clinic Rochester, MN. RESULTS: A total of 86 GS residents were identified. Twenty-one residents (1 United States Medical Graduates [USMG] and 20 IMGs) completed a preliminary GS year, before matching into a categorical position and 68 (58 USMGs and 10 IMGs) residents, who matched directly into a categorical position, were compared. Mean scores (%) for the summer and winter multistation assessments were higher for PGY-1 trainees with a preliminary year than those without (summer: 59 vs. 37, p < 0.001; winter: 69 vs. 61, p = 0.05). Summer and winter PGY-2 scores followed the same pattern (74 vs. 64, p < 0.01; 85 vs. 71, p < 0.01). For the PGY 3 to 5 assessments, differences in scores between these groups were not observed. IMGs and USMGs scored equivalently on all assessments. Overall, junior residents showed greater score improvement between tests than their senior colleagues (mean score increase: PGY 1-2 = 18 vs. PGY 3-5 = 3, p < 0.001). CONCLUSIONS: Residents with a previous preliminary GS year at our institution scored higher on initial assessments compared to trainees with no prior GS training at our institution. The scoring advantage of an added preliminary year decreased as trainees progressed through residency.

Objective Assessment of General Surgery Residents Followed by Remediation.

OBJECTIVE: Surgical training programs often lack objective assessment strategies. Complicated scheduling characteristics frequently make it difficult for surgical residents to undergo formal assessment; actually having the time and opportunity to remediate poor performance is an even greater problem. We developed a novel methodology of assessment for residents and created an efficient remediation system using a combination of simulation, online learning, and self-assessment options. DESIGN: Postgraduate year (PGY) 2 to 5 general surgery (GS) residents were tested in a 5 station, objective structured clinical examination style event called the Surgical X-Games. Stations were 15 minutes in length and tested both surgical knowledge and technical skills. Stations were scored on a scale of 1 to 5 (1 = Fail, 2 = Mediocre, 3 = Pass, 4 = Good, and 5 = Stellar). Station scores ≤ 2 were considered subpar and required remediation to a score ≥ 4. Five remediation sessions allowed residents the opportunity to practice the stations with staff surgeons. Videos of each skill or test of knowledge with clear instructions on how to perform at a stellar level were offered. Trainees also had the opportunity to checkout take-home task trainers to practice specific skills. Residents requiring remediation were then tested again in-person or sent in self-made videos of their performance. SETTING: Academic medical center. PARTICIPANTS: PGY2, 3, 4, and 5 GS residents at Mayo Clinic in Rochester, MN. RESULTS: A total of, 35 residents participated in the Surgical X-Games in the spring of 2015. Among all, 31 (89%) had scores that were deemed subpar on at least 1 station. Overall, 18 (58%) residents attempted remediation. All 18 (100%) achieved a score ≥ 4 on the respective stations during a makeup attempt. Overall X-Games scores and those of PGY2s, 3s, and 4s were higher after remediation (p < 0.05). No PGY5s attempted remediation. CONCLUSIONS: Despite difficulties with training logistics and busy resident schedules, it is feasible to objectively assess most GS trainees and offer opportunities to remediate if performance is poor. Our multifaceted remediation methodology allowed 18 residents to achieve good or stellar performance on each station after deliberate practice. Enticing chief residents to participate in remediation efforts in the spring of their final year of training remains a work in progress.

Catch me if you can early simulation efforts affect fundamental surgical skill assessment scores.

BACKGROUND: We evaluated whether early exposure to a simulation curriculum enhances acquired surgical skills. METHODS: The "Surgical Olympics" evaluates interns on basic surgical skills and knowledge. After the Summer Olympics (July), interns were randomly divided into groups: "A" participated in a 7-week curriculum once a week, whereas "B" attended 7 weeks of lectures once a week. All interns then participated in the October Olympics. The 2 groups then switched. Finally, all interns completed a January Olympics. RESULTS: Scores were tabulated for the July, October, and January Olympics. Mean scores (A = 182 ± 42, Group B = 188 ± 34; P = .70) were similar in July; in October, group A (mean score = 237 ± 31) outperformed group B (mean score = 200 ± 32; P = .01). Mean total scores in January (A = 290 ± 34, B = 276 ± 34; P = .32) were similar. CONCLUSIONS: Early exposure to a surgical simulation curriculum enhances surgical intern performance in our Surgical Olympics. Subsequent simulation experience helps learners close this gap.