Robotic performance metrics model fellow proficiency in living donor nephrectomy.

We investigated the use of robotic objective performance metrics (OPM) to predict number of cases to proficiency and independence among abdominal transplant fellows performing robot-assisted donor nephrectomy (RDN). 101 RDNs were performed by 5 transplant fellows from September 2020 to October 2023. OPM included fellow percent active control time (%ACT) and handoff counts (HC). Proficiency was defined as ACT ≥ 80% and HC ≤ 2, and independence as ACT ≥ 99% and HC ≤ 1. Case number was significantly associated with increasing fellow %ACT, with proficiency estimated at 14 cases and independence at 32 cases (R2 = 0.56, p < 0.001). Similarly, case number was significantly associated with decreasing HC, with proficiency at 18 cases and independence at 33 cases (R2 = 0.29, p < 0.001). Case number was not associated with total active console time (p = 0.91). Patient demographics, operative characteristics, and outcomes were not associated with OPM, except for donor estimated blood loss (EBL), which positively correlated with HC. Abdominal transplant fellows demonstrated proficiency at 14-18 cases and independence at 32-33 cases. Total active console time remained unchanged, suggesting that increasing fellow autonomy does not impede operative efficiency. These findings may serve as a benchmark for training abdominal transplant surgery fellows independently and safely in RDN.

The Advanced Certifying Exam Simulation-Pro assessment instrument: evaluating surgical trainee examsmanship in virtual oral exams.

Purpose: In response to the COVID-19 pandemic, many educational activities in general surgery residency have shifted to a virtual environment, including the American Board of Surgery (ABS) Certifying Exam. Virtual exams may become the new standard. In response, we developed an evaluation instrument, the ACES-Pro, to assess surgical trainee performance with a focus on examsmanship in virtual oral board examinations. The purpose of this study was two-fold: (1) to assess the utility and validity of the evaluation instrument, and (2) to characterize the unique components of strong examsmanship in the virtual setting, which has distinct challenges when compared to in-person examsmanship. Methods: We developed a 15-question evaluation instrument, the ACES-Pro, to assess oral board performance in the virtual environment. Nine attending surgeons viewed four pre-recorded oral board exam scenarios and scored examinees using this instrument. Evaluations were compared to assess for inter-rater reliability. Faculty were also surveyed about their experience using the instrument. Results: Pilot evaluators found the ACES-Pro instrument easy to use and felt it appropriately captured key professionalism metrics of oral board exam performance. We found acceptable inter-rater reliability in the domains of verbal communication, non-verbal communication, and effective use of technology (Guttmann's lambda-2 were 0.796, 0.916, and 0.739, respectively). Conclusions: The ACES-Pro instrument is an assessment with evidence for validity as understood by Kane's framework to evaluate multiple examsmanship domains in the virtual exam setting. Examinees must consider best practices for virtual examsmanship to perform well in this environment. Supplementary Information: The online version contains supplementary material available at 10.1007/s44186-023-00107-7.

The impact of virtual interviews on the Fellowship Council application and main match process.

BACKGROUND: The COVID-19 pandemic led the Fellowship Council (FC) to transition rapidly from in-person to virtual interviews. We investigated the impact of this transition on the FC application and main match process. METHODS: Five years (2018-2022) of deidentified FC applicant, program, and match rank data were used to assess differences between in-person (2018-2019) and virtual interview (2021-2022) cycles. Data are expressed as mean ± SD and one-way and two-way MANOVA tests were applied. RESULTS: Trainees applied to an average of 30.4 ± 24.3 programs and ranked an average of 10.7 ± 9.7 programs with a 57% match rate and average rank position of 3.6 ± 3.3. Fellowship programs received an average of 64.9 ± 28.6 applications and ranked an average of 15.4 ± 8.8 applicants with a 95% match rate and average applicant rank position of 3.0 ± 3.4. Applicants who interviewed virtually applied to a greater number of programs (32.7 vs. 27.0; p < 0.001) and ranked a greater number of programs (11.5 vs. 10.0; p = 0.004) with no difference in match rates (58% vs. 55%, p = 0.291). Among matched applicants, there was a significant difference in average rank position (3.20 vs. 4.30, p < 0.001), favoring the in-person cohort. Fellowship programs had more applicants per program (69.2 vs. 57.8; p < 0.001) and ranked more applicants (17.4 vs. 13.3; p < 0.001) during the virtual interview cycles. No difference in either match rates (93% vs. 96%, p = 0.178) or applicant rank position (3.09 vs. 2.93, p = 0.561) was seen between in-person and virtual application cycles. CONCLUSION: Virtual interviews were associated with an increased number of applications for fellowship and applicants ranked by programs but did not impact match rates of either group. Rank match position declined somewhat for applicants but not for fellowship programs. Virtual interviews offer more opportunities for applicants and a greater number of candidates for fellowship programs with only a slight decrement in fellow match rank position.

A novel assessment model for teaching robot-assisted living donor nephrectomy in abdominal transplant surgery fellowship.

BACKGROUND: An increasing number of transplant centers have adopted robot-assisted living donor nephrectomy. Thus, a transplant fellow assessment tool is needed for promoting operative independence in an objective and safe manner. METHODS: In this pilot study, data was prospectively collected on both fellow performance with focus on technique, efficiency, and communication ("overall RO-SCORE"), and operative steps ("operative steps RO-SCORE"). Robotic user performance metrics were analyzed from the da Vinci Xi system, including fellow percent active control time (ACT) and handoff counts. RESULTS: From July 2020 to February 2021, twenty-one robot-assisted donor nephrectomies were performed. In regression analysis, fellow performance (based on both RO-SCOREs and robot % ACT) was significantly associated with both time and case number, with time-to-independence modelled at 8.4-14.2 months, and case number-to-independence estimated at 15-22 cases. Robot user metrics provided valid objective measures alongside RO-SCOREs. CONCLUSIONS: This pilot study provides an effective assessment tool for promoting operative competency in robot-assisted donor nephrectomy among transplant fellows.