A Competency-based Laparoscopic Cholecystectomy Curriculum Significantly Improves General Surgery Residents' Operative Performance and Decreases Skill Variability: Cohort Study.

OBJECTIVE: To demonstrate the feasibility of implementing a CBE curriculum within a general surgery residency program and to evaluate its effectiveness in improving resident skill. SUMMARY OF BACKGROUND DATA: Operative skill variability affects residents and practicing surgeons and directly impacts patient outcomes. CBE can decrease this variability by ensuring uniform skill acquisition. We implemented a CBE LC curriculum to improve resident performance and decrease skill variability. METHODS: PGY-2 residents completed the curriculum during monthly rotations starting in July 2017. Once simulator proficiency was reached, residents performed elective LCs with a select group of faculty at 3 hospitals. Performance at curriculum completion was assessed using LC simulation metrics and intraoperative operative performance rating system scores and compared to both baseline and historical controls, comprised of rising PGY-3s, using a 2-sample Wilcoxon rank-sum test. PGY-2 group's performance variability was compared with PGY-3s using Levene robust test of equality of variances; P < 0.05 was considered significant. RESULTS: Twenty-one residents each performed 17.52 ± 4.15 consecutive LCs during the monthly rotation. Resident simulated and operative performance increased significantly with dedicated training and reached that of more experienced rising PGY-3s (n = 7) but with significantly decreased variability in performance ( P = 0.04). CONCLUSIONS: Completion of a CBE rotation led to significant improvements in PGY-2 residents' LC performance that reached that of PGY-3s and decreased performance variability. These results support wider implementation of CBE in resident training.

Career Advancement for Surgeon-Educators:Findings from a Modified Delphi Process.

OBJECTIVES: Reward and recognition of surgical education as an academic activity remains a highly variable process between institutions. The goal of this study is to provide expert consensus definition of an academic surgical educator, with focus on criteria for academic promotion. STUDY DESIGN AND SETTING: Following IRB approval, a Web-based modified Delphi process was used to generate prioritized academic promotion criteria for surgical educators. PARTICIPANTS AND SETTING: Participants were recruited nationally from a pool of senior academic surgeons who are members of the Society of University Surgeons and the Society of Surgical Chairs. RESULTS: Following a three-round modified Delphi process, the top domains of educational activity for promotion to associate professor and professor were scholarship, teaching, and administration; mentorship was also a priority category for promotion to professor. The top three activities described for promotion to Associate Professor were active participation in conferences/ departmental educational activities for medical students and residents; educational portfolio demonstrating commitment to activities as an educator; and clinical teaching excellence at their home institution. The three activities most highly scored items for promotion to Professor were mentorship of junior surgical educators; active participation in conferences/ departmental educational activities for medical students and residents; and a record of teaching excellence at the medical student and resident levels. CONCLUSIONS: These findings demonstrate a progression from teacher to scholar to leader across a surgical educator's career, with each level incorporating and building upon the prior activities. Identification of categories and criteria may meaningfully inform best practices to be incorporated into the career development and promotion processes for surgeons on an educator academic pathway.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

OBJECTIVE: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees. SUMMARY BACKGROUND DATA: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession. Current methods of operative skill assessment for surgeons rely heavily on global assessment strategies across a very broad domain of procedures. There is no mechanism to assure technical competence for individual procedures. The science and scalability of operative skill assessment has progressed significantly in recent decades, and can inform a much more meaningful strategy for competency-based assessment of operative skill than has been previously achieved. METHODS: The present article reviews the current status and science of operative skill assessment and proposes a template for competency-based assessment which could be used to update training, accreditation, and certification processes. The proposal is made in reference to general surgery but is more generally applicable to other procedural specialties. RESULTS: Streamlined, routine assessment of every procedure performed by surgical trainees is feasible and would enable a more competency-based educational paradigm. In light of the constraints imposed by both clinical volume and assessment bias, trainees should be expected to become proficient and be measured against a mastery learning standard only for the most important and highest-frequency procedures. For less frequently observed procedures, performance can be compared to a norm-referenced standard and, to provide an overall trajectory of performance, analyzed in aggregate. Key factors in implementing this approach are the number of evaluations, the number of raters, the timeliness of evaluation, and evaluation items. CONCLUSIONS: A competency-based operative skill assessment can be incorporated into surgical training, assessment, and certification. The time has come to develop a systematic approach to this issue as a means of demonstrating professional standards worthy of the public trust.

Development of a peer review of operative teaching process and assessment tool.

BACKGROUND: While teaching evaluation systems are common in academia, very little information is available regarding formal coaching and peer review of teaching performance in surgery. This article is a report on the development and implementation of a peer review of operative teaching program. METHODS: Our process was designed using a multistep sequential model which included developing a peer review of teaching instrument that was piloted to study the efficacy and utility of the tool. RESULTS: Thirty-nine peer reviews of teaching were conducted. Among the most frequent challenges that faculty identified were allowing residents to struggle/give autonomy, judging when to take over the case, communicating effectively, being patient, balancing education and patient safety, and giving feedback. CONCLUSIONS: Our peer review of teaching program is systematic, feasible, and can be adopted by other surgery departments. Faculty's identified strengths and challenges have been incorporated into our faculty development curricula.

Trainee Autonomy in Minimally Invasive General Surgery in the United States: Establishing a National Benchmark.

OBJECTIVE: Minimally invasive surgery (MIS) is an integral component of General Surgery training and practice. Yet, little is known about how much autonomy General Surgery residents achieve in MIS procedures, and whether that amount is sufficient. This study aims to establish a contemporary benchmark for trainee autonomy in MIS procedures. We hypothesize that trainees achieve progressive autonomy, but fail to achieve meaningful autonomy in a substantial percentage of MIS procedures prior to graduation. SETTING/PARTICIPANTS: Fifty General Surgery residency programs in the United States, from September 1, 2015 to March 19, 2020. All Categorical General Surgery Residents and Attending Surgeons within these programs were eligible. DESIGN: Data were collected prospectively from attending surgeons and categorical General Surgery residents. Trainee autonomy was assessed using the 4-level Zwisch scale (Show and Tell, Active Help, Passive Help, and Supervision Only) on a smartphone application (SIMPL). MIS procedures included all laparoscopic, thoracoscopic, endoscopic, and endovascular/percutaneous procedures performed by residents during the study. Primary outcomes of interest were "meaningful autonomy" rates (i.e., scores in the top 2 categories of the Zwisch scale) by postgraduate year (PGY), and "progressive autonomy" (i.e., differences in autonomy between PGYs) in MIS procedures, as rated by attending surgeons. Primary outcomes were determined with descriptive statistics, one-way analysis of variance (ANOVA) and Z-tests. Secondary analyses compared (i) progressive autonomy between common MIS procedures, and (ii) progressive autonomy in MIS vs. non-MIS procedures. RESULTS: A total of 106,054 evaluations were performed across 50 General Surgery residency programs, of which 38,985 (37%) were for MIS procedures. Attendings performed 44,842 (42%) of all evaluations, including 16,840 (43%) of MIS evaluations, while residents performed the rest. Overall, meaningful autonomy in MIS procedures increased from 14.1% (PGY1s) to 75.9% (PGY5s), with significant (p < 0.001) increases between each PGY level. Meaningful autonomy rates were higher in the MIS vs. non-MIS group [57.2% vs. 48.0%, p < 0.001], and progressed more rapidly in MIS vs. non-MIS, (p < 0.05). The 7 most common MIS procedures accounted for 83.5% (n = 14,058) of all MIS evaluations. Among PGY5s performing these procedures, meaningful autonomy rates (%) were: laparoscopic appendectomy (95%); laparoscopic cholecystectomy (93%); diagnostic laparoscopy (87%); upper/lower endoscopy (85%); laparoscopic hernia repair (72%); laparoscopic partial colectomy (58%); and laparoscopic sleeve gastrectomy (45%). CONCLUSIONS: US General Surgery residents receive progressive autonomy in MIS procedures, and appear to progress more rapidly in MIS versus non-MIS procedures. However, residents fail to achieve meaningful autonomy in nearly 25% of MIS cases in their final year of residency, with higher rates of meaningful autonomy only achieved in a small subset of basic MIS procedures.

A Randomized Cross-Over Trial Focused on Clinical Breast Exam Skill Acquisition Using High Fidelity versus Low Fidelity Simulation Models in Rwanda.

OBJECTIVE: Breast cancer incidence is rising for women in low and middle income country (LMIC)s. Growing the health care workforce trained in clinical breast exam (CBE) is critical to mitigating breast cancer globally. We developed a CBE simulation training course and determined whether training on a low-fidelity (LF) simulation model results in similar skill acquisition as training on high-fidelity (HF) models in Rwanda. DESIGN: A single-center randomized educational crossover trial was implemented. A preintervention baseline exam (exam 1), followed by a lecture series (exam 2), and training sessions with assigned simulation models was implemented (exam 3)-participants then crossed over to their unassigned model (exam 4). The primary outcome of this study determined mean difference in CBE exam scores between HF and LF groups. Secondary outcomes identified any provider level traits and changes in overall scores. SETTING: The study was implemented at the University Teaching Hospital, Kigali (CHUK) in Rwanda, Africa from July 2014 to March 2015 PARTICIPANTS: Medical students, residents in surgery, obstetrics and gynecology, and internal medicine residents participated in a 1-day CBE simulation training course. RESULTS: A total of 107 individuals were analyzed in each arm of the study. Mean difference in exam scores between HF and LF models in exam 1 to 4 was not significantly different (exam 1 0.08 standard error (SE) = 0.47, p = 0.42; exam 2 0.86, SE = 0.69, p = 0.16; exam 3 0.03, SE = 0.38, p = 0.66; exam 4 0.10 SE = 0.37, p = 0.29). Overall exam scores improved from pre- to post-intervention. CONCLUSIONS: Mean difference in exams scores were not significantly different between participants trained with HF versus LF models. LF models can be utilized as cost effective teaching tools for CBE skill acquisition, in resource poor areas.

Are surgeons behind the scientific eight ball: Delayed acquisition of the NIH K08 mentored career development award.

BACKGROUND: Surgery residents complete their research training early in residency. Non-surgical trainees typically have research incorporated toward the last two years of their fellowship, conferring an advantage to apply for grants with recent research experience and preliminary data. METHODS: The NIH RePORTER database was queried for K08 awardees trained in medicine, pediatrics, and surgery from 2013 to 2017. 406 K08 recipients were identified and time from completion of clinical training to achieving a K08 award was measured. Data were compared using ANOVA and expressed as mean. P < 0.05 was considered significant. RESULTS: Surgeons took longer to obtain a K08 than those trained in internal medicine (surgery = 3.7 years, internal medicine = 2.58 years p < 0.0001)). All K08 recipients without a PhD took longer to obtain a K08 than recipients with a PhD (MD = 3.50 years and MD/PhD = 2.42 years (p=<0.0001). CONCLUSIONS: Surgeons take longer to achieve a K08 award than clinicians trained in internal medicine, possibly due to an inherent disadvantage in training structure.

A Randomized Cross-Over Trial Focused on Breast Core Needle Biopsy Skill Acquisition and Safety Using High Fidelity Versus Low Fidelity Simulation Models in Rwanda.

OBJECTIVE: Breast cancer is the most common cancer diagnosed in low and middle-income countries. Growing the number of health care personnel trained in diagnostic procedures like breast core needle biopsy (BCNB) is critical. We developed a BCNB simulation-training course that evaluated skill acquisition, confidence, and safety, comparing low-cost low fidelity (LF) models to expensive high fidelity (HF) models. DESIGN: A single-center randomized education crossover trial was implemented. Participants were randomized to HF or LF groups. A preintervention baseline exam followed by lectures and training sessions with a HF or LF model was implemented. A postintervention simulation exam was conducted, and participants crossed over to the other simulation model. SETTING: The study was implemented at the University Teaching Hospital, Kigali (CHUK) in Rwanda, Africa from October 2014 to March 2015. PARTICIPANTS: Residents training in surgery or obstetrics and gynecology participated in a 1-day BCNB training course. RESULTS: A total of 36 residents were analyzed, 19 in the HF arm and 17 in the LF arm. Mean difference in exam scores for HF and LF groups in the baseline exam (exam 1) (0.067, p = 0.94, standard error [SE] of 1.57) postintervention exam (exam 2) (1.85, SE 1.46, p = 0.33), and the crossover exam (exam 3) (4.39, SE = 1.90, p = 0.11) were not significantly different between HF and LF. Overall exam scores improved from pre- to postintervention. CONCLUSIONS: Our results indicate that mean difference in exams scores were not significantly different between residents trained with HF versus LF models. In resources poor areas-LF models can be utilized as effective teaching tools for skill acquisition for diagnostic surgical procedures.

The effect of gender on operative autonomy in general surgery residents.

BACKGROUND: Despite an increasing number of women in the field of surgery, bias regarding cognitive or technical ability may continue to affect the experience of female trainees differently than their male counterparts. This study examines the differences in the degree of operative autonomy given to female compared with male general surgery trainees. METHODS: A smartphone app was used to collect evaluations of operative autonomy measured using the 4-point Zwisch scale, which describes defined steps in the progression from novice ("show and tell") to autonomous surgeon ("supervision only"). Differences in autonomy between male and female residents were compared using hierarchical logistic regression analysis. RESULTS: A total of 412 residents and 524 faculty from 14 general surgery training programs evaluated 8,900 cases over a 9-month period. Female residents received less autonomy from faculty than did male residents overall (P < .001). Resident level of training and case complexity were the strongest predictors of autonomy. Even after controlling for potential confounding factors, including level of training, intrinsic procedural difficulty, patient-related case complexity, faculty sex, and training program environment, female residents still received less operative autonomy than their male counterparts. The greatest discrepancy was in the fourth year of training. CONCLUSION: There is a sex-based difference in the autonomy granted to general surgery trainees. This gender gap may affect female residents' experience in training and possibly their preparation for practice. Strategies need to be developed to help faculty and residents work together to overcome this gender gap.

"Show Me the Data": A Recipe for Quality Improvement Success in an Academic Surgical Department.

BACKGROUND: Surgeons in academic medical centers have traditionally taken a siloed approach to reducing postoperative complications. We initiated a project focusing on transparency and sharing of data to engage surgeons in collaborative quality improvement. Its key features were the development of a comprehensive department quality dashboard and the creation of the Clinical Operations Council that oversaw quality. The purpose of this study was to assess the impact of those efforts. STUDY DESIGN: We compared inpatient outcomes before and after our intervention, allowing one quarter as the diffusion period. The outcomes analyzed were: risk-adjusted length of stay, mortality, direct cost and unadjusted incidence of complications, and 30-day all-cause readmissions, as determined by the Vizient Clinical Database. We examined the outcomes of three groups: group 1 (surgery); group 2, all other surgical departments (other surgery); and group 3, all other patients (non-surgery). Two-tailed Student's t-test was used for analysis and p < 0.05 was considered statistically significant. RESULTS: Group 1 demonstrated statistically significant improvements in mortality (p = 0.01), length of stay (p = 0.002), cost (p = 0.0001), and complications (p = 0.02), and the all-cause readmission rate was unchanged, resulting in mean decrease of 0.55 length of stay days and direct cost savings of $2,300 per surgical admission. The comparison groups had only modest decreases in some of the analyzed outcomes and an increase in complication rates. CONCLUSIONS: These data suggest that a collaborative, data-driven, and transparent approach to assessing the quality of surgical care can yield significant improvements in patient outcomes.

Designing a Comprehensive Professional Development Program in a Surgery Department: Process, Measures, and Lessons Learned.

BACKGROUND: Departments within academic medical centers are typically comprised of clinical and research faculty, administrative staff, residents, and in many instances advance practice providers (APPs). Each member of these groups of people, at 1 point, took time, effort, and money to recruit, hire, and train. It is therefore important to consider ways to increase the "return on investment" of hiring each member of a department as well as maintaining a high level of department vitality. The Department of Surgery at Indiana University has never had a robust professional development program for all of its members. A challenge, therefore, presented itself of how best to increase faculty engagement in faculty development and to initiate opportunities for professional development for APPs and staff. INTERVENTION: We implemented a professional development program that focused on academic success with emphasis on teaching and leadership and tailored activities to meet the needs of each member. Professional development was promoted by targeting select groups of people within our department to engage rather than have members go to the effort of seeking development on their own. SETTING: The intervention occurred in the Department of Surgery at Indiana University which is comprised of ∼125 faculty, 100 residents and/or fellows, 60 APPs, and 19 lead business administrators for 6 divisions, all working within 5 downtown hospitals comprising the academic health center. RESULTS: Great effort has been placed into defining measures for each activity including measures of engagement, completion of deliverables, and tracking new leadership positions obtained by participants. Between 2014 and 2017, the number of faculty development activities that faculty attended has tripled since inception of our professional development program. CONCLUSIONS: For those looking to enhance or begin a professional development program, appointing a director or vice chair to oversee and champion the initiative is key. From our experience, putting effort first into developing a junior faculty development program to capture internal motivation early on is likely best. For leadership development, selecting faculty to form faculty learning cohorts worked well. Finally, to deal with the barriers of time and competing interests, building in protected time for professional development is essential.

Optimizing the integration of advanced practitioners in a department of surgery: An operational improvement model.

Physician assistants (PAs) and nurse practitioners (NPs) have established themselves as key members of the healthcare team to supplement practicing physicians in patient care. PAs and NPs are collectively referred to as "advanced providers" (APs) and work not only in primary care but in general surgery and surgical subspecialties. Studies have addressed AP integration into the profession of medicine and have examined cost and efficacy of APs, attitudes about APs among residents, and educational impact of APs, but very little literature exists that describes a formalized approach to AP integration into a department of surgery, specifically with AP/resident integration. The purpose of this paper is to describe an initiative for developing an operational improvement model for APs working with residents on surgical inpatient services in a large academic health center. The model consists of four components and each component is described in detail from discovery state towards continuous improvement. Formal professional development opportunities for APs as well as appointing a Clinical Director for Surgical APs have positively impacted AP integration into the department of surgery.

The business of educating the next generation of surgeons.

Surgical education community needs to be informed about how education is funded and how it is threatened. In order to explore these issues the Association of Surgical Education convened a panel with significant experience in managing surgery departments to discuss the business of surgical education. They specifically addressed methods to recognize and reward faculty, educate residents on safety, quality and cost, and increase departmental revenue. This information is important in the current educational environment where there is an increased need for institutions to find alternate revenue streams to sustain graduate medical education. It is also important to find additional revenue streams to fund new residency slots to accommodate the greater number medical students who have been admitted to medical schools in response to meet the projected shortage of physicians.

How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?

OBJECTIVE: To establish the number of operative performance observations needed for reproducible assessments of operative competency. BACKGROUND: Surgical training is transitioning from a time-based to a competency-based approach, but the number of assessments needed to reliably establish operative competency remains unknown. METHODS: Using a smart phone based operative evaluation application (SIMPL), residents from 13 general surgery training programs were evaluated performing common surgical procedures. Two competency metrics were investigated separately: autonomy and overall performance. Analyses were performed for laparoscopic cholecystectomy performances alone and for all operative procedures combined. Variance component analyses determined operative performance score variance attributable to resident operative competency and measurement error. Generalizability and decision studies determined number of assessments needed to achieve desired reliability (0.80 or greater) and determine standard errors of measurement. RESULTS: For laparoscopic cholecystectomy, 23 ratings are needed to achieve reproducible autonomy ratings and 17 ratings are needed to achieve reproducible overall operative performance ratings. For the undifferentiated mix of procedures, 60 ratings are needed to achieve reproducible autonomy ratings and 40 are needed for reproducible overall operative performance ratings. CONCLUSION: The number of observations needed to achieve reproducible assessments of operative competency far exceeds current certification requirements, yet remains an important and achievable goal. Attention should also be paid to the mix of cases and raters in order to assure fair judgments about operative competency and fair comparisons of trainees.

Is the operative autonomy granted to a resident consistent with operative performance quality.

BACKGROUND: We investigated attending surgeon decisions regarding resident operative autonomy, including situations where operative autonomy was discordant with performance quality. METHODS: Attending surgeons assessed operative performance and documented operative autonomy granted to residents from 14 general surgery residency programs. Concordance between performance and autonomy was defined as "practice ready performance/meaningfully autonomous" or "not practice ready/not meaningfully autonomous." Discordant circumstances were practice ready/not meaningfully autonomous or not practice ready/meaningfully autonomous. Resident training level, patient-related case complexity, procedure complexity, and procedure commonality were investigated to determine impact on autonomy. RESULTS: A total of 8,798 assessments were collected from 429 unique surgeons assessing 496 unique residents. Practice-ready and exceptional performances were 20 times more likely to be performed under meaningfully autonomous conditions than were other performances. Meaningful autonomy occurred most often with high-volume, easy and common cases, and less complex procedures. Eighty percent of assessments were concordant (38% practice ready/meaningfully autonomous and 42% not practice ready/not meaningfully autonomous). Most discordant assessments (13.8%) were not practice ready/meaningfully autonomous. For fifth-year residents, practice ready/not meaningfully autonomous ratings (9.7%) were more frequent than not practice ready/meaningfully autonomous ratings (7.5%). Ten surgeons (2.3%) failed to afford residents meaningful autonomy on any occasion. CONCLUSION: Resident operative performance quality is the most important determinant in attending surgeon decisions regarding resident autonomy.

Leaders Growing Leaders: Designing a Tier-Based Leadership Program for Surgeons.

BACKGROUND: Leadership has emerged as a crucial component of professional development for physicians in academic medicine. Most leadership skills can be learned and therefore best practices of delivering leadership development are in high demand. For practicing surgeons, specific strategies to teach leadership have been lacking. OBJECTIVE: The purpose of this paper is to describe the structure of a tier-based leadership development program called Leaders Growing Leaders, to identify the major curricular components to each tier including measures and outcomes, and to share lessons learned for those who may want to begin a similar leadership development program.

Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents.

BACKGROUND: Concerns exist regarding the competency of general surgery graduates with performing core general surgery procedures. Current competence assessment incorporates minimal procedural numbers requirements. METHODS: Based on the Zwisch scale we evaluated the level of autonomy achieved by categorical PGY1-5 general surgery residents at 14 U.S. general surgery resident training programs between September 1, 2015 and December 31, 2016. With 5 of the most commonly performed core general surgery procedures, we correlated the level of autonomy achieved by each resident with the number of procedures they had performed before the evaluation period, with the intent of identifying specific target numbers that would correlate with the achievement of meaningful autonomy for each procedure with most residents. RESULTS: Whereas a definitive target number was identified for laparoscopic appendectomy (i.e. 25), for the other 4 procedures studied (i.e. laparoscopic cholecystectomy, 52; open inguinal hernia repair, 42; ventral hernia repair, 35; and partial colectomy, 60), target numbers identified were less definitive and/or were higher than many residents will experience during their surgical residency training. CONCLUSIONS: We conclude that procedural target numbers are generally not effective in predicting procedural competence and should not be used as the basis for determining residents' readiness for independent practice.