Practical Guide to Common Flaws With Surgical Education Research.

This Guide to Statistics and Methods provides an overview of common flaws with surgical education research, including how to recognize and avoid them.

Surgical Resident Perspectives on the Preferred Qualities of Effective Intraoperative Teachers: A Qualitative Analysis.

PURPOSE: Intraoperative teaching is a critical component of surgery residents' education. Although prior studies have investigated best practices from the viewpoint of the expert educator, the perspective of the learner has been less explored. This study examined the ideal faculty teaching behaviors that optimize intraoperative teaching from the surgical residents' perspective. METHOD: Using a grounded theory method, this study explored perspectives on intraoperative faculty teaching qualities of 5 focus groups of categorical clinical general surgical residents of the same postgraduate year from June to August 2021. Focus group discussions were recorded, transcribed, and coded. Emerging themes were identified, along with their corresponding subthemes. RESULTS: Thirty-nine general surgery residents participated in the focus groups. Overall, 6 themes emerged regarding resident priorities of intraoperative teaching, with 10 subthemes. Themes included the following: (1) character, with subthemes of caring, respect for resident, and self-control; (2) intraoperative skill, with subthemes of clinical and operative skill and modeling leadership in the operating room; (3) instructional approach; (4) feedback, with subthemes of content of feedback and debriefing; (5) discernment of resident needs, with subthemes of managing expectations, individualizing instruction, and autonomy; and (6) variety of teachers. CONCLUSIONS: Certain tangible strategies, such as demonstrating genuine care for the learner, using clear directional words, and giving actionable feedback, were considered vital by residents. In the development of great surgical educators, the emphasis should not be on conformity to a single idealized teaching style but should celebrate and encourage diversity of personas and teaching styles within a department or program.

An Online, Modular Curriculum Enhances Surgical Education and Improves Learning Outcomes in East, Central, and Southern Africa: A Mixed-Methods Study.

Objective: We aimed to determine the impact of a standardized curriculum on learning outcomes for surgical trainees in East, Central, and Southern Africa (ECSA). Background: As surgical education expands throughout ECSA, there is a recognized need for a standardized curriculum. We previously described the design of a novel, large-scale, flipped-classroom, surgical curriculum for trainees in ECSA. Methods: In January 2020, the first year of curricular content for trainees of the College of Surgeons of ECSA was released, containing 11 monthly thematic topics, each with 2 to 5 weekly modular subtopics. We aimed to evaluate 3 outcomes utilizing data sources incorporated into the curriculum structure. Learner engagement was assessed by the number of trainees completing curriculum topics. User experience was evaluated using quantitative and qualitative feedback responses to embedded surveys for each content week. Curriculum impact on trainee examination performance was assessed by comparing certification examination scores stratified by the number of curricular topics each trainee completed. Results: Two hundred seventy-one trainees (96%) in 17 countries accessed at least 1 weekly module. Trainees completed a median of 9 topics (interquartile range: 6-10). The feedback survey response rate was 92% (5742/6233). Quantitative and qualitative responses were positive in overall module value (93.7% + 2.6%), amount of learning experienced (97.9% + 1.4%), confidence in achieving learning objectives (97.1% + 2.4%), and ease of use of the module (77.6% + 5.98%). Topic-related certification examination performance improved significantly with increased completion of thematic topics. Conclusions: A standardized surgical curriculum in ECSA demonstrated excellent trainee usage, positive feedback, and improved examination scores.

Design of a Novel Online, Modular, Flipped-classroom Surgical Curriculum for East, Central, and Southern Africa.

Objective: We describe a structured approach to developing a standardized curriculum for surgical trainees in East, Central, and Southern Africa (ECSA). Summary Background Data: Surgical education is essential to closing the surgical access gap in ECSA. Given its importance for surgical education, the development of a standardized curriculum was deemed necessary. Methods: We utilized Kern's 6-step approach to curriculum development to design an online, modular, flipped-classroom surgical curriculum. Steps included global and targeted needs assessments, determination of goals and objectives, the establishment of educational strategies, implementation, and evaluation. Results: Global needs assessment identified the development of a standardized curriculum as an essential next step in the growth of surgical education programs in ECSA. Targeted needs assessment of stakeholders found medical knowledge challenges, regulatory requirements, language variance, content gaps, expense and availability of resources, faculty numbers, and content delivery method to be factors to inform curriculum design. Goals emerged to increase uniformity and consistency in training, create contextually relevant material, incorporate best educational practices, reduce faculty burden, and ease content delivery and updates. Educational strategies centered on developing an online, flipped-classroom, modular curriculum emphasizing textual simplicity, multimedia components, and incorporation of active learning strategies. The implementation process involved establishing thematic topics and subtopics, the content of which was authored by regional surgeon educators and edited by content experts. Evaluation was performed by recording participation, soliciting user feedback, and evaluating scores on a certification examination. Conclusions: We present the systematic design of a large-scale, context-relevant, data-driven surgical curriculum for the ECSA region.

Cutting the fat: Utilizing LEAN methodology to improve rounding efficiency of surgical residents.

BACKGROUND: LEAN was developed by Toyota to provide a systematic way to eliminate waste and standardize processes. We sought to introduce LEAN methodology to surgical residents with the goal of increasing rounding efficiency. METHODS: A Kaizen event was used as a rounding efficiency improvement strategy. A multidisciplinary healthcare team participated in the event; first to identify the current state of rounds, second to create the ideal state. Value-stream maps were created and improved efficiency was seen by increased ratio of productive time to lead time. RESULTS: Two interventions were prioritized. The first introduced table rounds before walk rounds and the second changed the sign-out tool from Baton to Microsoft Word with file encryption. The ratio of productive time to lead time during morning rounds was higher after implementing these interventions (3.73 vs. 2.03). CONCLUSIONS: This Kaizen event introduced surgical trainees to LEAN and resulted in improved efficiency of morning rounds.

Development of a peer review of operative teaching process and assessment tool.

BACKGROUND: While teaching evaluation systems are common in academia, very little information is available regarding formal coaching and peer review of teaching performance in surgery. This article is a report on the development and implementation of a peer review of operative teaching program. METHODS: Our process was designed using a multistep sequential model which included developing a peer review of teaching instrument that was piloted to study the efficacy and utility of the tool. RESULTS: Thirty-nine peer reviews of teaching were conducted. Among the most frequent challenges that faculty identified were allowing residents to struggle/give autonomy, judging when to take over the case, communicating effectively, being patient, balancing education and patient safety, and giving feedback. CONCLUSIONS: Our peer review of teaching program is systematic, feasible, and can be adopted by other surgery departments. Faculty's identified strengths and challenges have been incorporated into our faculty development curricula.

The effect of gender on operative autonomy in general surgery residents.

BACKGROUND: Despite an increasing number of women in the field of surgery, bias regarding cognitive or technical ability may continue to affect the experience of female trainees differently than their male counterparts. This study examines the differences in the degree of operative autonomy given to female compared with male general surgery trainees. METHODS: A smartphone app was used to collect evaluations of operative autonomy measured using the 4-point Zwisch scale, which describes defined steps in the progression from novice ("show and tell") to autonomous surgeon ("supervision only"). Differences in autonomy between male and female residents were compared using hierarchical logistic regression analysis. RESULTS: A total of 412 residents and 524 faculty from 14 general surgery training programs evaluated 8,900 cases over a 9-month period. Female residents received less autonomy from faculty than did male residents overall (P < .001). Resident level of training and case complexity were the strongest predictors of autonomy. Even after controlling for potential confounding factors, including level of training, intrinsic procedural difficulty, patient-related case complexity, faculty sex, and training program environment, female residents still received less operative autonomy than their male counterparts. The greatest discrepancy was in the fourth year of training. CONCLUSION: There is a sex-based difference in the autonomy granted to general surgery trainees. This gender gap may affect female residents' experience in training and possibly their preparation for practice. Strategies need to be developed to help faculty and residents work together to overcome this gender gap.

Optimizing the integration of advanced practitioners in a department of surgery: An operational improvement model.

Physician assistants (PAs) and nurse practitioners (NPs) have established themselves as key members of the healthcare team to supplement practicing physicians in patient care. PAs and NPs are collectively referred to as "advanced providers" (APs) and work not only in primary care but in general surgery and surgical subspecialties. Studies have addressed AP integration into the profession of medicine and have examined cost and efficacy of APs, attitudes about APs among residents, and educational impact of APs, but very little literature exists that describes a formalized approach to AP integration into a department of surgery, specifically with AP/resident integration. The purpose of this paper is to describe an initiative for developing an operational improvement model for APs working with residents on surgical inpatient services in a large academic health center. The model consists of four components and each component is described in detail from discovery state towards continuous improvement. Formal professional development opportunities for APs as well as appointing a Clinical Director for Surgical APs have positively impacted AP integration into the department of surgery.

Designing a Comprehensive Professional Development Program in a Surgery Department: Process, Measures, and Lessons Learned.

BACKGROUND: Departments within academic medical centers are typically comprised of clinical and research faculty, administrative staff, residents, and in many instances advance practice providers (APPs). Each member of these groups of people, at 1 point, took time, effort, and money to recruit, hire, and train. It is therefore important to consider ways to increase the "return on investment" of hiring each member of a department as well as maintaining a high level of department vitality. The Department of Surgery at Indiana University has never had a robust professional development program for all of its members. A challenge, therefore, presented itself of how best to increase faculty engagement in faculty development and to initiate opportunities for professional development for APPs and staff. INTERVENTION: We implemented a professional development program that focused on academic success with emphasis on teaching and leadership and tailored activities to meet the needs of each member. Professional development was promoted by targeting select groups of people within our department to engage rather than have members go to the effort of seeking development on their own. SETTING: The intervention occurred in the Department of Surgery at Indiana University which is comprised of ∼125 faculty, 100 residents and/or fellows, 60 APPs, and 19 lead business administrators for 6 divisions, all working within 5 downtown hospitals comprising the academic health center. RESULTS: Great effort has been placed into defining measures for each activity including measures of engagement, completion of deliverables, and tracking new leadership positions obtained by participants. Between 2014 and 2017, the number of faculty development activities that faculty attended has tripled since inception of our professional development program. CONCLUSIONS: For those looking to enhance or begin a professional development program, appointing a director or vice chair to oversee and champion the initiative is key. From our experience, putting effort first into developing a junior faculty development program to capture internal motivation early on is likely best. For leadership development, selecting faculty to form faculty learning cohorts worked well. Finally, to deal with the barriers of time and competing interests, building in protected time for professional development is essential.

Identifying Managerial Roles of General Surgery Coordinators: Making the Case for Utilization of a Standardized Job Description Framework.

OBJECTIVE: Residency coordinators are valuable members of the education leadership administration. In General Surgery, program directors must devote time to both their clinical practice and as the leader of the education program for surgical residents. With the introduction of competencies and the Next Accreditation System, the responsibilities of training programs have increased, with much of the necessary day to day management being driven by the residency coordinator. The purpose of this study was to identify the current roles of a residency coordinator in surgery to determine appropriate language for a standardized job description that accurately describes the responsibilities of a program coordinator. DESIGN AND PARTICIPANTS: A survey was created and distributed via email to 317 general surgery program coordinators in programs with continued, initial, or pre-accreditation status by the ACGME in October-December 2017. Questions were asked about coordinator demographics, ADS involvement, and communication with program director, recruitment, and professional development. 223 coordinators (70%) completed the survey. RESULTS: Thirty-five percent of coordinators reported that their program director expects them to complete the annual ADS update in its entirety with a final review by the program director before submission, whereas 15% stated that the program director expects the program coordinator to input, update, and submit the annual ADS update without oversite from the program director. Fifty percent of program coordinators speak with their program director 2 to 4 days a week, whereas 38% speak with their program director daily. Eighty-nine percent of coordinators reported that their program directors trust them to make appropriate administrative decisions during scheduled or emergent absences. Sixty-nine percent of coordinators strongly agreed that they assist their program directors with collating and analyzing recruitment data post-recruitment season. Eighty-six percent of coordinators regularly participate in one or more professional development activities. Forty-six percent of coordinators stated that they oversee administrative staff in their office, division, or department. CONCLUSION: Given the current makeup of today's residency coordinator in general surgery programs, the need for baseline qualifications and a standardized job description allowing for recruitment and retention of a coordinator capable of managing a residency along with a program director. The data from our survey indicate that most coordinators currently perform tasks and take on responsibilities of a manager, but they hold current job descriptions that do not adequately reflect the role. The current proposed ACGME revisions state that there must be a program coordinator for a residency program, citing the coordinator as an integral member of the residency leadership team. Therefore, human resource departments need a job description that identifies level of responsibility, contribution, leadership, and management required of a program coordinator.

Is the operative autonomy granted to a resident consistent with operative performance quality.

BACKGROUND: We investigated attending surgeon decisions regarding resident operative autonomy, including situations where operative autonomy was discordant with performance quality. METHODS: Attending surgeons assessed operative performance and documented operative autonomy granted to residents from 14 general surgery residency programs. Concordance between performance and autonomy was defined as "practice ready performance/meaningfully autonomous" or "not practice ready/not meaningfully autonomous." Discordant circumstances were practice ready/not meaningfully autonomous or not practice ready/meaningfully autonomous. Resident training level, patient-related case complexity, procedure complexity, and procedure commonality were investigated to determine impact on autonomy. RESULTS: A total of 8,798 assessments were collected from 429 unique surgeons assessing 496 unique residents. Practice-ready and exceptional performances were 20 times more likely to be performed under meaningfully autonomous conditions than were other performances. Meaningful autonomy occurred most often with high-volume, easy and common cases, and less complex procedures. Eighty percent of assessments were concordant (38% practice ready/meaningfully autonomous and 42% not practice ready/not meaningfully autonomous). Most discordant assessments (13.8%) were not practice ready/meaningfully autonomous. For fifth-year residents, practice ready/not meaningfully autonomous ratings (9.7%) were more frequent than not practice ready/meaningfully autonomous ratings (7.5%). Ten surgeons (2.3%) failed to afford residents meaningful autonomy on any occasion. CONCLUSION: Resident operative performance quality is the most important determinant in attending surgeon decisions regarding resident autonomy.

Leaders Growing Leaders: Designing a Tier-Based Leadership Program for Surgeons.

BACKGROUND: Leadership has emerged as a crucial component of professional development for physicians in academic medicine. Most leadership skills can be learned and therefore best practices of delivering leadership development are in high demand. For practicing surgeons, specific strategies to teach leadership have been lacking. OBJECTIVE: The purpose of this paper is to describe the structure of a tier-based leadership development program called Leaders Growing Leaders, to identify the major curricular components to each tier including measures and outcomes, and to share lessons learned for those who may want to begin a similar leadership development program.

Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents.

BACKGROUND: Concerns exist regarding the competency of general surgery graduates with performing core general surgery procedures. Current competence assessment incorporates minimal procedural numbers requirements. METHODS: Based on the Zwisch scale we evaluated the level of autonomy achieved by categorical PGY1-5 general surgery residents at 14 U.S. general surgery resident training programs between September 1, 2015 and December 31, 2016. With 5 of the most commonly performed core general surgery procedures, we correlated the level of autonomy achieved by each resident with the number of procedures they had performed before the evaluation period, with the intent of identifying specific target numbers that would correlate with the achievement of meaningful autonomy for each procedure with most residents. RESULTS: Whereas a definitive target number was identified for laparoscopic appendectomy (i.e. 25), for the other 4 procedures studied (i.e. laparoscopic cholecystectomy, 52; open inguinal hernia repair, 42; ventral hernia repair, 35; and partial colectomy, 60), target numbers identified were less definitive and/or were higher than many residents will experience during their surgical residency training. CONCLUSIONS: We conclude that procedural target numbers are generally not effective in predicting procedural competence and should not be used as the basis for determining residents' readiness for independent practice.

What factors influence attending surgeon decisions about resident autonomy in the operating room?

BACKGROUND: Educating residents in the operating room requires balancing patient safety, operating room efficiency demands, and resident learning needs. This study explores 4 factors that influence the amount of autonomy supervising surgeons afford to residents. METHODS: We evaluated 7,297 operations performed by 487 general surgery residents and evaluated by 424 supervising surgeons from 14 training programs. The primary outcome measure was supervising surgeon autonomy granted to the resident during the operative procedure. Predictor variables included resident performance on that case, supervising surgeon history with granting autonomy, resident training level, and case difficulty. RESULTS: Resident performance was the strongest predictor of autonomy granted. Typical autonomy by supervising surgeon was the second most important predictor. Each additional factor led to a smaller but still significant improvement in ability to predict the supervising surgeon's autonomy decision. The 4 factors together accounted for 54% of decision variance (r = 0.74). CONCLUSION: Residents' operative performance in each case was the strongest predictor of how much autonomy was allowed in that case. Typical autonomy granted by the supervising surgeon, the second most important predictor, is unrelated to resident proficiency and warrants efforts to ensure that residents perform each procedure with many different supervisors.

Readiness of US General Surgery Residents for Independent Practice.

OBJECTIVE: This study evaluates the current state of the General Surgery (GS) residency training model by investigating resident operative performance and autonomy. BACKGROUND: The American Board of Surgery has designated 132 procedures as being "Core" to the practice of GS. GS residents are expected to be able to safely and independently perform those procedures by the time they graduate. There is growing concern that not all residents achieve that standard. Lack of operative autonomy may play a role. METHODS: Attendings in 14 General Surgery programs were trained to use a) the 5-level System for Improving and Measuring Procedural Learning (SIMPL) Performance scale to assess resident readiness for independent practice and b) the 4-level Zwisch scale to assess the level of guidance (ie, autonomy) they provided to residents during specific procedures. Ratings were collected immediately after cases that involved a categorical GS resident. Data were analyzed using descriptive statistics and supplemented with Bayesian ordinal model-based estimation. RESULTS: A total of 444 attending surgeons rated 536 categorical residents after 10,130 procedures. Performance: from the first to the last year of training, the proportion of Performance ratings for Core procedures (n = 6931) at "Practice Ready" or above increased from 12.3% to 77.1%. The predicted probability that a typical trainee would be rated as Competent after performing an average Core procedure on an average complexity patient during the last week of residency training is 90.5% (95% CI: 85.7%-94%). This falls to 84.6% for more complex patients and to less than 80% for more difficult Core procedures. Autonomy: for all procedures, the proportion of Zwisch ratings indicating meaningful autonomy ("Passive Help" or "Supervision Only") increased from 15.1% to 65.7% from the first to the last year of training. For the Core procedures performed by residents in their final 6 months of training (cholecystectomy, inguinal/femoral hernia repair, appendectomy, ventral hernia repair, and partial colectomy), the proportion of Zwisch ratings (n = 357) indicating near-independence ("Supervision Only") was 33.3%. CONCLUSIONS: US General Surgery residents are not universally ready to independently perform Core procedures by the time they complete residency training. Progressive resident autonomy is also limited. It is unknown if the amount of autonomy residents do achieve is sufficient to ensure readiness for the entire spectrum of independent practice.

Impact of integrated programs on general surgery operative volume.

BACKGROUND: Integrated residencies are now commonplace, co-existing with categorical general surgery residencies. The purpose of this study was to define the impact of integrated programs on categorical general surgery operative volume. METHODS: Case logs from categorical general, integrated plastics, vascular, and thoracic surgery residents from a single institution from 2008 to 2016 were collected and analyzed. RESULTS: Integrated residents have increased the number of cases they perform that would have previously been general surgery resident cases from 11 in 2009-2010 to 1392 in 2015-2016. Despite this, there was no detrimental effect on total major cases of graduating chief residents. CONCLUSIONS: Multiple integrated programs can co-exist with a general surgery program through careful collaboration and thoughtful consideration to longitudinal needs of individual trainees. As additional programs continue to be created, both integrated and categorical program directors must continue to collaborate to insure the integrity of training for all residents.

The Role of PhD Faculty in Advancing Research in Departments of Surgery.

OBJECTIVE: To determine the academic contribution as measured by number of publications, citations, and National Institutes of Health (NIH) funding from PhD scientists in US departments of surgery. SUMMARY BACKGROUND DATA: The number of PhD faculty working in US medical school clinical departments now exceeds the number working in basic science departments. The academic impact of PhDs in surgery has not been previously evaluated. METHODS: Academic metrics for 3850 faculties at the top 55 NIH-funded university and hospital-based departments of surgery were collected using NIH RePORTER, Scopus, and departmental websites. RESULTS: MD/PhDs and PhDs had significantly higher numbers of publications and citations than MDs, regardless of academic or institutional rank. PhDs had the greatest proportion of NIH funding compared to both MDs and MD/PhDs. Across all academic ranks, 50.2% of PhDs had received NIH funding compared with 15.2% of MDs and 33.9% of MD/PhDs (P < 0.001). The proportion of PhDs with NIH funding in the top 10 departments did not differ from those working in departments ranked 11 to 50 (P = 0.456). A greater percentage of departmental PhD faculty was associated with increased rates of MD funding. CONCLUSIONS: The presence of dedicated research faculty with PhDs supports the academic mission of surgery departments by increasing both NIH funding and scholarly productivity. In contrast to MDs and MD/PhDs, PhDs seem to have similar levels of academic output and funding independent of the overall NIH funding environment of their department. This suggests that research programs in departments with limited resources may be enhanced by the recruitment of PhD faculty.

The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial.

PURPOSE: Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS: Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS: A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS: SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed.

Identification of behaviors and techniques for promoting autonomy in the operating room.

BACKGROUND: To further understand how faculty promote resident autonomy in the operating room (OR), we explored their perceptions, and those of senior residents, on the behaviors and techniques they employ to foster independence. METHODS: Twenty postgraduate year PGY4 and PGY5 residents were asked to list 3 general surgery faculty who give the most and least autonomy to residents in the OR. Two focus groups were conducted with residents to identify behaviors and techniques for promoting autonomy from the resident perspective. Individual semistructured interviews were conducted with top-rated and low-rated faculty to identify behaviors and techniques from the attending perspective. RESULTS: Behaviors and techniques faculty exhibit to promote increased autonomy included allowing residents time to struggle, letting residents begin case themselves, and forcing residents to think about steps of procedure before case. The main "triggers" for granting autonomy were increased familiarity and trust with resident's capabilities, how well resident comes prepared to OR, and faculty confidence that they can "fix anything." Reasons why low-rated faculty seldom give autonomy included feeling a deep moral obligation for personally seeing patient through the operation, residents not coming fully prepared to perform case, and being young in their own maturation process. CONCLUSION: Our results suggest that increased autonomy depends greatly on establishing a trusting relationship between faculty and resident; a partnership that can only happen when time is given for trust to mature. Program directors must work to refine the training paradigm in order to build relationships. Residents can also be coached to demonstrate increased OR preparedness.