Surgeons' Coaching Techniques in the Surgical Coaching for Operative Performance Enhancement (SCOPE) Program.

OBJECTIVE: To evaluate coaching techniques used by practicing surgeons who underwent dedicated coach training in a peer surgical coaching program. BACKGROUND: Surgical coaching is a developing strategy for improving surgeons' intraoperative performance. How to cultivate effective coaching skills among practicing surgeons is uncertain. METHODS: Through the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 surgeons within 4 US academic medical centers were assigned 1:1 into coach/coachee pairs. All attended a 3-hour Surgical Coaching Workshop-developed using evidence from the fields of surgery and education-then received weekly reminders. We analyzed workshop evaluations and audio transcripts of postoperative debriefs between coach/coachee pairs, co-coding themes based on established principles of effective coaching: (i) self-identified goals, (ii) collaborative analysis, (iii) constructive feedback, and (iv) action planning. Coaching principles were cross-referenced with intraoperative performance topics: technical, nontechnical, and teaching skills. RESULTS: For the 8 postoperative debriefs analyzed, mean duration was 24.4 min (range 7-47 minutes). Overall, 326 coaching examples were identified, demonstrating application of all 4 core principles of coaching. Constructive feedback (17.6 examples per debrief) and collaborative analysis (16.3) were utilized more frequently than goal-setting (3.9) and action planning (3.0). Debriefs focused more often on nontechnical skills (60%) than technical skills (32%) or teaching-specific skills (8%). Among surgeons who completed the workshop evaluation (82% completion rate), 90% rated the Surgical Coaching Workshop "good" or "excellent." CONCLUSIONS: Short-course coach trainings can help practicing surgeons use effective coaching techniques to guide their peers' performance improvement in a way that aligns with surgical culture.

Identifying Naturalistic Coaching Behavior Among Practicing Surgeons in the Operating Room.

OBJECTIVE: The aim of this study was to identify examples of naturalistic coaching behavior among practicing surgeons operating together by analyzing their intraoperative discussion. BACKGROUND: Opportunities to improve surgical performance are limited for practicing surgeons; surgical coaching is one strategy to address this need. To develop peer coaching programs that integrate with surgical culture, a better understanding is needed of how surgeons routinely discuss operative performance. METHODS: As part of a "co-surgery" quality improvement program, 20 faculty surgeons were randomized into 10 dyads who performed an operation together. Discourse analysis was conducted on transcribed intraoperative discussions. Themes were coded using an existing framework of surgical coaching principles (self-identified goals, collaborative analysis, constructive feedback, peer learning support) and surgical coaching content (technical skills, nontechnical skills). Coaching principles were cross-referenced with coaching content; c-coefficient measured the strength of association between pairs of themes. RESULTS: Overall, 44 unique coaching examples were identified in 10 operations. Of the 4 principles of surgical coaching, only self-identified goals and collaborative analysis were identified consistently. Self-identified goals were most associated with discussions regarding technical skills of "tissue exposure," "flow of operation," and "instrument handling" and the nontechnical skill "situation awareness." Collaborative analysis was most associated with discussions regarding technical skills of "respect for tissue" and "flow of operation" and nontechnical skills of "communication and teamwork." CONCLUSIONS: In naturalistic discussions between practicing surgeons in the operating room, numerous examples of unprompted coaching behavior were identified that target intraoperative performance. Prominent coaching gaps-constructive feedback and peer learning support-were also observed. Surgical coach trainings should address these gaps.

Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons' practice.

BACKGROUND: Evidence for surgical coaching has yet to demonstrate an impact on surgeons' practice. We evaluated a surgical coaching program by analyzing quantitative and qualitative data on surgeons' intraoperative performance. METHODS: In the 2018-2019 Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 practicing surgeons in multiple specialties at four academic medical centers were recruited to complete three peer coaching sessions, each comprising preoperative goal-setting, intraoperative observation, and postoperative debriefing. Coach and coachee rated the coachee's performance using modified Objective Structured Assessment of Technical Skills (OSATS, range 1-5) and Non-Technical Skills for Surgeons (NOTSS, range 4-16). We used generalized estimating equations to evaluate trends in skill ratings over time, adjusting for case difficulty, clinical experience, and coaching role. Upon program completion, we analyzed semi-structured interviews with individual participants regarding the perceived impact of coaching on their practice. RESULTS: Eleven of 23 coachees (48%) completed three coaching sessions, three (13%) completed two sessions, and six (26%) completed one session. Adjusted mean OSATS ratings did not vary over three coaching sessions (4.39 vs 4.52 vs 4.44, respectively; P = 0.655). Adjusted mean total NOTSS ratings also did not vary over three coaching sessions (15.05 vs 15.50 vs 15.08, respectively; P = 0.529). Regarding patient care, participants self-reported improved teamwork skills, communication skills, and awareness in and outside the operating room. Participants acknowledged the potential for coaching to improve burnout due to reduced intraoperative stress and enhanced peer support but also the potential to worsen burnout by adding to chronic work overload. CONCLUSIONS: Surgeons reported high perceived impact of peer coaching on patient care and surgeon well-being, although changes in coachees' technical and non-technical skills were not detected over three coaching sessions. While quantitative skill measurement warrants further study, longitudinal peer surgical coaching should be considered a meaningful strategy for surgeons' professional development.

Hospital Teaching Status and Medicare Expenditures for Complex Surgery.

OBJECTIVE: To evaluate the relationship between hospital teaching intensity, Medicare payments, and perioperative outcomes. BACKGROUND: Several emerging payment policies penalize hospitals for low-value healthcare. Teaching hospitals may be at a disadvantage given the perception that they deliver care less efficiently. METHODS: Using Medicare Provider and Analysis Review files, we studied patients from age 65 to 100 years who underwent abdominal aortic aneurysm (AAA) repair (n = 71,422), pulmonary resection (n = 93,056), or colectomy (n = 277,619) from 2009 to 2012. Patients' hospitals were categorized into quintiles of teaching intensity (very major, major, minor, very minor, and nonteaching hospitals) based on the resident-to-bed ratio. Risk-adjusted 30-day Medicare payments were price-standardized to account for graduate medical education payments, disproportionate share costs, and regional wage-index adjustments. Risk-adjusted perioperative outcomes were also assessed. RESULTS: Comparing risk-adjusted Medicare payments per episode of surgery, very major teaching hospitals were $14,145 more expensive than nonteaching hospitals for AAA repair ($45,570 vs $31,426; P < 0.001), $9,812 more expensive for pulmonary resection ($39,550 vs $29,738; P < 0.001), and $19,147 more expensive for colectomy ($51,893 vs $32,746; P < 0.001). However, after accounting for social subsidies and regional variation in Medicare spending, risk-adjusted Medicare payments were not statistically different between very major teaching hospitals and nonteaching hospitals for AAA repair ($29,946 vs $27,993; P = 0.18) and pulmonary resection ($25,407 vs $26,813; P = 1.00); a statistically significant but attenuated difference persisted for colectomy ($34,949 vs $30,352; P < 0.001). Very major teaching hospitals generally had higher risk-adjusted rates of serious complications and readmissions, but lower risk-adjusted rates of failure to rescue and 30-day mortality than did nonteaching hospitals. CONCLUSIONS: After price-standardization to account for intended differences in Medicare spending, risk-adjusted Medicare payments for an episode of surgical care were similar at teaching hospitals and nonteaching hospitals for three complex inpatient operations.

Safety considerations in learning new procedures: a survey of surgeons.

BACKGROUND: There exists a tension between surgical innovation and safety. The learning curve associated with the introduction of new procedures/technologies has been associated with preventable patient harm. Surgeon's perceptions regarding the safety of methods for learning new procedures/technologies are largely uncharacterized. MATERIALS AND METHODS: A survey was designed to evaluate surgeons' perceptions related to learning new procedures/technologies. This included clinical vignettes across two domains: (1) experience with an operation (e.g., colectomy) and (2) experience with a technology (e.g., laparoscopy). This study also focuses on a surgeon's perceptions of existing credentialing/privileging requirements. Participants were faculty surgeons (n = 150) at two large Midwestern academic health centers. RESULTS: Survey response rate was 77% (116/150). 69% of respondents believed the processes of credentialing/privileging is "far too relaxed" or "too relaxed" for ensuring patient safety. Surgeons most commonly indicated a mini-fellowship is required to learn a new laparoscopic procedure. However, that requirement differed based on a surgeon's prior experience with laparoscopy. For example, to learn laparoscopic colectomy, 35% of respondents felt a surgeon with limited laparoscopic experience should complete a mini-fellowship, whereas 3% felt this was necessary if the surgeon had extensive laparoscopic experience. In the latter scenario, most respondents felt a surgeon should scrub in cases performed by an expert (38%) or perform cases under a proctor's supervision (33%) when learning laparoscopic colectomy. CONCLUSIONS: Many surgeons believe existing hospital credentialing/privileging practices may be too relaxed. Moreover, surgeons believe the "one-size-fits-all" approach for training practicing surgeons may not protect patients from unsafe introduction of new procedures/technologies.

Insurance Status and Hospital Payer Mix Are Linked With Variation in Metastatic Site Resection in Patients With Advanced Colorectal Cancers.

BACKGROUND: Despite substantially improved survival with metastatic site resection in colorectal cancers, uptake of aggressive surgical approaches remains low among certain patients. It is unknown whether financial determinants of care, such as insurance status, play a role in this treatment gap. OBJECTIVE: We sought to evaluate the effect of insurance status on metastasectomy in patients with advanced colorectal cancers. DESIGN: This was a retrospective cohort study. SETTINGS: Using the National Cancer Data Base Participant User File, incident cases of colorectal cancer metastatic to the lung and/or liver with diagnosis from 2010 to 2013 were identified. PATIENTS: We identified 42,300 patients in our cohort with a mean age 64 years. MAIN OUTCOME MEASURES: Controlling for patient, tumor, and hospital characteristics, hierarchical regression was used to examine associations between hospital payer mix and metastatic site resection. Metastatic site resection occurred in 12.3% of all patients. RESULTS: Adjusting for patient and hospital fixed effects, we found that patients who were uninsured or on Medicaid were 38% less likely to undergo metastasectomy (OR = 0.62 (95% CI, 0.56-0.66)). Patients in hospitals with staff treating a high percentage of uninsured patients or patients with Medicaid were less likely to undergo metastasectomy, even after controlling for individual patient insurance status. LIMITATIONS: The study was limited by its retrospective design and the granularity and accuracy of the National Cancer Data Base. CONCLUSIONS: Differences in insurance status and hospital payer mix are associated with differences in rates of metastatic site resection in patients with colorectal cancer that is metastatic to the lung and/or liver. There is a need for improved access to metastatic site resection for individual patients who are uninsured or who have Medicaid insurance, as well as for all patients who seek care at hospitals treating a large proportion of patients who are uninsured or on Medicaid. Remedies for individual patients could include improved access to private insurance through employment or individual plans or improved reimbursement from Medicaid for this procedure. Strategies for patients at low-performing hospitals include selective referral to centers that perform mestastectomy more frequently when appropriate.

Designing a leadership development program for surgeons.

BACKGROUND: Although numerous leadership development programs (LDPs) exist in health care, no programs have been specifically designed to meet the needs of surgeons. This study aimed to elicit practicing surgeons' motivations and desired goals for leadership training to design an evidence-based LDP in surgery. MATERIALS AND METHODS: At a large academic health center, we conducted semistructured interviews with 24 surgical faculty members who voluntarily applied and were selected for participation in a newly created LDP. Transcriptions of the interviews were analyzed using analyst triangulation and thematic coding to extract major themes regarding surgeons' motivations and perceived needs for leadership knowledge and skills. Themes from interview responses were then used to design the program curriculum specifically to meet the leadership needs of surgical faculty. RESULTS: Three major themes emerged regarding surgeons' motivations for seeking leadership training: (1) Recognizing key gaps in their formal preparation for leadership roles; (2) Exhibiting an appetite for personal self-improvement; and (3) Seeking leadership guidance for career advancement. Participants' interviews revealed four specific domains of knowledge and skills that they indicated as desired takeaways from a LDP: (1) leadership and communication; (2) team building; (3) business acumen/finance; and (4) greater understanding of the health care context. CONCLUSIONS: Interviews with surgical faculty members identified gaps in prior leadership training and demonstrated concrete motivations and specific goals for participating in a formal leadership program. A LDP that is specifically tailored to address the needs of surgical faculty may benefit surgeons at a personal and institutional level.

Training Novice Raters to Assess Nontechnical Skills of Operating Room Teams.

OBJECTIVE: To our knowledge, no curricula have been described for training novice, nonclinician raters of nontechnical skills in the operating room (OR). We aimed to report the reliability of Oxford Non-Technical Skills (NOTECHS) ratings provided by novice raters who underwent a scalable curriculum for learning to assess nontechnical skills of OR teams. DESIGN: In-person training course to apply the NOTECHS framework to assessing OR teams' nontechnical skill performance, led by 2 facilitators and involving 5 partial-day sessions of didactic presentations, video simulation, and live OR observation with postassessment debriefing. NOTECHS ratings were submitted after each of 11 video scenarios and 8 live operations for the total NOTECHS team rating (including surgical/anesthesiology/nursing subteams) and for each NOTECHS skill category-situation awareness, problem solving and decision making, teamwork and cooperation, leadership and management. Inter-rater reliability was determined by calculating the intraclass correlation coefficient (ICC, range 0-1). SETTING: Training for outcome measurement during a quality improvement initiative focused on surgical safety in 3 public hospitals in Singapore. Two trainings were conducted in May 2019 and January 2020. PARTICIPANTS: Ten novice raters who were existing hospital staff and had overall minimal OR experience and no prior experience with nontechnical skill assessment. RESULTS: ICC for the total NOTECHS team rating was 0.89 (95% confidence interval [CI], 0.87-0.91). ICCs for each NOTECHS category were as follows: situation awareness, 0.83 (95% CI, 0.78-0.88); problem solving and decision-making, 0.76 (95% CI, 0.70-0.83); teamwork and cooperation, 0.84 (95% CI, 0.79-0.88); leadership and management, 0.81 (95% CI, 0.75-0.86). CONCLUSIONS: This training curriculum for nontechnical skill assessments of OR teams was associated with high inter-rater reliability from novice raters with minimal collective OR experience. Using scalable training materials to produce reliable measurements of OR team performance, this nontechnical skills assessment curriculum may contribute to future QI projects aimed at improving surgical safety.

Surgical coaching to achieve the ABMS vision for the future of continuing board certification.

In February 2019, the American Board of Medical Specialties (ABMS) released the final report of the Continuing Board Certification: Vision for the Future initiative, issuing strong recommendations to replace ineffective, traditional mechanisms for physicians' maintenance of certification with meaningful strategies that strengthen professional self-regulation and simultaneously engender public trust. The Vision report charges ABMS Member Boards, including the American Board of Surgery (ABS), to develop and implement a more formative, less summative approach to continuing certification. To realize the ABMS's Vision in surgery, new programs must support the assessment of surgeons' performance in practice, identification of individualized performance gaps, tailored goals to address those gaps, and execution of personalized action plans with accountability and longitudinal support. Peer surgical coaching, especially when paired with video-based assessment, provides a structured approach that can meet this need. Surgical coaching was one of the approaches to continuing professional development that was discussed at an ABS-sponsored retreat in January 2020; this commentary review provides an overview of that discussion. The professional surgical societies, in partnership with the ABS, are uniquely positioned to implement surgical coaching programs to support the continuing certification of their membership. In this article, we provide historical context for board certification in surgery, interpret how the ABMS's Vision applies to surgical performance, and highlight recent developments in video-based assessment and peer surgical coaching. We propose surgical coaching as a foundational strategy for accomplishing the ABMS's Vision for continuing board certification in surgery.

Non-technical skill assessments across levels of US surgical training.

BACKGROUND: To ensure safe patient care, regulatory bodies worldwide have incorporated non-technical skills proficiency in core competencies for graduation from surgical residency. We describe normative data on non-technical skill ratings of surgical residents across training levels using the US-adapted Non-Technical Skills for Surgeons (NOTSS-US) assessment tool. METHODS: We undertook an exploratory, prospective cohort study of 32 residents-interns (postgraduate year 1), junior residents (postgraduate years 2-3), and senior residents (postgraduate years 4-5)-across 3 US academic surgery residency programs. Faculty went through online training to rate residents, directly observed residents while operating together, then submitted NOTSS-US ratings on specific resident's intraoperative performance. Mean NOTSS-US ratings (total range 4-20, sum of category scores; situation awareness, decision-making, communication/teamwork, leadership each ranged 1-5, with 1=poor, 3=average, 5=excellent) were stratified by residents' training level and adjusted for resident-, rater-, and case-level variables, using mixed-effects linear regression. RESULTS: For 80 operations, the overall mean total NOTSS-US rating was 12.9 (standard deviation, 3.5). The adjusted mean total NOTSS-US rating was 16.0 for senior residents, 11.6 for junior residents, and 9.5 for interns. Adjusted differences for total NOTSS-US ratings were statistically significant across the following training levels: senior residents to interns (6.5; 95% confidence interval, 4.3-8.7; P < .001), senior to junior residents (4.4; 95% confidence interval, 2.5-6.2; P < .001), and junior residents to interns (2.1; 95% confidence interval, 0.3-3.9; P = .017). Differences in adjusted NOTSS-US ratings across residents' training levels persisted for individual NOTSS-US behavior categories. CONCLUSION: These data and online training materials can support US residency programs in determining competency-based performance milestones to develop surgical trainees' non-technical skills.

Optimizing the Implementation of Surgical Coaching Through Feedback From Practicing Surgeons.

Importance: Surgical coaching is maturing as a tangible strategy for surgeons' continuing professional development. Resources to spread this innovation are not yet widely available. Objective: To identify surgeon-derived implementation recommendations for surgical coaching programs from participants' exit interviews and ratings of their coaching interactions. Design, Setting, and Participants: This qualitative analysis of the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, a quality improvement intervention, was conducted at 4 US academic medical centers. Participants included 46 practicing surgeons. The SCOPE program ran from December 7, 2018, to October 31, 2019. Data were analyzed from November 1, 2019, to January 31, 2020. Interventions: Surgeons were assigned as either a coach or a coachee, and each coach was paired with 1 coachee by a local champion who knew the surgeons professionally. Coaching pairs underwent training and were instructed to complete 3 coaching sessions-consisting of preoperative goal setting, intraoperative observation, and postoperative debriefing-focused on intraoperative performance. Main Outcomes and Measures: Themes from the participants' exit interviews covering 3 major domains: (1) describing the experience, (2) coach-coachee relationship, and (3) facilitators and barriers to implementing surgical coaching. Surgeons' responses were stratified by the net promoter score (NPS), a scale ranging from 0 to 10 points, indicating how likely they were to recommend their coaching session to others, with 9 to 10 indicating promoters; 7 to 8, passives; and 0 to 6, detractors. Results: Among the 46 participants (36 men [78.3%]), 23 were interviewed (50.0%); thematic saturation was reached with 5 coach-coachee pairs (10 interviews). Overall, coaches and coachees agreed on key implementation recommendations for surgical coaching, including how to optimize coach-coachee relationships and facilitate productive coaching sessions. The NPS categories were associated with how participants experienced their own coaching sessions. Specifically, participants who reported excellent first sessions, had a coaching partner in the same clinical specialty, and were transparent about each other's intentions in the program tended to be promoters. Participants who described suboptimal first sessions, less clinical overlap, and unclear goals with their partner were more likely detractors. Conclusions and Relevance: These exit interviews with practicing surgeons offer critical insights for addressing cultural barriers and practical challenges for successful implementation of peer coaching programs focused on surgical performance improvement. With empirical evidence on optimizing coach-coachee relationships and facilitating participants' experience, organizations can establish effective coaching programs that enable meaningful continuous professional development for surgeons and ultimately enhance patient care.

Performance Coaching for Practicing Surgeons Enhancing Clinical Performance, Well-Being, and Trainee Experience.

Performance coaching can help surgeons, hospitals, and healthcare systems to continually improve patient care delivery by enhancing surgeons' professional development. Equally important, coaching also has great potential to combat burnout, promote physician well-being, and subsequently improve trainee education and experience. In this article, we discuss the rationale for, early evidence for, and ways to address implementation barriers for performance coaching among practicing surgeons and surgical trainees.

Using Smartphones to Capture Novel Recovery Metrics After Cancer Surgery.

Importance: Patient-generated health data captured from smartphone sensors have the potential to better quantify the physical outcomes of surgery. The ability of these data to discriminate between postoperative trends in physical activity remains unknown. Objective: To assess whether physical activity captured from smartphone accelerometer data can be used to describe postoperative recovery among patients undergoing cancer operations. Design, Setting, and Participants: This prospective observational cohort study was conducted from July 2017 to April 2019 in a single academic tertiary care hospital in the United States. Preoperatively, adults (age ≥18 years) who spoke English and were undergoing elective operations for skin, soft tissue, head, neck, and abdominal cancers were approached. Patients were excluded if they did not own a smartphone. Exposures: Study participants downloaded an application that collected smartphone accelerometer data continuously for 1 week preoperatively and 6 months postoperatively. Main Outcomes and Measures: The primary end points were trends in daily exertional activity and the ability to achieve at least 60 minutes of daily exertional activity after surgery among patients with vs without a clinically significant postoperative event. Postoperative events were defined as complications, emergency department presentations, readmissions, reoperations, and mortality. Results: A total of 139 individuals were approached. In the 62 enrolled patients, who were followed up for a median (interquartile range [IQR]) of 147 (77-179) days, there were no preprocedural differences between patients with vs without a postoperative event. Seventeen patients (27%) experienced a postoperative event. These patients had longer operations than those without a postoperative event (median [IQR], 225 [152-402] minutes vs 107 [68-174] minutes; P < .001), as well as greater blood loss (median [IQR], 200 [35-515] mL vs 25 [5-100] mL; P = .006) and more follow-up visits (median [IQR], 2 [2-4] visits vs 1 [1-2] visits; P = .002). Compared with mean baseline daily exertional activity, patients with a postoperative event had lower activity at week 1 (difference, -41.6 [95% CI, -75.1 to -8.0] minutes; P = .02), week 3 (difference, -40.0 [95% CI, -72.3 to -3.6] minutes; P = .03), week 5 (difference, -39.6 [95% CI, -69.1 to -10.1] minutes; P = .01), and week 6 (difference, -36.2 [95% CI, -64.5 to -7.8] minutes; P = .01) postoperatively. Fewer of these patients were able to achieve 60 minutes of daily exertional activity in the 6 weeks postoperatively (proportions: week 1, 0.40 [95% CI, 0.31-0.49]; P < .001; week 2, 0.49 [95% CI, 0.40-0.58]; P = .003; week 3, 0.39 [95% CI, 0.30-0.48]; P < .001; week 4, 0.47 [95% CI, 0.38-0.57]; P < .001; week 5, 0.51 [95% CI, 0.42-0.60]; P < .001; week 6, 0.73 [95% CI, 0.68-0.79] vs 0.43 [95% CI, 0.33-0.52]; P < .001). Conclusions and Relevance: Smartphone accelerometer data can describe differences in postoperative physical activity among patients with vs without a postoperative event. These data help objectively quantify patient-centered surgical recovery, which have the potential to improve and promote shared decision-making, recovery monitoring, and patient engagement.

"Development of an Academic Surgical Student Program for Enhancing Student-Faculty Engagement".

OBJECTIVE: We describe an innovative medical student surgery interest group and its influence on mentorship and career exploration. DESIGN: SCRUBS, created to promote interest in academic surgery, is student-led, with continual surgical faculty and resident involvement. Its 3-component programming focuses on clinical skills, research, and mentorship opportunities for medical students to get involved in academic surgery early in medical education. SETTING: The University of Michigan Medical School, Ann Arbor, MI. PARTICIPANTS: First through fourth year medical students, surgery residents, and attending surgeons. RESULTS: SCRUBS is a multifaceted student organization providing longitudinal exposure to various aspects of surgery and academic medicine. The group grew annually from 2010 to 2014, with students and faculty expressing positive feedback. Over the time period reviewed, we had a greater percentage of students applying into surgical specialties compared with the national average (16.8 vs 12% in 2014). The group supported and facilitated mentorship, clinical skills development, and research opportunities for interested students. CONCLUSIONS: This innovative surgery interest group has been well received by students and surgeons, and our institution has seen above-average interest in surgical careers. Early, preclinical mentorship and exposure provided by SCRUBS may contribute to this higher surgical interest.

Comparing the Learning Effectiveness of Healthcare Simulation in the Observer Versus Active Role: Systematic Review and Meta-Analysis.

STATEMENT: The benefits of observation in simulation-based education in healthcare are increasingly recognized. However, how it compares with active participation remains unclear. We aimed to compare effectiveness of observation versus active participation through a systematic review and meta-analysis. Effectiveness was defined using Kirkpatrick's 4-level model, namely, participants' reactions, learning outcomes, behavior changes, and patient outcomes. The peer-reviewed search strategy included 8 major databases and gray literature. Only randomized controlled trials were included. A total of 13 trials were included (426 active participants and 374 observers). There was no significant difference in reactions (Kirkpatrick level 1) to training between groups, but active participants learned (Kirkpatrick level 2) significantly better than observers (standardized mean difference = -0.2, 95% confidence interval = -0.37 to -0.02, P = 0.03). Only one study reported behavior change (Kirkpatrick level 3) and found no significant difference. No studies reported effects on patient outcomes (Kirkpatrick level 4). Further research is needed to understand how to effectively integrate and leverage the benefits of observation in simulation-based education in healthcare.

Strategies for New Skill Acquisition by Practicing Surgeons.

OBJECTIVE: To understand how practicing surgeons utilize available training methods, which methods are perceived as effective, and important barriers to using more effective methods. DESIGN: Online survey designed to characterize surgeon utilization and perception of available training methods. SETTING: Two large Midwestern academic health centers. PARTICIPANTS: 150 faculty surgeons. METHODS: Nominal values were compared using a McNemar's Test and Likert-like values were compared using a paired t-test (IBM SPSS Statistics v. 21.0; New York, NY). RESULTS: Survey response rate was 81% (122/150). 98% of surgeons reported learning a new procedure or technology after formal training. Many surgeons reported scrubbing in expert cases (78%) and self-directed study (66%), while few surgeons (6%) completed a mini-fellowship. The modalities used most commonly were scrubbing in expert cases (34%) and self-directed study (27%). Few surgeons (7%) believed self-directed study would be most effective, whereas 31% and 16% believed operating under supervision and mini-fellowships would be most effective, respectively. Surgeons believed more effective methods "would require too much time" or they had "confidence in their ability to implement safely." CONCLUSIONS: Practicing surgeons use a variety of training methods when learning new procedures and technologies, and there is disconnect between commonly used training methods and those deemed most effective. Confidence in surgeon's ability was cited as a reason for this discrepancy; and surgeons found time associated with more effective methods to be prohibitive.