The APGO Academic Scholars and Leaders Program: Supporting obstetrics and gynecology faculty development for 20 years.

PURPOSE: We evaluate the impacts of the Academic Scholars and Leaders (ASL) Program in achieving 3 key objectives: treatment of education as a scholarly pursuit, improved education leadership, and career advancement. MATERIALS AND METHODS: We report on the twenty-year experience of the ASL Program-a national, longitudinal faculty development program of the Association of Professors of Obstetrics and Gynecology (APGO) covering instruction, curriculum development/program evaluation, assessment/feedback, leadership/professional development, and educational scholarship. We conducted a cross-sectional, online survey of ASL participants who graduated in 1999-2017. We sought evidence of impact using Kirkpatrick's 4-level framework. Descriptive quantitative data were analyzed, and open-ended comments were organized using content analysis. RESULTS: 64% (260) of graduates responded. The vast majority (96%) felt the program was extremely worthwhile (Kirkpatrick level 1). Graduates cited learned skills they had applied to their work, most commonly curricular development (48%) and direct teaching (38%) (Kirkpatrick 2&3 A). Since participation, 82% of graduates have held institutional, education-focused leadership roles (Kirkpatrick 3B). Nineteen percent had published the ASL project as a manuscript and 46% additional education papers (Kirkpatrick 3B). CONCLUSIONS: The APGO ASL program has been associated with successful outcomes in treatment of education as a scholarly pursuit, education leadership, and career advancement. Going forward, APGO is considering ways to diversify the ASL community and to support educational research training.

Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment.

OBJECTIVE: We have previously demonstrated the feasibility and validity of a smartphone-based system called Procedural Autonomy and Supervision System (PASS), which uses the Zwisch autonomy scale to facilitate assessment of the operative performances of surgical residents and promote progressive autonomy. To determine whether the use of PASS in a general surgery residency program is associated with any negative consequences, we tested the null hypothesis that PASS implementation at our institution would not negatively affect resident or faculty satisfaction in the operating room (OR) nor increase mean OR times for cases performed together by residents and faculty. METHODS: Mean OR times were obtained from the electronic medical record at Northwestern Memorial Hospital for the 20 procedures most commonly performed by faculty members with residents before and after PASS implementation. OR times were compared via two-sample t-test. The OR Educational Environment Measure tool was used to assess OR satisfaction with all clinically active general surgery residents (n = 31) and full-time general surgery faculty members (n = 27) before and after PASS implementation. Results were compared using the Mann-Whitney rank sum test. RESULTS: A significant prolongation in mean OR time between control and study period was found for only 1 of the 20 operative procedures performed at least 20 times by participating faculty members with residents. Based on the overall survey score, no significant differences were found between resident and faculty responses to the OR Educational Environment Measure survey before and after PASS implementation. When individual survey items were compared, while no differences were found with resident responses, differences were noted with faculty responses for 7 of the 35 items addressed although after Bonferroni correction none of these differences remained significant. CONCLUSIONS: Our data suggest that PASS does not increase mean OR times for the most commonly performed procedures. Resident OR satisfaction did not significantly change during PASS implementation, whereas some changes in faculty satisfaction were noted suggesting that PASS implementation may have had some negative effect with them. Although the effect on faculty satisfaction clearly requires further investigation, our findings support that use of an autonomy-based OR performance assessment system such as PASS does not appear to have a major negative influence on OR times nor OR satisfaction.

Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents.

BACKGROUND: Concerns exist regarding the competency of general surgery graduates with performing core general surgery procedures. Current competence assessment incorporates minimal procedural numbers requirements. METHODS: Based on the Zwisch scale we evaluated the level of autonomy achieved by categorical PGY1-5 general surgery residents at 14 U.S. general surgery resident training programs between September 1, 2015 and December 31, 2016. With 5 of the most commonly performed core general surgery procedures, we correlated the level of autonomy achieved by each resident with the number of procedures they had performed before the evaluation period, with the intent of identifying specific target numbers that would correlate with the achievement of meaningful autonomy for each procedure with most residents. RESULTS: Whereas a definitive target number was identified for laparoscopic appendectomy (i.e. 25), for the other 4 procedures studied (i.e. laparoscopic cholecystectomy, 52; open inguinal hernia repair, 42; ventral hernia repair, 35; and partial colectomy, 60), target numbers identified were less definitive and/or were higher than many residents will experience during their surgical residency training. CONCLUSIONS: We conclude that procedural target numbers are generally not effective in predicting procedural competence and should not be used as the basis for determining residents' readiness for independent practice.

What factors influence attending surgeon decisions about resident autonomy in the operating room?

BACKGROUND: Educating residents in the operating room requires balancing patient safety, operating room efficiency demands, and resident learning needs. This study explores 4 factors that influence the amount of autonomy supervising surgeons afford to residents. METHODS: We evaluated 7,297 operations performed by 487 general surgery residents and evaluated by 424 supervising surgeons from 14 training programs. The primary outcome measure was supervising surgeon autonomy granted to the resident during the operative procedure. Predictor variables included resident performance on that case, supervising surgeon history with granting autonomy, resident training level, and case difficulty. RESULTS: Resident performance was the strongest predictor of autonomy granted. Typical autonomy by supervising surgeon was the second most important predictor. Each additional factor led to a smaller but still significant improvement in ability to predict the supervising surgeon's autonomy decision. The 4 factors together accounted for 54% of decision variance (r = 0.74). CONCLUSION: Residents' operative performance in each case was the strongest predictor of how much autonomy was allowed in that case. Typical autonomy granted by the supervising surgeon, the second most important predictor, is unrelated to resident proficiency and warrants efforts to ensure that residents perform each procedure with many different supervisors.

Readiness of US General Surgery Residents for Independent Practice.

OBJECTIVE: This study evaluates the current state of the General Surgery (GS) residency training model by investigating resident operative performance and autonomy. BACKGROUND: The American Board of Surgery has designated 132 procedures as being "Core" to the practice of GS. GS residents are expected to be able to safely and independently perform those procedures by the time they graduate. There is growing concern that not all residents achieve that standard. Lack of operative autonomy may play a role. METHODS: Attendings in 14 General Surgery programs were trained to use a) the 5-level System for Improving and Measuring Procedural Learning (SIMPL) Performance scale to assess resident readiness for independent practice and b) the 4-level Zwisch scale to assess the level of guidance (ie, autonomy) they provided to residents during specific procedures. Ratings were collected immediately after cases that involved a categorical GS resident. Data were analyzed using descriptive statistics and supplemented with Bayesian ordinal model-based estimation. RESULTS: A total of 444 attending surgeons rated 536 categorical residents after 10,130 procedures. Performance: from the first to the last year of training, the proportion of Performance ratings for Core procedures (n = 6931) at "Practice Ready" or above increased from 12.3% to 77.1%. The predicted probability that a typical trainee would be rated as Competent after performing an average Core procedure on an average complexity patient during the last week of residency training is 90.5% (95% CI: 85.7%-94%). This falls to 84.6% for more complex patients and to less than 80% for more difficult Core procedures. Autonomy: for all procedures, the proportion of Zwisch ratings indicating meaningful autonomy ("Passive Help" or "Supervision Only") increased from 15.1% to 65.7% from the first to the last year of training. For the Core procedures performed by residents in their final 6 months of training (cholecystectomy, inguinal/femoral hernia repair, appendectomy, ventral hernia repair, and partial colectomy), the proportion of Zwisch ratings (n = 357) indicating near-independence ("Supervision Only") was 33.3%. CONCLUSIONS: US General Surgery residents are not universally ready to independently perform Core procedures by the time they complete residency training. Progressive resident autonomy is also limited. It is unknown if the amount of autonomy residents do achieve is sufficient to ensure readiness for the entire spectrum of independent practice.

The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial.

PURPOSE: Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS: Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS: A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS: SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed.

Beta Test of Web-Based Virtual Patient Decision-Making Exercises for Residents Demonstrates Discriminant Validity and Learning.

INTRODUCTION: Correct clinical decision-making is a key competency of surgical trainees. The purpose of this study was to assess validity and effect on resident decision-making accuracy of web-based virtual patient case scenarios in general surgery training. MATERIAL AND METHODS: During the 2013-2014 academic year, the use of web-based virtual patient scenarios for teaching and assessment of resident critical thinking and decision-making was assessed in 2 urban university-based residency programs. In all, 71 residents (PGY [postgraduate year] 1 = 21, PGY2 = 11, PGY3 = 14, PGY4 = 13, and PGY5 = 12) took the cases over the course of the academic year. Cases were made available to the residents online 1 week before a scheduled debriefing conference with a faculty facilitator and were completed by residents individually on their own schedule. At the completion of each case attempt, residents were given a computer-generated score and feedback. Residents were allowed to repeat the cases before the debriefing if they wished. Cases were required to be completed by 48 hours before the conference, at which time a faculty report was computer generated that measured group and individual performance and identified the frequency of errors in decision-making. This report was reviewed with the residents in the faculty debriefing, and teaching focused on the knowledge gaps identified in the reports. RESULTS: The mean percentage of assigned cases completed by categorical residents was 85.7%. Mean score (maximum possible = 100) on the cases increased by resident year (PGY1 = 45.3, PGY2 = 49.3, PGY3 = 53.6, PGY4 = 57.5, and PGY5 = 61.8), a 25% increase between PGY1 and PGY5 (p < 0.001 by analysis of variance). In all, 45 (63%) residents chose to repeat at least 1 case before the debriefing. The number of repetitions of individuals on the same case varied from a minimum of 1 to a maximum of 5. On repeated cases, mean scores rose (attempt 1 = 22.6, attempt 2 = 69.3, attempt 3 = 72.1, attempt 4 = 77.5, attempt 5 = 100, p < 0.0001 by analysis of variance). Paired t tests on case repetition using each resident as his-her own control showed that scores rose by 46 points between attempt 1 and attempt 2 (p < 0.001). CONCLUSIONS: (1) In a beta test of web-based scenarios that teach and assess clinical decision-making, resident scores improved by 25% from PGY 1 to PGY5 in a stepwise and statistically significant manner, suggesting that such exercises could serve as milestones for competency assessment. Additional studies are needed to acquire evidence for other forms of validity. (2) Repetition of cases after feedback led to highly significant increases in performance, suggesting that requiring repeated training to reach defined levels of competence is practical.

Using just-in-time teaching and peer instruction in a residency program's core curriculum: enhancing satisfaction, engagement, and retention.

PURPOSE: To assess use of the combined just-in-time teaching (JiTT) and peer instruction (PI) instructional strategy in a residency program's core curriculum. METHOD: In 2010-2011, JiTT/PI was piloted in 31 core curriculum sessions taught by 22 faculty in the Northwestern University Feinberg School of Medicine's general surgery residency program. JiTT/PI required preliminary and categorical residents (n=31) to complete Web-based study questions before weekly specialty topic sessions. Responses were examined by faculty members "just in time" to tailor session content to residents' learning needs. In the sessions, residents answered multiple-choice questions (MCQs) using clickers and engaged in PI. Participants completed surveys assessing their perceptions of JiTT/PI. Videos were coded to assess resident engagement time in JiTT/PI sessions versus prior lecture-based sessions. Responses to topic session MCQs repeated in review sessions were evaluated to study retention. RESULTS: More than 70% of resident survey respondents indicated that JiTT/PI aided in the learning of key points. At least 90% of faculty survey respondents reported positive perceptions of aspects of the JiTT/PI strategy. Resident engagement time for JiTT/PI sessions was significantly greater than for prior lecture-based sessions (z=-2.4, P=.016). Significantly more review session MCQ responses were correct for residents who had attended corresponding JiTT/PI sessions than for residents who had not (chi-square=13.7; df=1; P<.001). CONCLUSIONS: JiTT/PI increased learner participation, learner retention, and the amount of learner-centered time. JiTT/PI represents an effective approach for meaningful and active learning in core curriculum sessions.

Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.

PURPOSE: The existing methods for evaluating resident operative performance interrupt the workflow of the attending physician, are resource intensive, and are often completed well after the end of the procedure in question. These limitations lead to low faculty compliance and potential significant recall bias. In this study, we deployed a smartphone-based system, the Procedural Autonomy and Supervisions System, to facilitate assessment of resident performance according to the Zwisch scale with minimal workflow disruption. We aimed to demonstrate that this is a reliable, valid, and feasible method of measuring resident operative autonomy. METHODS: Before implementation, general surgery residents and faculty underwent frame-of-reference training to the Zwisch scale. Immediately after any operation in which a resident participated, the system automatically sent a text message prompting the attending physician to rate the resident's level of operative autonomy according to the 4-level Zwisch scale. Of these procedures, 8 were videotaped and independently rated by 2 additional surgeons. The Zwisch ratings of the 3 raters were compared using an intraclass correlation coefficient. Videotaped procedures were also scored using 2 alternative operating room (OR) performance assessment instruments (Operative Performance Rating System and Ottawa Surgical Competency OR Evaluation), against which the item correlations were calculated. RESULTS: Between December 2012 and June 2013, 27 faculty used the smartphone system to complete 1490 operative performance assessments on 31 residents. During this period, faculty completed evaluations for 92% of all operations performed with general surgery residents. The Zwisch scores were shown to correlate with postgraduate year (PGY) levels based on sequential pairwise chi-squared tests: PGY 1 vs PGY 2 (χ(2) = 106.9, df = 3, p < 0.001); PGY 2 vs PGY 3 (χ(2) = 22.2, df = 3, p < 0.001); and PGY 3 vs PGY 4 (χ(2) = 56.4, df = 3, p < 0.001). Comparison of PGY 4 to PGY 5 scores were not significantly different (χ(2) = 4.5, df = 3, p = 0.21). For the 8 operations reviewed for interrater reliability, the intraclass correlation coefficient was 0.90 (95% CI: 0.72-0.98, p < 0.01). Correlation of Procedural Autonomy and Supervisions System ratings with both Operative Performance Rating System items (each r > 0.90, all p's < 0.01) and Ottawa Surgical Competency OR Evaluation items (each r > 0.86, all p's < 0.01) was high. CONCLUSIONS: The Zwisch scale can be used to make reliable and valid measurements of faculty guidance and resident autonomy. Our data also suggest that Zwisch ratings may be used to infer resident operative performance. Deployed on an automated smartphone-based system, it can be used to feasibly record evaluations for most operations performed by residents. This information can be used to council individual residents, modify programmatic curricula, and potentially inform national training guidelines.

Defining the autonomy gap: when expectations do not meet reality in the operating room.

OBJECTIVE: To develop operative independence with essential procedures by the end of their training, residents need graded autonomy as they progress through training. This study compares autonomy expectations, as defined by faculty and residents, with autonomy measured in the operating room. METHODS: Operative procedures performed by general surgery residents between November 2012 and June 2013 were each assigned an autonomy score by the operating attending physician using a previously described rating scale (Zwisch). Scores range from minimum autonomy, "show and tell," to maximum autonomy, "supervision only." Autonomy expectations were defined by a survey asking faculty and residents what autonomy-level residents should achieve during each year of training for each of the 10 most commonly performed procedures. Faculty expectations, resident expectations, and actual operating room autonomy data were compared using analysis of variance with post hoc analysis by Tukey honestly significant difference test. RESULTS: A total of 1467 operative cases were scored using the Zwisch scale over the period of the study. The 10 most common procedures accounted for 56.3% (827) of the cases. Resident and faculty expectations of resident operative autonomy were similar. For only laparoscopic cholecystectomy, residents expected significantly more autonomy than the faculty did during the junior years but they agreed with the faculty for the chief year. When expectations were compared with actual performance, the resident autonomy level achieved was significantly less than that expected by residents or faculty or both for all 10 procedures in at least one postgraduate level. For every procedure performed more than 5 times during the study period by postgraduate years 3 to 5 residents, autonomy was significantly less than expected. CONCLUSIONS: Surgical faculty and residents had similar expectations for resident operative autonomy, yet actual resident performance failed to achieve those shared expectations for even the most common procedures. This autonomy gap provides more evidence for concerns about the preparedness of graduating residents for independent practice.

Evaluation of a blended learning surgical skills course for rural surgeons.

BACKGROUND: Rural surgeons have unique learning needs not easily met by traditional continuing medical education courses. METHODS: A multidisciplinary team developed and implemented a skills curriculum focused on leadership and communication, advanced endoscopy, emergency urology, emergency gynecology, facial plastic surgery, ultrasound, and management of fingertip amputations. RESULTS: Twenty-five of 30 (89%) rural surgeons who completed a follow-up course evaluation reported that the knowledge acquired during the course had improved their practice and/or the quality of patient care, particularly by refining commonly used skills and expanding the care options they could offer to their patients. The surgeons reported incorporating changes in their communication and interaction with colleagues. CONCLUSIONS: This course was successful, from participants' perspectives, in providing hands-on mentored training for a variety of skills that reflect the broad scope of practice of surgeons in rural areas. Attendees felt that their participation resulted in important behavior and practice changes.

Development and verification of a taxonomy of assessment metrics for surgical technical skills.

PURPOSE: To create and empirically verify a taxonomy of metrics for assessing surgical technical skills, and to determine which types of metrics, skills, settings, learners, models, and instruments were most commonly reported in the technical skills assessment literature. METHOD: In 2011-2012, the authors used a rational analysis of existing and emerging metrics to create the taxonomy, and used PubMed to conduct a systematic literature review (2001-2011) to test the taxonomy's comprehensiveness and verifiability. Using 202 articles identified from the review, the authors classified metrics according to the taxonomy and coded data concerning their context and use. Frequencies (counts, percentages) were calculated for all variables. RESULTS: The taxonomy contained 12 objective and 4 subjective categories. Of 567 metrics identified in the literature, 520 (92%) were classified using the new taxonomy. Process metrics outnumbered outcome metrics by 8:1. The most frequent metrics were "time," "manual techniques" (objective and subjective), "errors," and "procedural steps." Only one new metric, "learning curve," emerged. Assessments of basic motor skills and skills germane to laparoscopic surgery dominated the literature. Novices, beginners, and intermediate learners were the most frequent subjects, and box trainers and virtual reality simulators were the most frequent models used for assessing performance. CONCLUSIONS: Metrics convey what is valued in human performance. This taxonomy provides a common nomenclature. It may help educators and researchers in procedurally oriented disciplines to use metrics more precisely and consistently. Future assessments should focus more on bedside tasks and open surgical procedures and should include more outcome metrics.

Duration of faculty training needed to ensure reliable or performance ratings.

OBJECTIVES: The American Board of Surgery has mandated intraoperative assessment of general surgery residents, yet the time required to train faculty to accurately and reliably complete operating room performance evaluation forms is unknown. Outside of surgical education, frame-of-reference (FOR) training has been shown to be an effective training modality to teach raters the specific performance indicators associated with each point on a rating scale. Little is known, however, about what form and duration of FOR training is needed to accomplish reliable ratings among surgical faculty. DESIGN: Two groups of surgical faculty separately underwent either an accelerated 1-hour (n = 10) or immersive four-hour (n = 34) FOR faculty development program. Both programs included a formal presentation and a facilitated discussion of sample behaviors for each point on the Zwisch operating room performance rating scale (see DaRosa et al.(8)). The immersive group additionally participated in a small group exercise that included additional practice. After training, both groups were tested using 10 video clips of trainees at various levels. Responses were scored against expert consensus ratings. The 2-sided Mann-Whitney U test was used to compare between group means. SETTING AND PARTICIPANTS: All trainees were faculty members in the Department of Surgery of a large midwestern private medical school. RESULTS: Faculty undergoing the 1-hour FOR training program did not have a statistically different mean correct response rate on the video test when compared with those undergoing the 4-hour training program (88% vs 80%; p = 0.07). CONCLUSIONS: One-hour FOR training sessions are likely sufficient to train surgical faculty to reliably use a simple evaluation instrument for the assessment of intraoperative performance. Additional research is needed to determine how these results generalize to different assessment instruments.

Implementing an excellence in teaching recognition system: needs analysis and recommendations.

PURPOSE: Teaching awards have been suggested to serve a variety of purposes. The specific characteristics of teaching awards and the associated effectiveness at achieving planned purposes are poorly understood. A needs analysis was performed to inform recommendations for an Excellence in Teaching Recognition System to meet the needs of surgical education leadership. METHOD: We performed a 2-part needs analysis beginning with a review of the literature. We then, developed, piloted, and administered a survey instrument to General Surgery program leaders. The survey examined the features and perceived effectiveness of existing teaching awards systems. A multi-institution committee of program directors, clerkship directors, and Vice-Chairs of education then met to identify goals and develop recommendations for implementation of an "Excellence in Teaching Recognition System." RESULTS: There is limited evidence demonstrating effectiveness of existing teaching awards in medical education. Evidence supports the ability of such awards to demonstrate value placed on teaching, to inspire faculty to teach, and to contribute to promotion. Survey findings indicate that existing awards strive to achieve these purposes and that educational leaders believe awards have the potential to do this and more. Leaders are moderately satisfied with existing awards for providing recognition and demonstrating value placed on teaching, but they are less satisfied with awards for motivating faculty to participate in teaching or for contributing to promotion. Most departments and institutions honor only a few recipients annually. CONCLUSIONS: There is a paucity of literature addressing teaching recognition systems in medical education and little evidence to support the success of such systems in achieving their intended purposes. The ability of awards to affect outcomes such as participation in teaching and promotion may be limited by the small number of recipients for most existing awards. We propose goals for a Teaching Recognition System and provide guidelines for implementation and evaluation of such systems. Future analysis should study the effectiveness of systems designed using these guidelines in achieving the outlined goals.

Use of cognitive task analysis to guide the development of performance-based assessments for intraoperative decision making.

BACKGROUND: There is a paucity of performance-based assessments that focus on intraoperative decision making. The purpose of this article is to review the performance outcomes and usefulness of two performance-based assessments that were developed using cognitive task analysis (CTA) frameworks. METHODS: Assessment-A used CTA to create a "think aloud" oral examination that was administered while junior residents (PGY 1-2's, N = 69) performed a porcine-based laparoscopic cholecystectomy. Assessment-B used CTA to create a simulation-based, formative assessment of senior residents' (PGY 4-5's, N = 29) decision making during a laparoscopic ventral hernia repair. In addition to survey-based assessments of usefulness, a multiconstruct evaluation was performed using eight variables. RESULTS: When comparing performance outcomes, both approaches revealed major deficiencies in residents' intraoperative decision-making skills. Multiconstruct evaluation of the two CTA approaches revealed assessment method advantages for five of the eight evaluation areas: (1) Cognitive Complexity, (2) Content Quality, (3) Content Coverage, (4) Meaningfulness, and (5) Transfer and Generalizability. CONCLUSIONS: The two CTA performance assessments were useful in identifying significant training needs. While there are pros and cons to each approach, the results serve as a useful blueprint for program directors seeking to develop performance-based assessments for intraoperative decision making.

What surgical skills rural surgeons need to master.

BACKGROUND: As new technology is developed and scientific evidence demonstrates strategies to improve the quality of care, it is essential that surgeons keep current with their skills. Rural surgeons need efficient and targeted continuing medical education that matches their broader scope of practice. Developing such a program begins with an assessment of the learning needs of the rural surgeon. The aim of this study was to assess the learning needs considered most important to surgeons practicing in rural areas. STUDY DESIGN: A needs assessment questionnaire was administered to surgeons practicing in rural areas. An additional gap analysis questionnaire was administered to registrants of a skills course for rural surgeons. RESULTS: Seventy-one needs assessment questionnaires were completed. The self-reported procedures most commonly performed included laparoscopic cholecystectomy (n = 44), hernia repair (n = 42), endoscopy (n = 43), breast surgery (n = 23), appendectomy (n = 20), and colon resection (n = 18). Respondents indicated that they would most like to learn more skills related to laparoscopic colon resection (n = 16), laparoscopic antireflux procedures (n = 6), laparoscopic common bile duct exploration/ERCP (n = 5), colonoscopy/advanced techniques and esophagogastroscopy (n = 4), and breast surgery (n = 4). Ultrasound, hand surgery, and leadership and communication were additional topics rated as useful by the respondents. Skills course participants indicated varying levels of experience and confidence with breast ultrasound, ultrasound for central line insertion, hand injury, and facial soft tissue injury. CONCLUSIONS: Our results demonstrated that surgeons practicing in rural areas have a strong interest in acquiring additional skills in a variety of general and subspecialty surgical procedures. The information obtained in this study may be used to guide curriculum development of further postgraduate skills courses targeted to rural surgeons.

Development and participant assessment of a practical quality improvement educational initiative for surgical residents.

BACKGROUND: As patient-safety and quality efforts spread throughout health care, the need for physician involvement is critical, yet structured training programs during surgical residency are still uncommon. Our objective was to develop an extended quality-improvement curriculum for surgical residents that included formal didactics and structured practical experience. METHODS: Surgical trainees completed an 8-hour didactic program in quality-improvement methodology at the start of PGY3. Small teams developed practical quality-improvement projects based on needs identified during clinical experience. With the assistance of the hospital's process-improvement team and surgical faculty, residents worked through their selected projects during the following year. Residents were anonymously surveyed after their participation to assess the experience. RESULTS: During the first 3 years of the program, 17 residents participated, with 100% survey completion. Seven quality-improvement projects were developed, with 57% completing all DMAIC (Define, Measure, Analyze, Improve, Control) phases. Initial projects involved issues of clinical efficiency and later projects increasingly focused on clinical care questions. Residents found the experience educationally important (65%) and believed they were well equipped to lead similar initiatives in the future (70%). Based on feedback, the timeline was expanded from 12 to 24 months and changed to start in PGY2. CONCLUSIONS: Developing an extended curriculum using both didactic sessions and applied projects to teach residents the theory and implementation of quality improvement is possible and effective. It addresses the ACGME competencies of practice-based improvement and learning and systems-based practice. Our iterative experience during the past 3 years can serve as a guide for other programs.