The American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA): Validity Evidence From a Three-Year National Study.

OBJECTIVE: To gather validity evidence supporting the use and interpretation of scores from the American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA) Program. SUMMARY AND BACKGROUND DATA: ACS ERRA is an online formative assessment program developed to assess entering surgery residents' ability to make critical clinical decisions, and includes 12 clinical areas and 20 topics identified by a national panel of surgeon educators and residency program directors. METHODS: Data from 3 national testing administrations of ACS ERRA (2018-2020) were used to gather validity evidence regarding content, response process, internal structure (reliability), relations to other variables, and consequences. RESULTS: Over the 3 administrations, 1975 surgery residents participated from 125 distinct residency programs. Overall scores [Mean = 64% (SD = 7%)] remained consistent across the 3 years ( P = 0.670). There were no significant differences among resident characteristics (gender, age, international medical graduate status). The mean case discrimination index was 0.54 [SD = 0.15]. Kappa inter-rater reliability for scoring was 0.87; the overall test score reliability (G-coefficient) was 0.86 (Ф-coefficient = 0.83). Residents who completed residency readiness programs had higher ACS ERRA scores (66% versus 63%, Cohen's d = 0.23, P < 0.001). On average, 15% of decisions made (21/140 per test) involved potentially harmful actions. Variability in scores from graduating medical schools (7%) carried over twice as much weight than from matched residency programs (3%). CONCLUSIONS: ACS ERRA scores provide valuable information to entering surgery residents and surgery program directors to aid in development of individual and group learning plans.

The American College of Surgeons Entering Resident Readiness Assessment Program: Development and National Pilot Testing Results.

OBJECTIVE: To assess the readiness of entering residents for clinical responsibilities, the American College of Surgeons (ACS) Division of Education developed the "Entering Resident Readiness Assessment" (ACS-ERRA) Program. SUMMARY BACKGROUND: ACS-ERRA is an online formative assessment that uses a key features approach to measure clinical decision-making skills and focuses on cases encountered at the beginning of residency. Results can be used to develop learning plans to address areas that may need reinforcement. METHODS: A national panel of 16 content experts, 3 medical educators, and a psychometrician developed 98 short, key features cases. Each case required medical knowledge to be applied appropriately at challenging decision points during case management. Four pilot testing studies were conducted sequentially to gather validity evidence. RESULTS: Residents from programs across the United States participated in the studies (n = 58, 20, 87, 154, respectively). Results from the pilot studies enabled improvements after each pilot test. For the psychometric pilot (final pilot test), 2 parallel test forms of the ACS-ERRA were administered, each containing 40 cases, resulting in overall mean testing time of 2 hours 2 minutes (SD = 43 min). The mean test score was 61% (SD = 9%) and the G-coefficient reliability was 0.90. CONCLUSIONS: Results can be used to identify strengths and weaknesses in residents' decision-making skills and yield valuable information to create individualized learning plans. The data can also support efforts directed at the transition into residency training and inform discussions about levels of supervision. In addition, surgery program directors can use the aggregate test results to make curricular changes.

Piloting a National Curricular Resource for the Transition to Surgical Residency: Characteristics of Participating Schools and Their Students.

OBJECTIVE: To determine the extent to which U.S. Liaison Committee of Medical Education (LCME)-accredited medical schools chose to participate in piloting a national curricular resource, the American College of Surgeons/Association of Program Directors in Surgery/Association of Surgical Education Resident Prep Curriculum ("ACS-surgery-prep curriculum"), and implications of such participation for student access nationally to this resource. DESIGN: We examined the significance of school-level differences in ACS-surgery-prep curriculum pilot participation and student-level differences in curriculum access based on medical school attended in bivariate analysis. SETTING: U.S. medical schools choosing to participate in the ACS-surgery-prep curriculum through 2021. Students graduating from U.S. LCME-accredited medical schools in 2020-2021 were invited to complete the Association of American Medical Colleges 2021 Graduation Questionnaire (GQ). PARTICIPANTS: Our study included data for 2569 students intending surgery specialties (16% of 16,353 2021 GQ respondents) from ACS-surgery-prep curriculum pilot and non-pilot schools. RESULTS: Of 148 medical schools attended by 2021 GQ respondents, 93 (63%) were identified as ACS-surgery-prep curriculum pilot schools. Pilot participation varied by school region, community-based designation, and research intensity (each p < 0.05) but not by ownership or transition to residency (TTR) course requirements (each p > 0.05). Of 2569 GQ respondents nationally intending surgery specialties, 1697 (66%) attended an ACS-surgery-prep curriculum pilot school; this proportion did not vary by gender or race/ethnicity (each p > 0.05) but varied by students' school TTR course requirements (p < 0.001). Findings were similar among the 1059 students intending general surgery specialties specifically (41% of all 2569 students intending surgery specialties). CONCLUSIONS: Many U.S. LCME-accredited medical schools piloted this national TTR surgery curriculum. School-level characteristics associated with pilot participation can inform outreach efforts to encourage the participation of interested schools in piloting this TTR resource. With this curriculum distribution model, we observed no gender or racial/ethnic disparities in curriculum access nationally among students intending surgery specialties.

Resident Readiness for Senior Level Decision Making: Identifying the Domains for Formative Assessment and Feedback.

OBJECTIVE: To establish expert consensus regarding the domains and topics for senior surgery residents (PGY-4) to make critical decisions and assume senior-level responsibilities, and to develop the formative American College of Surgeons Senior Resident Readiness Assessment (ACS SRRA) Program. DESIGN: The American College of Surgeons (ACS) education leadership team conducted a focus group with surgical experts to identify the content for an assessment tool to evaluate senior residents' readiness for their increased levels of responsibility. After the focus group, national experts were recruited to develop consensus on the topics through three rounds of surveys using Delphi methodology. The Delphi participants rated topics using Likert-type scales and their comments were incorporated into subsequent rounds. Consensus was defined as ≥ 80% agreement with internal-consistency reliability (Cronbach's alpha) ≥ 0.8. In a stepwise fashion, topics that did not achieve consensus for inclusion were removed from subsequent survey rounds. SETTING: The surveys were administered via an online questionnaire. PARTICIPANTS: Twelve program directors and assistant program directors made up the focus group. The 39 Delphi participants represented seven different surgical subspecialties and were from diverse practice settings. The median length of experience in general surgery resident education was 20 years (IQR 14.3-30.0) with 64% of the experts being either current or past general surgery residency program directors. RESULTS: The response rate was 100% and Cronbach's alpha was ≥ 0.9 for each round. The Delphi participants contributed a large number of comments. Of the 201 topics that were evaluated initially, 120 topics in 25 core clinical areas were included to create the final domains of ACS SRRA. CONCLUSIONS: National consensus on the domain of the ACS SRRA has been achieved via the modified Delphi method among expert surgeon educators. ACS SRRA will identify clinical topics and areas in which each senior resident needs improvement and provide data to residents and residency programs to develop individualized learning plans. This would help in preparing the senior residents to assume their responsibilities and support their readiness for future fellowship training or surgical practice.

Impact of the COVID-19 pandemic on surgical trainee education and well-being spring 2020-winter 2020: A path forward.

BACKGROUND: The time course and longitudinal impact of the COVID -19 pandemic on surgical education(SE) and learner well-being (LWB)is unknown. MATERIAL AND METHODS: Check-in surveys were distributed to Surgery Program Directors and Department Chairs, including general surgery and surgical specialties, in the summer and winter of 2020 and compared to a survey from spring 2020. Statistical associations for items with self-reported ACGME Stage and the survey period were assessed using categorical analysis. RESULTS: Stage 3 institutions were reported in spring (30%), summer (4%) [p < 0.0001] and increased in the winter (18%). Severe disruption (SD) was stage dependent (Stage 3; 45% (83/184) vs. Stages 1 and 2; 26% (206/801)[p < 0.0001]). This lessened in the winter (23%) vs. spring (32%) p = 0.02. LWB severe disruption was similar in spring 27%, summer 22%, winter 25% and was associated with Stage 3. CONCLUSIONS: Steps taken during the pandemic reduced SD but did not improve LWB. Systemic efforts are needed to protect learners and combat isolation pervasive in a pandemic.

Surgery Clerkship Directors' Perceptions of the COVID-19 Pandemic's Impact on Medical Student Education.

BACKGROUND: This study assessed the national impact of the COVID-19 pandemic on the education of medical students assigned to surgery clerkship rotations, as reported by surgery clerkship directors(CDs). STUDY DESIGN: In the spring of 2020 and 2021, the authors surveyed 164 CDs from 144 LCME-accredited US medical schools regarding their views of the pandemic's impact on the surgery clerkship curriculum, students' experiences, outcomes, and institutional responses. RESULTS: Overall survey response rates, calculated as no. respondents/no. surveyed were 44.5%(73/164) and 50.6%(83/164) for the spring 2020 and 2021 surveys, respectively. Nearly all CDs(>95%) pivoted to virtual platforms and solutions. Most returned to some form of in-person learning by winter 2020, and pre-pandemic status by spring 2021(46%, 38/83). Students' progression to the next year was delayed by 12%(9/73), and preparation was negatively impacted by 45%(37/83). Despite these data, CDs perceived students' interest in surgical careers was not significantly affected(89% vs. 77.0%, p=0.09). Over the one-year study, the proportion of CDs reporting a severe negative impact on the curriculum dropped significantly(p<0.0001) for most parameters assessed except summative evaluations(40.3% vs. 45.7%,p=0.53). CDs(n=83) also noted the pandemic's positive impact with respect to virtual patient encounters(21.7%), didactics(16.9%), student test performance(16.9%), continuous personal learning(14.5%), engagement in the clerkship(9.6%) and student interest in surgery as a career(7.2%). CONCLUSION: During the pandemic, the severe negative impact on student educational programs lessened and novel virtual curricular solutions emerged. Student interest in surgery as a career was sustained. Measures of student competency and effectiveness of new curriculum, including telehealth, remain areas for future investigation.

Impact of the COVID-19 Pandemic on Surgical Training and Learner Well-Being: Report of a Survey of General Surgery and Other Surgical Specialty Educators.

BACKGROUND: The COVID-19 pandemic disrupted the delivery of surgical services. The purpose of this communication was to report the impact of the pandemic on surgical training and learner well-being and to document adaptations made by surgery departments. STUDY DESIGN: A 37-item survey was distributed to educational leaders in general surgery and other surgical specialty training programs. It included both closed- and open-ended questions and the self-reported stages of GME during the COVID-19 pandemic, as defined by the ACGME. Statistical associations for items with stage were assessed using categorical analysis. RESULTS: The response rate was 21% (472 of 2,196). US stage distribution (n = 447) was as follows: stage 1, 22%; stage 2, 48%; and stage 3, 30%. Impact on clinical education significantly increased by stage, with severe reductions in nonemergency operations (73% and 86% vs 98%) and emergency operations (8% and 16% vs 34%). Variable effects were reported on minimal expected case numbers across all stages. Reductions were reported in outpatient experience (83%), in-hospital experience (70%), and outside rotations (57%). Increases in ICU rotations were reported with advancing stage (7% and 13% vs 37%). Severity of impact on didactic education increased with stage (14% and 30% vs 46%). Virtual conferences were adopted by 97% across all stages. Severity of impact on learner well-being increased by stage-physical safety (6% and 9% vs 31%), physical health (0% and 7% vs 17%), and emotional health (11% and 24% vs 42%). Regardless of stage, most but not all made adaptations to support trainees' well-being. CONCLUSIONS: The pandemic adversely impacted surgical training and the well-being of learners across all surgical specialties proportional to increasing ACGME stage. There is a need to develop education disaster plans to support technical competency and learner well-being. Careful assessment for program advancement will also be necessary. The experience during this pandemic shows that virtual learning and telemedicine will have a considerable impact on the future of surgical education.

Needs Assessment for an American College of Surgeons Certificate in Applied Surgical Education Leadership (CASEL).

OBJECTIVE: Surgeon educators in departments of surgery play key roles in leading and advancing surgical education. Their activities include ensuring sound curricula and evaluation systems, monitoring education resources, overseeing faculty development, and providing mentorship. For more than 25 years, the American College of Surgeons (ACS) has offered a comprehensive "Surgeons as Educators" (SAE) course to address fundamental topics in surgical education. This study aims to identify future career needs of SAE graduates to inform the development of an American College of Surgeons Certificate in Applied Surgical Education Leadership program. DESIGN: An IRB exempt, anonymous electronic survey was developed to determine educational roles, career aspirations, and needs of SAE Graduates. SETTING AND PARTICIPANTS: Participants included all 763 1993-2016 SAE graduates. RESULTS: One hundred and thirty-five responses were received from 600 (22.5%) graduates with valid email addresses. Sixty (45%) respondents completed the SAE Course > 5 years prior to the study (M5YRS) and 75 (55%) within the last 5 years (L5YRS). L5YRS respondents were less likely to be full professors (8% vs. 44%) or to serve as program directors (32% vs. 57%), and more likely to be associate program directors (25% vs. 17%) or clerkship directors (40% vs. 18%). High percentages of both L5YRS and M5YRS reported not pursuing additional educational opportunities post-SAE due to time and fiscal constraints. One-fifth of respondents were unaware of additional opportunities and 19% of M5YRS versus 6% of L5YRS stated that existing programs did not meet their needs. Overall improving skills as educational leaders, developing faculty development programs, and conducting educational research were noted as priorities for future development. Differences were observed between the L5YRS and M5YRS groups. The dominant preferences for course format were full-time face-to-face (41%) or a combination of full-time face-to-face with online modules (24%). The most important considerations in deciding to pursue a certificate course were course content, and interest in advancing career and time constraints. CONCLUSIONS: An SAE graduate survey has confirmed the need for additional formal training in surgical education leadership in order to permit surgeon educators meet the demands of the changing landscape of surgical education. The needs of early career faculty may differ from those of more senior surgeon educators.

Patient safety curriculum for surgical residency programs: results of a national consensus conference.

BACKGROUND: The American College of Surgeons (ACS) and the Accreditation Council for Graduate Medical Education (ACGME) are committed to promoting patient safety through education. In view of the critical role of residents in the delivery of safe patient care, the ACS and ACGME sponsored jointly a national consensus conference to initiate the development of a curriculum on patient safety that may be used across all surgical residency programs. CONCLUSIONS: National leaders in surgery with expertise in surgical care and surgical education, patient safety experts, medical educators, key stakeholders from national organizations, and surgical residents were invited to participate in the conference. Attendees considered patient safety issues within the context of the 6 core competencies defined by the ACGME and American Board of Medical Specialties (ABMS). Discussions resulted in the development of a curriculum matrix that includes listings of patient safety topics, teaching and learning strategies, and assessment methods. Guidelines for implementation and dissemination are also provided. The curriculum content underscores the need to create an organizational culture of safety and focuses on both individuals and systems. Individual residency programs may prioritize the curriculum content based on their specific needs. The ACS and ACGME will pursue development of educational modules to address the curriculum content, disseminate helpful information, and assist in implementation of new educational interventions. This effort has the potential to positively impact residency education in surgery, help surgical program directors address the core competencies, and enhance patient safety.

National efforts to reform residency education in surgery.

Major changes in surgical practice and myriad external mandates have affected residency education in surgery. The traditional surgery residency education and training model has come under scrutiny, and calls for major reform of this model have been made by a variety of stakeholders. The American Surgical Association appointed a Blue Ribbon Committee in 2002 to consider the recent changes in surgical practice and surgical education and propose solutions that would ensure a well-educated and well-trained surgical workforce for the future. This committee included representatives from the American Surgical Association, the American College of Surgeons, the American Board of Surgery, and the Residency Review Committee for Surgery. The committee made several far-reaching recommendations relating to residency education in surgery. After the Blue Ribbon Committee completed its task in 2004, representatives from the aforementioned four organizations, the Association of Program Directors in Surgery, and the Association for Surgical Education created a national consortium called the Surgical Council on Resident Education (SCORE). This consortium is pursuing efforts to reform residency education in surgery and implement several key recommendations of the Blue Ribbon Committee. The principal area of focus of SCORE is the development of a national curriculum for surgery residency education and training. Other activities of SCORE include the development of a Web site to support surgery residency education and pursuit of international collaboration. SCORE's efforts will be key to offering surgery residents the best educational experiences, preparing residents for future practice, and supporting delivery of surgical care of the highest quality. The authors examine the current state of residency education in surgery and explore efforts underway to reform this educational model.

"Taking Training to the Next Level": The American College of Surgeons Committee on Residency Training Survey.

OBJECTIVE: The American College of Surgeons (ACS) appointed a committee of leaders from the ACS, Association of Program Directors in Surgery, Accreditation Council for Graduate Medical Education, and American Board of Surgery to define key challenges facing surgery resident training programs and to explore solutions. The committee wanted to solicit the perspectives of surgery resident program directors (PDs) given their pivotal role in residency training. DESIGN: Two surveys were developed, pilot tested, and administered to PDs following Institutional Review Board approval. PDs from 247 Accreditation Council for Graduate Medical Education-accredited general surgery programs were randomized to receive 1 of the 2 surveys. Bias analyses were conducted, and adjusted Pearson χ2 tests were used to test for differences in response patterns by program type and size. SETTING: All accredited general surgery programs in the United States were included in the sampling frame of the survey; 10 programs with initial or withdrawn accreditation were excluded from the sampling frame. PARTICIPANTS: A total of 135 PDs responded, resulting in a 54.7% response rate (Survey A: n = 67 and Survey B: n = 68). The respondent sample was determined to be representative of program type and size. RESULTS: Nearly 52% of PD responses were from university-based programs, and 41% had over 6 residents per graduating cohort. More than 61% of PDs reported that, compared to 10 years ago, both entering and graduating residents are less prepared in technical skills. PDs expressed significant concerns regarding the effect of duty-hour restrictions on the overall preparation of graduating residents (61%) and quality of patient care (57%). The current 5-year training structure was viewed as needing a significant or extensive increase in opportunities for resident autonomy (63%), and the greatest barriers to resident autonomy were viewed to be patient preferences not to be cared for by residents (68%), liability concerns (68%), and Centers for Medicare and Medicaid Services regulations (65%). Although 64% of PDs believe that moderate or significant changes are needed in the current structure of residency training, 35% believe that no changes in the structure are needed. When asked for their 1 best recommendation regarding the structure of surgical residency, only 22% of PDs selected retaining the current 5-year structure. The greatest percentage of PDs (28%) selected the "4 + 2" model as their 1 best recommendation for the structure to be used. In the area of faculty development, 56% of PDs supported a significant or extensive increase in Train the Teacher programs, and 41% supported a significant or extensive increase in faculty certification in education. CONCLUSIONS: Information regarding the valuable perspectives of PDs gathered through these surveys should help in implementing important changes in residency training and faculty development. These efforts will need to be pursued collaboratively with involvement of key stakeholders, including the organizations represented on this ACS committee.

Education and Training to Address Specific Needs During the Career Progression of Surgeons.

Surgeons have specific education and training needs as they enter practice, progress through the core period of active practice, and then as they wind down their clinical work before retirement. These transitions and the career progression process, combined with the dynamic health care environment, present specific opportunities for innovative education and training based on practice-based learning and improvement, and continuous professional development methods. Cutting-edge technologies, blended models, simulation, mentoring, preceptoring, and integrated approaches can play critical roles in supporting surgeons as they provide the best surgical care throughout various phases of their careers.

Management of adverse surgical events: a structured education module for residents.

BACKGROUND: This pilot project involved the development of a structured, experiential, educational module using a bench model technical skills simulation and standardized patients. It integrated teaching and assessment of clinical, technical, and interpersonal skills, as well as professionalism within the context of an adverse surgical event. METHODS: General surgery residents (postgraduate year [PGY] 2, 3) were asked to participate in the pre-, intra-, and postoperative management of a patient with a retroperitoneal sarcoma. Residents' performances during the module were assessed by standardized patients and faculty, and residents were provided feedback during debriefing sessions. RESULTS: Resident performance during the module was appropriate for the level of training. Residents found this module to be a realistic, challenging, and beneficial learning experience. CONCLUSIONS: Novel educational modules such as this one may serve as a useful addition to resident education in surgery residency programs, particularly in addressing patient safety and the core competencies. Reliability of the model may be enhanced by modifications of the module.

Transition to surgical residency: a multi-institutional study of perceived intern preparedness and the effect of a formal residency preparatory course in the fourth year of medical school.

PURPOSE: To evaluate interns' perceived preparedness for defined surgical residency responsibilities and to determine whether fourth-year medical school (M4) preparatory courses ("bootcamps") facilitate transition to internship. METHOD: The authors conducted a multi-institutional, mixed-methods study (June 2009) evaluating interns from 11 U.S. and Canadian surgery residency programs. Interns completed structured surveys and answered open-ended reflective questions about their preparedness for their surgery internship. Analyses include t tests comparing ratings of interns who had and had not participated in formal internship preparation programs. The authors calculated Cohen d for effect size and used grounded theory to identify themes in the interns' reflections. RESULTS: Of 221 eligible interns, 158 (71.5%) participated. Interns self-reported only moderate preparation for most defined care responsibilities in the medical knowledge and patient care domains but, overall, felt well prepared in the professionalism, interpersonal communication, practice-based learning, and systems-based practice domains. Interns who participated in M4 preparatory curricula had higher self-assessed ratings of surgical technical skills, professionalism, interpersonal communication skills, and overall preparation, at statistically significant levels (P < .05) with medium effect sizes. Themes identified in interns' characterizations of their greatest internship challenges included anxiety or lack of preparation related to performance of technical skills or procedures, managing simultaneous demands, being first responders for critically ill patients, clinical management of predictable postoperative conditions, and difficult communications. CONCLUSIONS: Entering surgical residency, interns report not feeling prepared to fulfill common clinical and professional responsibilities. As M4 curricula may enhance preparation, programs facilitating transition to residency should be developed and evaluated.

Interventions to address challenges associated with the transition from residency training to independent surgical practice.

BACKGROUND: Concerns regarding preparation of residents for independent surgical practice are widespread and support for junior surgeons entering practice is variable across institutions and practices. The American College of Surgeons (ACS) Division of Education partnered with the Accreditation Council for Graduate Medical Education (ACGME) to convene a National Invitational Conference to define key issues relating to the transition to practice and develop recommendations to address various challenges. OUTCOMES OF THE NATIONAL INVITATIONAL CONFERENCE: Leaders from ACS, ACGME, certifying boards, residency review committees, program director organizations, and professional societies representing the breadth of surgical specialties, along with other key stakeholders, were invited to participate in the 1.5-day conference in July 2012. Key recommendations generated during the conference included the need to focus on the transition to practice within the context of the continuum of professional development; definition of specific levels of knowledge and skills expected of graduating surgery residents; development and adoption of competency-based methods for training, assessment, and advancement of residents; implementation of special interventions during the chief resident year to prepare residents for practice; robust evaluations of residents before graduation; intake assessments of junior surgeons during the onboarding processes; and effective mentorship for junior surgeons as they enter practice. Recommendations also highlighted major regulatory, legal, and financial issues. The key role of ACS and other national organizations in implementing the recommendations was underscored. CONCLUSION: The recommendations from the conference should be of great help in addressing various challenges associated with the transition from surgery residency to independent practice.

Reactions of surgical program directors to a web-based interactive educational program focusing on cognitive skills.

OBJECTIVE: To assess reactions by program directors (PDs) to a preview of a scenario from the Fundamentals of Surgery Curriculum (FSC), which is a case-based interactive curriculum developed by the American College of Surgeons's (ACS) Division of Education and designed to be delivered online to first-year (PGY-1) surgical residents. DESIGN: After previewing a scenario, each PD completed a questionnaire requesting age and ratings of comfort using computers, the scenario's utility in addressing 9 educational goals (eg, provides a solid foundation for future learning), and 6 separate features of the scenario (eg, ease of use and feasibility). All ratings were based on a 1-9 scale. For items related to educational goals, ratings were anchored: 1-3 = poor/needs revision; 4-6 = adequate/as good as current methods; 7-9 = excellent/superior to current methods. Informal discussions were also conducted and comments were collected. SETTING: October 2007 ACS Clinical Congress. PARTICIPANTS: In all, 31 PDs participated in the study. RESULTS: Most PDs perceived that the scenario addressed 8 of the 9 educational goals in a manner superior to current methods [eg, provides a solid foundation for future learning (97%), challenges residents (90%), and delivers content consistent with current practices and/or evidence (90%)]. The mean ratings of all scenario features were 7 or greater on the 9-point scale. CONCLUSION: Most PDs reacted very positively to a preview of FSC perceiving that it can address several important educational goals in a manner superior to existing methods. Comments from PDs suggest a high level of interest in incorporating FSC into their residency programs as well as participating in a coordinated multi-institutional evaluation project. The results provide baseline data concerning PD expectations of the utility of FSC that will help to guide and evaluate further developments and applications of this curriculum.

Comparison of resident self-assessments with trained faculty and standardized patient assessments of clinical and technical skills in a structured educational module.

BACKGROUND: This study assessed the reliability of surgical resident self-assessment in comparison with faculty and standardized patient (SP) assessments during a structured educational module focused on perioperative management of a simulated adverse event. METHODS: Seven general surgery residents participated in this module. Residents were assessed during videotaped preoperative and postoperative SP encounters and when dissecting a tumor off of a standardized inanimate vena cava model in a simulated operating room. RESULTS: Preoperative and postoperative assessments by SPs correlated significantly (P < .05) with faculty assessments (r = .75 and r = .79, respectively), but not resident self-assessments. Coefficient alpha was greater than .70 for all assessments except resident preoperative self-assessments. CONCLUSIONS: Faculty and SP assessments can provide reliable data useful for formative feedback. Although resident self-assessment may be useful for the formative assessment of technical skills, results suggest that in the absence of training, residents are not reliable self-assessors of preoperative and postoperative interactions with SPs.