Implementation of a Needs-Based, Online Feedback Tool for Anesthesia Residents With Subsequent Mapping of the Feedback to the ACGME Milestones.

BACKGROUND: Optimizing feedback that residents receive from faculty is important for learning. The goals of this study were to (1) conduct focus groups of anesthesia residents to define what constitutes optimal feedback; (2) develop, test, and implement a web-based feedback tool; and (3) then map the contents of the written comments collected on the feedback tool to the Accreditation Council for Graduate Medical Education (ACGME) anesthesiology milestones. METHODS: All 72 anesthesia residents in the program were invited to participate in 1 of 5 focus groups scheduled over a 2-month period. Thirty-seven (51%) participated in the focus groups and completed a written survey on previous feedback experiences. On the basis of the focus group input, an initial online feedback tool was pilot-tested with 20 residents and 62 feedback sessions, and then a final feedback tool was deployed to the entire residency to facilitate the feedback process. The completed feedback written entries were mapped onto the 25 ACGME anesthesiology milestones. RESULTS: Focus groups revealed 3 major barriers to good feedback: (1) too late such as, for example, at the end of month-long clinical rotations, which was not useful because the feedback was delayed; (2) too general and not specific enough to immediately remedy behavior; and (3) too many in that the large number of evaluations that existed that were unhelpful such as those with unclear behavioral anchors compromised the overall feedback culture. Thirty residents (42% of 72 residents in the program) used the final online feedback tool with 121 feedback sessions with 61 attendings on 15 rotations at 3 hospital sites. The number of feedback tool uses per resident averaged 4.03 (standard deviation 5.08, median 2, range 1-21, 25th-75th % quartile 1-4). Feedback tool uses per faculty averaged 1.98 (standard deviation 3.2, median 1, range 1-25, 25th-75th % quartile 1-2). For the feedback question item "specific learning objective demonstrated well by the resident," this yielded 296 milestone-specific responses. The majority (71.3%) were related to the patient care competency, most commonly the anesthetic plan and conduct (35.8%) and airway management (11.1%) milestones; 10.5% were related to the interpersonal and communication skills competency, most commonly the milestones communication with other professionals (4.4%) or with patients and families (4.4%); and 8.4% were related to the practice-based learning and improvement competency, most commonly self-directed learning (6.1%). For the feedback tool item "specific learning objective that resident may improve," 67.0% were related to patient care, most commonly anesthetic plan and conduct (33.5%) followed by use/interpretation of monitoring and equipment (8.5%) and airway management (8.5%); 10.2% were related to practice-based learning and improvement, most commonly self-directed learning (6.8%); and 9.7% were related to the systems-based practice competency. CONCLUSIONS: Resident focus groups recommended that feedback be timely and specific and be structured around a tool. A customized online feedback tool was developed and implemented. Mapping of the free-text feedback comments may assist in assessing milestones. Use of the feedback tool was lower than expected, which may indicate that it is just 1 of many implementation steps required for behavioral and culture change to support a learning environment with frequent and useful feedback.

Impact of an Innovative Classroom-Based Lecture Series on Residents' Evaluations of an Anesthesiology Rotation.

Introduction. Millennial resident learners may benefit from innovative instructional methods. The goal of this study is to assess the impact of a new daily, 15 minutes on one anesthesia keyword, lecture series given by faculty member each weekday on resident postrotation evaluation scores. Methods. A quasi-experimental study design was implemented with the residents' rotation evaluations for the 24-month period ending by 7/30/2013 before the new lecture series was implemented which was compared to the 14-month period after the lecture series began on 8/1/2013. The primary endpoint was "overall teaching quality of this rotation." We also collected survey data from residents at clinical rotations at two other different institutions during the same two evaluation periods that did not have the education intervention. Results. One hundred and thirty-one residents were eligible to participate in the study. Completed surveys ranged from 77 to 87% for the eight-question evaluation instrument. On a 5-point Likert-type scale the mean score on "overall teaching quality of this rotation" increased significantly from 3.9 (SD 0.8) to 4.2 (SD 0.7) after addition of the lecture series, whereas the scores decreased slightly at the comparison sites. Conclusion. Rotation evaluation scores for overall teaching quality improved with implementation of a new structured slide daily lectures series.