American society for metabolic and bariatric surgery: intra-operative care pathway for minimally invasive Roux-en-Y gastric bypass.

BACKGROUND: Clinical care pathways help guide and provide structure to clinicians and providers to improve healthcare delivery and quality. The Quality Improvement and Patient Safety Committee (QIPS) of the American Society for Metabolic and Bariatric Surgery (ASMBS) has previously published care pathways for the performance of laparoscopic sleeve gastrectomy (LSG) and pre-operative care of patients undergoing Roux-en-Y gastric bypass (RYGB). OBJECTIVE: This current RYGB care pathway was created to address intraoperative care, defined as care occurring on the day of surgery from the preoperative holding area, through the operating room, and into the postanesthesia care unit (PACU). METHODS: PubMed queries were performed from January 2001 to December 2019 and reviewed according to Level of Evidence regarding specific key questions developed by the committee. RESULTS: Evidence-based recommendations are made for care of patients undergoing RYGB including the pre-operative holding area, intra-operative management and performance of RYGB, and concurrent procedures. CONCLUSIONS: This document may provide guidance based on recent evidence to bariatric surgeons and providers for the intra-operative care for minimally invasive RYGB.

Validation of a Virtual Simulator With Haptic Feedback for Endotracheal Intubation Training.

INTRODUCTION: Endotracheal intubation (ETI) is a procedure that varies in difficulty because of patient characteristics and clinical conditions. Existing physical simulators do not encompass these variations. The Virtual Airway Skills Trainer for Endotracheal Intubation (VAST-ETI) was developed to provide different patient characteristics and high-fidelity haptic feedback to improve training. METHODS: We demonstrate the effectiveness of VAST-ETI as a training and evaluation tool for ETI. Construct validation was evaluated by scoring the performance of experts ( N = 15) and novices ( N = 15) on the simulator to ensure its ability to distinguish technical proficiency. Convergent and predictive validity were evaluated by performing a learning curve study, in which a group of novices ( N = 7) were trained for 2 weeks using VAST-ETI and then compared with a control group ( N = 9). RESULTS: The VAST-ETI was able to distinguish between expert and novice based on mean simulator scores ( t [88] = -6.61, P < 0.0005). When used during repeated practice, individuals demonstrated a significant increase in their score on VAST-ETI over the learning period ( F [11,220] = 7206, P < 0.001); however when compared with a control group, there was not a significant interaction effect on the simulator score. There was a significant difference between the simulator-trained and control groups ( t [12.85] = -2.258, P = 0.042) when tested in the operating room. CONCLUSIONS: Our results demonstrate the effectiveness of virtual simulation with haptic feedback for assessing performance and training of ETI. The simulator was not able to differentiate performance between more experienced trainees and experts because of limits in simulator difficulty.

Gastrointestinal Complications After Transesophageal Echocardiography for Mitral Valve Transcatheter Edge-to-Edge Repair: Insights From a Large Contemporary Cohort.

OBJECTIVES: Transesophageal echocardiography-related complications (TEE-RC) are higher in structural heart interventions than in traditional operative settings. In mitral valve transcatheter edge-to-edge repair (MV-TEER), the incidence of TEE-RC may be higher than in other structural interventions. However, existing reports are limited and robust data evaluating TEE safety in this patient population are lacking. The authors sought to describe the incidence and risk factors of upper gastrointestinal injuries after TEE in patients undergoing MV-TEER. DESIGN: A retrospective observational study. SETTING: A single tertiary academic hospital. PARTICIPANTS: A total of 442 consecutive patients who underwent MV-TEER, specifically with MitraClip, between December 2015 and March 2022. INTERVENTIONS: Transesophageal echocardiography was performed intraoperatively to guide all MV-TEERs. MEASUREMENTS AND MAIN RESULTS: The study's primary goal was to investigate an association between TEE procedure duration and TEE-RC risk. The contribution of demographic risk factors and intraprocedural characteristics also was investigated. Transesophageal echocardiography-RCs were observed in 17 out of 442 patients (3.8%). Dysphagia was the most common TEE-RC (n = 9/17, 53%), followed by new gastroesophageal reflux (n = 6/17, 35%) and odynophagia (n = 3/17, 18%). There were no esophageal perforations or upper gastrointestinal bleeds. History of dysphagia was the only variable associated with TEE-RCs (p = 0.008; n = 9 [2.1%] v n = 3 [18%]), with a relative risk of 8.67 (95% CI 2.57, 29.16). The TEE procedure duration was not statistically different between the 2 groups (46 minutes [39-64] in TEE-RCs v 49 minutes [36-77] in no complications). CONCLUSION: In patients undergoing MV-TEER, TEE-RCs are uncommon, and major complications are rare. The authors' outcomes reflect those of a high-volume referral center with TEEs performed by cardiac anesthesiologists.

Preclinical Proficiency-Based Model of Ultrasound Training.

BACKGROUND: Graduate medical education is being transformed from a time-based training model to a competency-based training model. While the application of ultrasound in the perioperative arena has become an expected skill set for anesthesiologists, clinical exposure during training is intermittent and nongraduated without a structured program. We developed a formal structured perioperative ultrasound program to efficiently train first-year clinical anesthesia (CA-1) residents and evaluated its effectiveness quantitatively in the form of a proficiency index. METHODS: In this prospective study, a multimodal perioperative ultrasound training program spread over 3 months was designed by experts at an accredited anesthesiology residency program to train the CA-1 residents. The training model was based on self-learning through web-based modules and instructor-based learning by performing perioperative ultrasound techniques on simulators and live models. The effectiveness of the program was evaluated by comparing the CA-1 residents who completed the training to graduating third-year clinical anesthesia (CA-3) residents who underwent the traditional ultrasound training in the residency program using a designed index called a "proficiency index." The proficiency index was composed of scores on a cognitive knowledge test (20%) and scores on an objective structured clinical examination (OSCE) to evaluate the workflow understanding (40%) and psychomotor skills (40%). RESULTS: Sixteen CA-1 residents successfully completed the perioperative ultrasound training program and the subsequent evaluation with the proficiency index. The total duration of training was 60 hours of self-based learning and instructor-based learning. There was a significant improvement observed in the cognitive knowledge test scores for the CA-1 residents after the training program (pretest: 71% [0.141 ± 0.019]; posttest: 83% [0.165 ± 0.041]; P < .001). At the end of the program, the CA-1 residents achieved an average proficiency index that was not significantly different from the average proficiency index of graduating CA-3 residents who underwent traditional ultrasound training (CA-1: 0.803 ± 0.049; CA-3: 0.823 ± 0.063, P = .307). CONCLUSIONS: Our results suggest that the implementation of a formal, structured curriculum allows CA-1 residents to achieve a level of proficiency in perioperative ultrasound applications before clinical exposure.

Does your team know how to respond safely to an operating room fire? Outcomes of a virtual reality, AI-enhanced simulation training.

BACKGROUND: Operating room (OR) fires are rare but devastating events requiring immediate and effective response. Virtual Reality (VR) simulation training can provide a safe environment for practice of skills in such highly stressful situation. This study assessed interprofessional participants' ability to respond to VR-simulated OR fire scenarios, attitudes, numbers of attempt of the VR simulation do participants need to successfully respond to OR fires and does prior experience, confidence level, or professional role predict the number of attempts needed to demonstrate safety and pass the simulation. METHODS: 180 surgical team members volunteered to participate in this study at Beth Israel Deaconess Medical Center, Boston, MA. Each participant completed five VR OR simulation trials; the final two trials incorporated AI assistance. Primary outcomes were performance scores, number of attempts needed to pass, and pre- and post-survey results describing participant confidence and experiences. Differences across professional or training role were assessed using chi-square tests and analyses of variance. Differences in pass rates over time were assessed using repeated measures logistic regression. RESULTS: One hundred eighty participants completed simulation testing; 170 (94.4%) completed surveys. Participants included surgeons (17.2%), anesthesiologists (10.0%), allied health professionals (41.7%), and medical trainees (31.1%). Prior to training, 45.4% of participants reported feeling moderately or very confident in their ability to respond to an OR fire. Eight participants (4.4%) responded safely on the first simulation attempt. Forty-three participants (23.9%) passed by the third attempt (VR only); an additional 97 participants (53.9%) passed within the 4-5th attempt (VR with AI assistance). CONCLUSIONS: Providers are unprepared to respond to OR fires. VR-based simulation training provides a practical platform for individuals to improve their knowledge and performance in the management of OR fires with a 79% pass rate in our study. A VR AI approach to teaching this essential skill is innovative, feasible, and effective.

Operating room team training using simulation: Hope or hype?

BACKGROUND: This study sought to determine the long-term impact of multidisciplinary simulated operating room (OR) team training. METHOD: Two-wave survey study (immediate post-training survey 2010-2017, follow-up 2018). Differences across time, specialty, and experience with adverse events were assessed using chi-square and t -tests. RESULTS: Immediately after training, more than 90% of respondents found simulation scenarios realistic and reported team training would provide safer patient care. However, follow-up participants reported less enthusiasm toward training, with 58% stating they would like to take similar training again. A majority of participants (77%) experienced adverse events after training; those reporting adverse events reported more positive long-term evaluations. CONCLUSIONS: Simulated OR team training is initially highly valued by participants and is perceived as contributing to patient safety. Diminution of participant enthusiasm over time suggests that repeat training requirements be reconsidered, and less costly, alternative methods (such as asynchronous learning or virtual reality) should be explored.

Teaching Medical Students to Create Anesthetic Plans Using a Branched-Chain Learning Module.

It is challenging to include medical students in the anesthetic decision-making process during their introductory clinical rotation in anesthesiology. Furthermore, the rotation is often too brief to even expose students to the variety inherent in anesthesia practice. We created a web-based, branched-chain learning module (BCLM) for medical students with alternate narratives and patient outcomes based on their anesthetic choices. In semistructured group interviews, students described how the BCLM met its educational objectives and gave constructive feedback on several ways to improve their experience with it, guiding the future use of BCLMs in medical education.

Using Machine Learning to Evaluate Attending Feedback on Resident Performance.

BACKGROUND: High-quality and high-utility feedback allows for the development of improvement plans for trainees. The current manual assessment of the quality of this feedback is time consuming and subjective. We propose the use of machine learning to rapidly distinguish the quality of attending feedback on resident performance. METHODS: Using a preexisting databank of 1925 manually reviewed feedback comments from 4 anesthesiology residency programs, we trained machine learning models to predict whether comments contained 6 predefined feedback traits (actionable, behavior focused, detailed, negative feedback, professionalism/communication, and specific) and predict the utility score of the comment on a scale of 1-5. Comments with ≥4 feedback traits were classified as high-quality and comments with ≥4 utility scores were classified as high-utility; otherwise comments were considered low-quality or low-utility, respectively. We used RapidMiner Studio (RapidMiner, Inc, Boston, MA), a data science platform, to train, validate, and score performance of models. RESULTS: Models for predicting the presence of feedback traits had accuracies of 74.4%-82.2%. Predictions on utility category were 82.1% accurate, with 89.2% sensitivity, and 89.8% class precision for low-utility predictions. Predictions on quality category were 78.5% accurate, with 86.1% sensitivity, and 85.0% class precision for low-quality predictions. Fifteen to 20 hours were spent by a research assistant with no prior experience in machine learning to become familiar with software, create models, and review performance on predictions made. The program read data, applied models, and generated predictions within minutes. In contrast, a recent manual feedback scoring effort by an author took 15 hours to manually collate and score 200 comments during the course of 2 weeks. CONCLUSIONS: Harnessing the potential of machine learning allows for rapid assessment of attending feedback on resident performance. Using predictive models to rapidly screen for low-quality and low-utility feedback can aid programs in improving feedback provision, both globally and by individual faculty.

Development of an Endotracheal Intubation Formative Assessment Tool.

BACKGROUND: Valid methods for providing detailed formative feedback on direct laryngoscopy and endotracheal intubation (ETI) performance do not exist. We are developing an observation-based assessment tool for measuring performance and providing feedback during ETI. METHODS: Based on the literature and interviews of experts, we proposed an initial ETI metric with 22 items. Six anesthesiology experts used it to assess the quality of ETI performance in videotaped intubations. Following metric revisions, 2 expert groups assessed 2 collections of videos (27 total) using the revised metric. Two reference standards for comparison with metric scores were created with a third and fourth group of experts; (1) an average global rating (1-100) of each ETI performance and (2) average rank-ordered performance from best to worst. Rater agreement and correlations between the 2 methods were calculated. Regression analysis determined items that optimally discriminated quality. When calculating a score based on all clinically important terms, multiple weightings were evaluated. RESULTS: Metric items had high average rater agreement (80%) with intraclass correlation coefficients averaging 0.83. Correlations of the reference rank and score were high for both video collections (-0.96, P < .05, and -0.95, P < .05). Regression coefficients for different item weighting methods indicated strong relationships with global ratings (averaging r = 0.89, P < .05) and rankings averaging -0.85, P < .05). Prediction of global ratings using regression achieved high accuracy (R 2 = 0.8218). CONCLUSIONS: High observer agreement and strong correlations between metric and rank data support the validity of using this metric to assess ETI performance. Different weighting models yielded scores that correlated strongly with the ratings and ranks from global assessment. When using the metric to predict competency, a 3-item regression model is most accurate in predicting a global score.

Less pain and earlier discharge after implementation of a multidisciplinary enhanced recovery after surgery (ERAS) protocol for laparoscopic sleeve gastrectomy.

BACKGROUND: Laparoscopic sleeve gastrectomy (LSG) may be complicated by postoperative pain, nausea, and vomiting, with consequent increases in length of stay (LOS), decreased patient satisfaction, and higher costs. While enhanced recovery after surgery (ERAS) protocols have been in circulation for many years, there is no standard ERAS protocol for bariatric surgery. METHODS: Data were collected prospectively and compared to a historical control. All patients undergoing LSG, ages 18 to 75, were included in the pathway; those with preoperative chronic opioid use were excluded from our results. Statistical analysis was performed using t-statistics and chi-squared test. Ninety patients undergoing LSG, performed by a single surgeon, were included in our ERAS group from November 26, 2018, to April 30, 2019, and were compared to a historical control of 570 patients who underwent LSG over the previous 5 years (pre-ERAS). Measured outcomes included discharge opioid prescriptions issued, hospital length of stay, 30-day readmissions, reoperations, morbidity, and mortality. RESULTS: Ten (11%) ERAS patients vs 100% of pre-ERAS patients received opioid prescriptions upon, or after, discharge (p < 0.001). The ERAS group LOS decreased to 1.36 days vs 2.40 days in the pre-ERAS group (p < 0.001). 30-day readmission rates were 0% for ERAS patients vs 3.09% for pre-ERAS patients (p = 0.149). 30-day reoperation rates were 0% for ERAS patients vs 0.54% for pre-ERAS patients (p = 1). Thirty-day morbidity rates were 3.33% (3) for ERAS patients vs 3.27% for pre-ERAS patients (p = 1); there was no 30-day mortality in either group. CONCLUSION: ERAS for LSG results in a clinical and statistically significant reduction in postoperative opioid use and LOS, without increasing 30-day readmissions, reoperations, morbidity, or mortality.

Customizable Curriculum to Enhance Resident Communication Skills.

Communication remains challenging to teach and evaluate. We designed an online patient survey to assess anesthesia residents' communication skills from August 2014 to July 2015. In December 2014, we implemented a customized, simulation-based curriculum. We calculated an overall rating for each survey by averaging the ratings for the individual questions. Based on the Hodges-Lehmann 2-sample aligned rank-sum test, overall ratings, reported as the median (interquartile range) of residents' average overall ratings, differed significantly between the preintervention (3.86 [3.76-3.94]) and postintervention (3.91 [3.84-3.95]) periods (P = .025). Future studies should assess the intervention's effectiveness and generalizability.

Faculty-Focused Perioperative Ultrasound Training Program: A Single-Center Experience.

OBJECTIVE: Comprehensive educational initiatives in ultrasound for practicing physicians are lacking. We developed a perioperative ultrasound training program for faculty to offer a broad orientation to the principles and clinical applications of ultrasound. DESIGN AND SETTING: The program consisted of two phases. Phase one, which introduced ultrasound via didactic and hands-on components, had six live sessions. At the end of each, participants completed a knowledge quiz. During the sixth session, faculty participated in an objective structured clinical examination (OSCE). For phase two, faculty attended two to three sessions (8-10 hours each) of in-depth individualized training and demonstrated supervised performance of ultrasound-related procedures of their choice. PARTICIPANTS: Participants included Anesthesia faculty at Beth Israel Deaconess Medical Center. MEASUREMENTS AND MAIN RESULTS: On average 30 faculty members attended each live session for phase one; 12 completed phase two. There was a significant difference in quiz scores across the six sessions (p < 0.001) with scores for Session 6 being significantly higher than for Session 1 (p < 0.001). The average mean and median scores on the three OSCE stations were 95.63% and 98.33%, respectively. For phase two, the 8 participants who received training in regional anesthesia each performed > 10 blocks on patients over two days; 5 of the 7 participants who received training in transthoracic echocardiography each completed more than 15 examinations on simulators and 10 examinations on patients. CONCLUSION: It is possible to implement a departmental educational program for ultrasound to improve ultrasound knowledge and skills in practicing anesthesiologists.

Evaluation of the quality of transesophageal echocardiography images and verification of proficiency

Various metrics have been used in curriculum-based transesophageal echocardiography (TEE) training programs to evaluate acquisition of proficiency. However, the quality of task completion, that is the final image quality, was subjectively evaluated in these studies. Ideally, the endpoint metric should be an objective comparison of the trainee-acquired image with a reference ideal image. Therefore, we developed a simulator-based methodology of preclinical verification of proficiency (VOP) in trainees by tracking objective evaluation of the final acquired images. We utilized geometric data from the simulator probes to compare image acquisition of anesthesia residents who participated in our structured longitudinal simulator-based TEE educational program vs ideal image planes determined from a panel of experts. Thirty-three participants completed the study (15 experts, 7 postgraduate year (PGY)-1 and 11 PGY-4). The results of our study demonstrated a significant difference in image capture success rates between learners and experts (χ2 = 14.716, df = 2, P < 0.001) with the difference between learners (PGY-1 and PGY-4) not being statistically significant (χ2 = 0, df = 1, P = 1.000). Therefore, our results suggest that novices (i.e. PGY-1 residents) are capable of attaining a level of proficiency comparable to those with modest training (i.e. PGY-4 residents) after completion of a simulation-based training curriculum. However, professionals with years of clinical training (i.e. attending physicians) exhibit a superior mastery of such skills. It is hence feasible to develop a simulator-based VOP program in performance of TEE for junior anesthesia residents.

Updated panel report: best practices for the surgical treatment of obesity.

BACKGROUND: During the 2004 annual meeting of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), a panel of experts convened to provide updated information on best practices in bariatric surgery. The rapid evolution of endoluminal technologies, surgical indications, and training in bariatric surgery since 2004 has led to new questions and concerns about optimal treatment algorithms, patient selection, and the preparation of our current and future bariatric workforce. METHODS: An expert panel was convened at the SAGES 2017 annual meeting to provide a summative update on current practice patterns, techniques, and training in bariatric surgery in order to review and establish best practices. This was a joint effort by SAGES, International Society for the Perioperative Care of the Obese Patient, and the American Society for Metabolic and Bariatric Surgery. RESULTS: On March 23, 2017, seven expert faculty convened to address current areas of controversy in bariatric surgery and provide updated guidelines and practice recommendations. Areas addressed included the expanded indications for use of metabolic surgery in the treatment of diabetes, the safety and efficacy of new and investigational endoluminal procedures, updates on new guidelines for the management of airway and sleep apnea in the obese patient, the development of clinical pathways to reduce variation in the management of the bariatric patient, and new guidelines for training, credentialing, and bariatric program accreditation. The following article is a summary of this panel. CONCLUSION: Bariatric surgery is a field that continues to evolve. A timely, systematic approach, such as described here, that coalesces data and establishes best practices on the current body of available evidence is imperative for optimal patient care and to inform provider, insurer, and policy decisions.

Summative Objective Structured Clinical Examination Assessment at the End of Anesthesia Residency for Perioperative Ultrasound.

While standardized examinations and data from simulators and phantom models can assess knowledge and manual skills for ultrasound, an Objective Structured Clinical Examination (OSCE) could assess workflow understanding. We recruited 8 experts to develop an OSCE to assess workflow understanding in perioperative ultrasound. The experts used a binary grading system to score 19 graduating anesthesia residents at 6 stations. Overall average performance was 86.2%, and 3 stations had an acceptable internal reliability (Kuder-Richardson formula 20 coefficient >0.5). After refinement, this OSCE can be combined with standardized examinations and data from simulators and phantom models to assess proficiency in ultrasound.

Simulation-based Assessment to Reliably Identify Key Resident Performance Attributes.

BACKGROUND: Obtaining reliable and valid information on resident performance is critical to patient safety and training program improvement. The goals were to characterize important anesthesia resident performance gaps that are not typically evaluated, and to further validate scores from a multiscenario simulation-based assessment. METHODS: Seven high-fidelity scenarios reflecting core anesthesiology skills were administered to 51 first-year residents (CA-1s) and 16 third-year residents (CA-3s) from three residency programs. Twenty trained attending anesthesiologists rated resident performances using a seven-point behaviorally anchored rating scale for five domains: (1) formulate a clear plan, (2) modify the plan under changing conditions, (3) communicate effectively, (4) identify performance improvement opportunities, and (5) recognize limits. A second rater assessed 10% of encounters. Scores and variances for each domain, each scenario, and the total were compared. Low domain ratings (1, 2) were examined in detail. RESULTS: Interrater agreement was 0.76; reliability of the seven-scenario assessment was r = 0.70. CA-3s had a significantly higher average total score (4.9 ± 1.1 vs. 4.6 ± 1.1, P = 0.01, effect size = 0.33). CA-3s significantly outscored CA-1s for five of seven scenarios and domains 1, 2, and 3. CA-1s had a significantly higher proportion of worrisome ratings than CA-3s (chi-square = 24.1, P < 0.01, effect size = 1.50). Ninety-eight percent of residents rated the simulations more educational than an average day in the operating room. CONCLUSIONS: Sensitivity of the assessment to CA-1 versus CA-3 performance differences for most scenarios and domains supports validity. No differences, by experience level, were detected for two domains associated with reflective practice. Smaller score variances for CA-3s likely reflect a training effect; however, worrisome performance scores for both CA-1s and CA-3s suggest room for improvement.

Enhancing Feedback on Professionalism and Communication Skills in Anesthesia Residency Programs.

BACKGROUND: Despite its importance, training faculty to provide feedback to residents remains challenging. We hypothesized that, overall, at 4 institutions, a faculty development program on providing feedback on professionalism and communication skills would lead to (1) an improvement in the quantity, quality, and utility of feedback and (2) an increase in feedback containing negative/constructive feedback and pertaining to professionalism/communication. As secondary analyses, we explored these outcomes at the individual institutions. METHODS: In this prospective cohort study (October 2013 to July 2014), we implemented a video-based educational program on feedback at 4 institutions. Feedback records from 3 months before to 3 months after the intervention were rated for quality (0-5), utility (0-5), and whether they had negative/constructive feedback and/or were related to professionalism/communication. Feedback records during the preintervention, intervention, and postintervention periods were compared using the Kruskal-Wallis and χ tests. Data are reported as median (interquartile range) or proportion/percentage. RESULTS: A total of 1926 feedback records were rated. The institutions overall did not have a significant difference in feedback quantity (preintervention: 855/3046 [28.1%]; postintervention: 896/3327 [26.9%]; odds ratio: 1.06; 95% confidence interval, 0.95-1.18; P = .31), feedback quality (preintervention: 2 [1-4]; intervention: 2 [1-4]; postintervention: 2 [1-4]; P = .90), feedback utility (preintervention: 1 [1-3]; intervention: 2 [1-3]; postintervention: 1 [1-2]; P = .61), or percentage of feedback records containing negative/constructive feedback (preintervention: 27%; intervention: 32%; postintervention: 25%; P = .12) or related to professionalism/communication (preintervention: 23%; intervention: 33%; postintervention: 24%; P = .03). Institution 1 had a significant difference in feedback quality (preintervention: 2 [1-3]; intervention: 3 [2-4]; postintervention: 3 [2-4]; P = .001) and utility (preintervention: 1 [1-3]; intervention: 2 [1-3]; postintervention: 2 [1-4]; P = .008). Institution 3 had a significant difference in the percentage of feedback records containing negative/constructive feedback (preintervention: 16%; intervention: 28%; postintervention: 17%; P = .02). Institution 2 had a significant difference in the percentage of feedback records related to professionalism/communication (preintervention: 26%; intervention: 57%; postintervention: 31%; P < .001). CONCLUSIONS: We detected no overall changes but did detect different changes at each institution despite the identical intervention. The intervention may be more effective with new faculty and/or smaller discussion sessions. Future steps include refining the rating system, exploring ways to sustain changes, and investigating other factors contributing to feedback quality and utility.