How well do faculty do in providing general surgery EPA feedback?

INTRODUCTION: Entrustable Professional Activities (EPAs) provide a framework for competency-based assessment in surgery. EPA descriptions include observable behaviors by trainees at progressive levels of autonomy. The American Board of Surgery (ABS) required all General Surgery (GS) residency programs to implement assessment of 18 EPAs at the beginning of academic year 2023-2024. Microassessments provide formative self-reflection by the resident and feedback by faculty upon completion of the EPA. These frequent assessments culminate in a resident performance profile utilized by the trainee for formative growth and the clinical competency committee for summative feedback. Assessor free text comments are an opportunity to provide meaningful, constructive feedback to residents. Our aim was to analyze comments provided by faculty to residents in terms of their alignment with EPA descriptors and provision of actionable feedback. METHODS: A total of 540 â€‹GS EPA assessments for inguinal hernia, gallbladder disease, appendicitis, trauma, and surgical consultation were evaluated from 6/2021-12/2022. We assessed free text EPA comments from faculty compared to EPA behavior descriptions for alignment with the selected EPA level of entrustment. The comments were judged on a binary scale of "Align" vs "Not Align" by two independent evaluators, with a third evaluator to address discordance. Comments were then evaluated for resident behavioral descriptions, suggestions for improvement, and positive or negative feedback. RESULTS: Approximately 77 â€‹% of EPA microassessments had alignment between level of autonomy and free text feedback. A common example of feedback discordant with level of autonomy was rating a trainee at an intraoperative level 4 (independent practice) with comments such as "required some guidance with retrocecal case and upsizing port." Based on behavior descriptions this would be a level 3 (indirect supervision). Approximately 88 â€‹% of feedback contained positive comments with minimal negative feedback (e.g., "this did not go well."). Actionable feedback including "work on optimization of retracting hand" or "continue to work clamp/tie technique and square off each knot" was present in 28.3 â€‹% of feedback. CONCLUSIONS: The majority of faculty provide feedback that is aligned with the behavioral anchors of the EPAs assessed, but frequently did not provide actionable feedback to the resident regarding how to advance to the next level of entrustment. EPA entrustment behaviors provide a framework for the development of practice-ready behaviors, and if assessors anchor their feedback in the behaviors for a given entrustment level and project how a resident could proceed to the next level, they can provide a clear trajectory for skill development. Faculty development should focus on improving the frequency of actionable free text feedback, outlining how residents can advance in the future.

Surgical Residents' Perception of Feedback on Their Education: Protocol for a Scoping Review.

BACKGROUND: Feedback is an essential tool for learning and improving performance in any sphere of education, including training of resident physicians. The learner's perception of the feedback they receive is extremely relevant to their learning progress, which must aim at providing qualified care for patients. Studies pertinent to the matter differ substantially with respect to methodology, population, context, and objective, which makes it even more difficult to achieve a clear understanding of the topic. A scoping review on this theme will unequivocally enhance and organize what is already known. OBJECTIVE: The aim of this study is to identify and map out data from studies that report surgical residents' perception of the feedback received during their education. METHODS: The review will consider studies on the feedback perception of resident physicians of any surgical specialty and age group, attending any year of residency, regardless of the type of feedback given and the way the perceptions were measured. Primary studies published in English, Spanish, and Portuguese since 2017 will be considered. The search will be carried out in 6 databases and reference lists will also be searched for additional studies. Duplicates will be removed, and 2 independent reviewers will screen the selected studies' titles, abstracts, and full texts. Data extraction will be performed through a tool developed by the researchers. Descriptive statistics and qualitative analysis (content analysis) will be used to analyze the data. A summary of the results will be presented in the form of diagrams, narratives, and tables. RESULTS: The findings of this scoping review were submitted to an indexed journal in July 2024, currently awaiting reviewer approval. The search was executed on March 15, 2024, and resulted in 588 articles. After the exclusion of the duplicate articles and those that did not meet the eligibility criteria as well as the inclusion of articles through a manual search, 13 articles were included in the review. CONCLUSIONS: Conducting a scoping review is the best way to map what is known about a subject. By focusing on the feedback perception more than the feedback itself, the results of this study will surely contribute to gaining a deeper understanding of how to proceed to enhance internal feedback and surgical residents' learning progress. TRIAL REGISTRATION: Open Science Framework yexb; https://osf.io/yexkb. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/56727.

An online training and feedback module enhances the musculoskeletal examination performance of medical interns.

BACKGROUND: Pathologies of the locomotor system are frequent and can cause disability and impact the quality of life of the people affected. In recent years, online training and feedback have emerged as learning tools in many fields of medicine. OBJECTIVE: This study aims to evaluate medical interns' musculoskeletal examination performance after completing an online training and feedback module. METHODS: This study employed a quasi-experimental design. Medical interns were invited to complete a 4-week musculoskeletal physical examination training and feedback module via an e-learning platform. The course included written and audiovisual content pertaining to medical history, physical examination, and specific tests for the diagnosis of the most common knee, spine, shoulder, ankle, and foot conditions. Before and after completing the module, their ability to perform the physical examination was evaluated using an objective structured clinical examination (OSCE) with simulated patients that took place face-to-face. A control group of experts was assessed using the OSCE, and their performance was compared to that of the interns before and after the training. At the end of the module feedback on the OSCE was provided to participants through the platform asynchronously and two evaluation questions about the user experience were conducted at the end of the study. RESULTS: A total of 35 subjects were assessed using the OSCE, including 29 interns and 6 experts. At the beginning of the training module, the group of interns obtained an average score of 50.6 ± 15.1. At the end of the module, 18 interns retook the OSCE, and their performance increased significantly to an average of 76.6 ± 12.8 (p < 0.01). Prior to the training, the experts performed significantly better than the interns (71.2 vs. 50.6; p = 0.01). After the interns received the training and feedback, there were no significant differences between the two groups (71.2 vs. 76.6; p = 0.43). Two evaluation questions were conducted at the end of the study, revealing that 93% of the participants affirm that the training module will be useful in their clinical practice, and 100% of the participants would recommend the training module to a colleague. CONCLUSION: The online training and feedback module enhances the musculoskeletal examination performance of medical interns.

A mixed-methods approach to repetitive formative assessment with timely feedback on instructional perception in doctor of pharmacy students.

INTRODUCTION: Formative assessment assists learning, but how Doctor of Pharmacy (PharmD) students perceive repeated formative assessment is unclear. METHODS AND MATERIALS: This study sought to determine perceptions of repeated formative assessment with timely feedback on student learning in third-year PharmD students. This mixed methods approach included four surveys and a qualitative interview. Five formative assessments were assigned to third-year PharmD students throughout a fall course, and then repeated in a spring course for the same cohort. Paired pre-and post-course surveys administered in both courses contained items corresponding to formative assessment perceptions. Survey items included domains of knowledge, engagement, feedback, and confidence, and effect size was determined using Cohen's d. Following the second course, students were invited to take part in a qualitative interview to further characterize perceptions. RESULTS: Overall, 19 and 18 students participated in paired fall and spring pre- and post-surveys, respectively. The standardized mean difference for 12 out of 24 total survey items (58.3%) indicated small to medium positive effect sizes following the intervention, two out of 24 (8.3%) with medium to strong positive effect sizes, and one out of 24 (4.17%) with a strong positive effect size. Eight students participated in a quantitative interview; response themes included "think," "critiquing," "helped," and "helpful." CONCLUSIONS: In two PharmD courses, students expressed a generally small to moderate perceived benefit on repeat formative assessment in domains representing knowledge, engagement, feedback, and confidence.

[Development of the capacity to communicate risks related to childhood exposure to fluorides through an online educational strategy]. / Desarrollo de la capacidad de comunicar riesgos relacionados con la exposición infantil a fluoruros, a través de una estrategia educativa en línea.

Fluorides are contaminants that occur frequently and, generally, naturally in groundwater, affecting countries that depend on these waters for irrigation and human consumption. Chronic exposure to fluorides generates various health effects; therefore, this research was based on education and risk communication to contribute to the resolution of the problem of fluoride exposure in the population. The objective was to develop the capacity to design risk communication programs for personnel involved in the response and management of environmental health risks, with emphasis on fluoride exposure. An online pilot training course on risk communication and fluoride exposure was designed and implemented. For the analysis of the risk perception and knowledge of the participants, before and after the course, a questionnaire was applied and a focus group was conducted. In addition, the participants carried out a series of activities and designed a risk communication program to assess the degree to which the capacity to develop risk communication programs was achieved. To improve the pilot course, two satisfaction surveys were designed and implemented, and a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis was conducted. The results showed an increase in the level of knowledge and changes in the participants' perception; regarding the ability to design risk communication programs, two participants were able to design them excellently. Previous experience, motivation, commitment to learn and the feedback provided during the course influenced the development of this ability.

Los fluoruros son contaminantes presentes con frecuencia y generalmente de forma natural en aguas subterráneas, y afectan a países que dependen de estas aguas para el riego y el consumo humano. La exposición crónica a fluoruros genera diversos efectos a la salud; por lo anterior, esta investigación se basó en la educación y la comunicación de riesgos para contribuir a la resolución del problema de exposición a fluoruros en la población. El objetivo fue desarrollar la capacidad de diseñar programas de comunicación de riesgos del personal involucrado en la respuesta y manejo de los riesgos ambientales para la salud, con énfasis en la exposición a fluoruros. Se diseñó e implementó un curso piloto de formación en línea sobre comunicación de riesgos y exposición a fluoruros. Para el análisis de la percepción de riesgos y conocimientos de los participantes, antes y después del curso, se aplicó un cuestionario y se llevó a cabo un grupo focal. Además, los asistentes realizaron una serie de actividades y diseñaron un programa de comunicación de riesgos con el que se valoró el grado en que se alcanzó la capacidad de desarrollar programas de comunicación de riesgos. Para mejorar el curso piloto se diseñaron y aplicaron dos encuestas de satisfacción y se realizó un análisis FODA (Fortalezas, Oportunidades, Debilidades, Amenazas). Los resultados mostraron un incremento en el nivel de conocimientos y cambios en la percepción de los participantes; en cuanto a la capacidad de diseñar programas de comunicación de riesgos, dos participantes lograron diseñarlo de manera excelente. La experiencia previa, la motivación, el compromiso para aprender y la retroalimentación brindada durante el curso, influyeron en el desarrollo de esta capacidad.

Os fluoretos são contaminantes que ocorrem com frequência e, geralmente, de forma natural nas águas subterrâneas, afetando os países que dependem dessas águas para irrigação e consumo humano. A exposição crônica aos fluoretos gera vários efeitos à saúde; portanto, esta pesquisa baseou-se na educação e na comunicação de riscos para contribuir com a solução do problema da exposição ao fluoreto na população. O objetivo foi desenvolver a capacidade de elaborar programas de comunicação de risco para o pessoal envolvido na resposta e no gerenciamento de riscos ambientais à saúde, com ênfase na exposição à fluoretos. Foi elaborado e implementado um curso piloto de treinamento online sobre comunicação de riscos e exposição. Para a análise da percepção de risco e do conhecimento dos participantes antes e depois do curso, foi aplicado um questionário e aplicado um grupo de foco. Além disso, os participantes realizaram uma série de atividades e elaboraram um programa de comunicação de riscos para avaliar até que ponto a capacidade de desenvolver programas de comunicação de riscos foi alcançada. Para aprimorar o curso piloto, foram duas pesquisas de satisfação foram desenvolvidas e implementadas e uma análise FOFA (Forças, Oportunidades, Fraquezas e Ameaças) foi aplicada. Os resultados mostraram um aumento no nível de conhecimento e mudanças nas percepções dos participantes; em termos da capacidade de elaborar programas de comunicação de riscos, dois participantes conseguiram elaborar excelentes programas de comunicação de riscos. A experiência prévia, a motivação, o compromisso com o aprendizado e o feedback fornecido durante o curso influenciaram o desenvolvimento dessa capacidade.

Mobile Application to Improve Just-in-Time 2-Way Formative Feedback in Graduate Medical Education.

Background An easy-to-use application to facilitate direct observation and allow for 2-way feedback between residents and faculty is needed. Objective To develop a mobile-based application (app) with the goals of (1) providing just-in-time feedback to residents; (2) improving timeliness of feedback by faculty; and (3) allowing residents to comment on the value of faculty feedback. Methods Fifty-one of 69 (74%) internal medicine (IM) residents and 20 of 25 (80%) IM core faculty participated in the study from July 1, 2020, to December 31, 2021. An iOS app was designed by authors with expertise in medical education and application development to capture entrustable professional activities (EPAs)-based feedback (eg, informed consent) based on direct observation of residents' skills in the workplace. App utilization and narrative feedback characteristics of faculty comments were examined by exporting the data from the database server. The end user satisfaction was examined using a survey instrument. Results Eighty-seven percent of assessments (117 of 134) initiated were fully completed by residents and faculty. Faculty narrative comments were noted in 97% (114 of 117) of completed assessments and 64% (75 of 117) of residents' feedback to the faculty contained narrative comments. Eighty-three percent (97 of 117) of comments were behaviorally specific and 71% (83 of 117) contained an actionable item. Eighty-six percent (18 of 21) of residents and 90% (9 of 10) of core faculty stated that this application promoted an educational interaction between them. Conclusions This app facilitates the efficient completion of EPA-based formative assessments and captures bidirectional feedback in the workplace setting.

Keys to successful clinical audit and feedback: essential steps to making impactful improvements in patient care.

ABSTRACT: Up to 40% of care provided to patients is either wasteful or harmful. The practice of audit and feedback can help identify where care can be improved. However, such audits must be executed in a systematic way that engages with clinicians to maximize the impact of feedback, ultimately improving patient outcomes. Currently, audit training is not integrated into formal education pathways and clinicians need guidance to support them in this activity. This paper explores contemporary research, with the aim of providing practical advice for recommendations to maximize the impact of audit and feedback. SPANISH ABSTRACT: http://links.lww.com/IJEBH/A239.

Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency.

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen's University Internal Medicine residency program introduced a facilitated, team-based assessment initiative ("Feedback Fridays") in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program's assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions: The introduction of facilitated team-based assessment in the Queen's Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.

Objectif: La formation médicale fondée sur les compétences s'appuie sur la rétroaction faite lors de l'évaluation des apprentissages par observation directe dans le milieu de travail. Malheureusement, les évaluations dans le milieu de travail omettent souvent de fournir une rétroaction narrative exhaustive et privilégient les aspects des soins relevant de l'expertise médicale. En se basant sur la recherche ayant étudié les approches d'évaluation interactive, le programme de résidence en médecine interne de l'Université Queen's a introduit en juillet 2017 une initiative d'évaluation facilitée et en équipe (« Les vendredis rétroaction ¼), visant à améliorer l'évaluation holistique du rendement des résidents dans les unités d'enseignement clinique en médecine interne. Dans cette étude, nous visons à explorer comment ces « vendredis rétroaction ¼ ont contribué à l'évaluation formative des résidents en médecine interne dans le cadre de notre modèle actuel de formation axée sur les compétences. Méthode: Au total, 53 résidents ont participé à des séances d'évaluation de groupe facilitées et bi-hebdomadaires au cours de l'année universitaire 2017-2018. Chaque séance consistait en une discussion d'évaluation facilitée de 30 minutes menée avec une équipe de l'unité de soins, qui comprenait des étudiants en médecine, des résidents et le médecin superviseur. Les commentaires issus de la discussion ont été recueillis, résumés et documentés sous forme narrative dans des formulaires électroniques d'observation directe dans le milieu de travail par le responsable de l'évaluation du programme de résidence. À des fins de recherche, les transcriptions verbatim des séances de rétroaction ont été analysées de façon thématique. Résultats: Les chercheurs ont identifié quatre thèmes principaux pour les commentaires : la communication, la conscience intra- et interpersonnelle, le leadership et le travail d'équipe, et les occasions d'apprentissage. Bien que la rétroaction concerne un large éventail d'activités, elle met fortement l'accent sur les compétences liées aux rôles intrinsèques de CanMEDS. De plus, le fait que la rétroaction avait un rôle clairement formatif est une autre constatation importante. Conclusions: L'introduction de l'évaluation en équipe facilitée dans le programme de médecine interne à Queen's a comblé une lacune importante dans l'apprentissage par observation directe dans le milieu de travail en fournissant aux apprenants une rétroaction détaillée sur tous les rôles CanMEDS et en formulant des recommandations constructives sur les domaines à améliorer.

Feedback in an Entrustment-Based Objective Structured Clinical Examination: Analysis of Content and Scoring Methods.

Background The integration of entrustable professional activities (EPAs) within objective structured clinical examinations (OSCEs) has yielded a valuable avenue for delivering timely feedback to residents. However, concerns about feedback quality persist. Objective This study aimed to assess the quality and content alignment of verbal feedback provided by examiners during an entrustment-based OSCE. Methods We conducted a progress test OSCE for internal medicine residents in 2022, assessing 7 EPAs. The immediate 2-minute feedback provided by examiners was recorded and analyzed using the Quality of Assessment of Learning (QuAL) score. We also analyzed the degree of alignment with EPA learning objectives: competency milestones and task-specific abilities. In a randomized crossover experiment, we compared the impact of 2 scoring methods used to assess residents' clinical performance (3-point entrustability scales vs task-specific checklists) on feedback quality and alignment. Results Twenty-one examiners provided feedback to 67 residents. The feedback demonstrated high quality (mean QuAL score 4.3 of 5) and significant alignment with the learning objectives of the EPAs. On average, examiners addressed in their feedback 2.5 milestones (61%) and 1.2 task-specific abilities (46%). The scoring methods used had no significant impact on QuAL scores (95% CI -0.3, 0.1, P=.28), alignment with competency milestones (95% CI -0.4, 0.1, P=.13), or alignment with task-specific abilities (95% CI -0.3, 0.1, P=.29). Conclusions In our entrustment-based OSCE, examiners consistently offered valuable feedback aligned with intended learning outcomes. Notably, we explored high-quality feedback and alignment as separate dimensions, finding no significant impact from our 2 scoring methods on either aspect.

Comparing Peer and Faculty Feedback for Asynchronous Laparoscopic Skill Acquisition.

OBJECTIVE: Traditionally, expert surgeons have provided surgical trainees with feedback about their simulation performance, including for asynchronous practice. Unfortunately, innumerable time demands may limit experts' ability to provide feedback. It is unknown whether and how peer feedback is an effective mechanism to help residents acquire laparoscopic skill in an asynchronous setting. As such, we aimed to assess the effect of peer feedback on laparoscopic performance and determine how residents perceive giving and receiving peer feedback. DESIGN: We conducted a convergent mixed methods study. In the quantitative component, we randomized residents to receive feedback on home laparoscopic tasks from peers or faculty. We then held an end-of-curriculum, in-person laparoscopic assessment with members from both groups and compared performance on the in-person assessment between the groups. In the qualitative component, we conducted interviews with resident participants to explore experiences with feedback and performance. Three authors coded and rigorously reviewed interview data using a directed content analysis. SETTING: We performed this study at a single tertiary academic institution: the University of California, San Francisco. PARTICIPANTS: We invited 47 junior residents in general surgery, obstetrics-gynecology, and urology to participate, of whom 37 (79%) participated in the home curriculum and 25 (53%) participated in the end-of-curriculum assessment. RESULTS: Residents in the peer feedback group scored similarly on the final assessment (mean 70.7%; SD 16.1%) as residents in the faculty feedback group (mean 71.8%; SD 11.9%) (p = 0.86). Through qualitative analysis of interviews with 13 residents, we identified key reasons for peer feedback's efficacy: shared mental models, the ability to brainstorm and appreciate new approaches, and a low-stakes learning environment. CONCLUSIONS: We found that peer and faculty feedback led to similar performance in basic laparoscopy and that residents engaged positively with peer feedback, suggesting that peer feedback can be used when residents learn basic laparoscopy.

Effects of Real Time Feedback on Novice's Laparoscopic Learning Curve.

OBJECTIVE: Analyze the learning curve of medical students when they are trained with a laparoscopic box trainer and are presented with different modes of real-time feedback on their performance in the laparoscopic suturing procedure. DESIGN: A prospective randomized controlled trial (RTC) was performed. Three groups were proposed: control, visual, and haptic. The block randomization technique was used to assign the participants to one of the three groups. Each group trained the intracorporeal square flat knot task in a standardized FLS box simulator. A total of 11 sessions were programmed for this study: a pre-training session, eight training sessions, a post-training session, and a follow-up session. Measurement of the generated reaction force during the task were taken weekly for the first 10 sessions (pre-training, training, and post-training); then, the follow-up measurement was taken a month after. SETTING: This study was carried out in a single center at the Unidad de Simulación de Posgrado (USIP) of the postgraduate medicine program of the Universidad Nacional Autónoma de México (UNAM) between May and August 2023. PARTICIPANTS: The eligible participants were medical students without experience in minimally invasive surgery. All social service intern medics doing their social service in the USIP were invited to participate. A total of 20 participants entered the study from which 18 of them finished all the programmed sessions. RESULTS: A total of ten metrics were extracted from the reaction force signal measured at each session. All metrics are directly proportional to the reaction force and low magnitudes imply high tissue-handling proficiency. All groups improved their tissue handling skills, being the visual group the one who achieved better performance, followed by the haptic group and lastly the control group. CONCLUSION: The use of real time feedback, especially visual feedback can help novices to shorten the learning process of tissue handling and achieve a better proficiency in advanced tasks in shorter training periods.

Experiences of medical students and faculty regarding the use of long case as a formative assessment method at a tertiary care teaching hospital in a low resource setting: a qualitative study.

INTRODUCTION: The long case is used to assess medical students' proficiency in performing clinical tasks. As a formative assessment, the purpose is to offer feedback on performance, aiming to enhance and expedite clinical learning. The long case stands out as one of the primary formative assessment methods for clinical clerkship in low-resource settings but has received little attention in the literature. OBJECTIVE: To explore the experiences of medical students and faculty regarding the use of the Long Case Study as a formative assessment method at a tertiary care teaching hospital in a low-resource setting. METHODOLOGY: A qualitative study design was used. The study was conducted at Makerere University, a low-resource setting. The study participants were third- and fifth-year medical students as well as lecturers. Purposive sampling was utilized to recruit participants. Data collection comprised six Focus Group Discussions with students and five Key Informant Interviews with lecturers. The qualitative data were analyzed by inductive thematic analysis. RESULTS: Three themes emerged from the study: ward placement, case presentation, and case assessment and feedback. The findings revealed that students conduct their long cases at patients' bedside within specific wards/units assigned for the entire clerkship. Effective supervision, feedback, and marks were highlighted as crucial practices that positively impact the learning process. However, challenges such as insufficient orientation to the long case, the super-specialization of the hospital wards, pressure to hunt for marks, and inadequate feedback practices were identified. CONCLUSION: The long case offers students exposure to real patients in a clinical setting. However, in tertiary care teaching hospitals, it's crucial to ensure proper design and implementation of this practice to enable students' exposure to a variety of cases. Adequate and effective supervision and feedback create valuable opportunities for each learner to present cases and receive corrections.

Feedback-based learning from viewpoints of surgical nursing students: A mixed-method study.

BACKGROUND: Feedback-based learning (FBL) focuses on guiding the learning process according to educational objectives and the student's needs. This study aimed to investigate surgical nursing students' perceptions and explore their experiences of FBL. METHOD: The present study used a mixed-methods sequential explanatory design that was conducted in the quantitative and qualitative phases. Surgical nursing students participated in the quantitative phase (n = 105). In the first phase, students completed two questionnaires about FBL and clinical feedback. Semi-structured face-to-face interviews were used to collect qualitative data in the second phase. Graneheim and Lundman's inductive approaches were used to analyse the qualitative data. RESULTS: The mean (SD) score for students' perception of FBL was 3.99 ± 0.70. The qualitative results were explored in two themes, "motivational support for improvement" and "unpleasant learning". CONCLUSION: In this study, the positive and negative aspects of FBL were explained. FBL is perceived as a motivational support mechanism to improve students' capabilities during their academic courses and also prepare them for future careers. Conversely, FBL may experience unpleasant learning due to negative feedback and negative emotions.

Development and validation of immediate self-feedback very short answer questions for medical students: practical implementation of generalizability theory to estimate reliability in formative examination designs.

BACKGROUND: Very Short Answer Questions (VSAQs) reduce cueing and simulate better real-clinical practice compared with multiple-choice questions (MCQs). While integrating them into formative exams has potential, addressing marking time and ideal occasions and items is crucial. This study gathers validity evidence of novel immediate self-feedback VSAQ (ISF-VSAQ) format and determines the optimal number of items and occasions for reliable assessment. METHODS: Ninety-four third-year pre-clinical students took two ten-item ISF-VSAQ exams on cardiovascular drugs. Each question comprised two sections: (1) Questions with space for student responses and (2) a list of possible correct answers offering partial-credit scores ranging from 0.00 to 1.00, along with self-marking and self-feedback options to indicate whether they fully, partially, or did not understand the possible answers. Messick's validity framework guided the collection of validity evidence. RESULTS: Validity evidence included five sources: (1) Content: The expert reviewed the ISF-VSAQ format, and the question was aligned with a standard examination blueprint. (2) Response process: Before starting, students received an example and guide to the ISF-VSAQ, and the teacher detailed the steps in the initial session to aid self-assessment. Unexpected answers were comprehensively reviewed by experts. (3) Internal structure: The Cronbach alphas are good for both occasions (≥ 0.70). A generalizability study revealed Phi-coefficients of 0.60, 0.71, 0.76, and 0.79 for one to four occasions with ten items, respectively. One occasion requires twenty-five items for acceptable reliability (Phi-coefficient = 0.72). (4) Relations to other variables: Inter-rater reliability between self-marking and teacher is excellent for each item (rs(186) = 0.87-0.98,p = 0.001). (5) Consequences: Path analysis revealed that the self-reflected understanding score in the second attempt directly affected the final MCQ score (ß = 0.25,p = 0.033). However, the VSAQ score did not. Regarding perceptions, over 80% of students strongly agreed/agreed that the ISF-VSAQ format enhances problem analysis, presents realistic scenarios, develops knowledge, offers feedback, and supports electronic usability. CONCLUSION: Electronic ISF-VSAQs enhanced understanding elevates learning outcomes, rendering them suitable for formative assessments with clinical scenarios. Increasing the number of occasions effectively enhances reliability. While self-marking is reliable and may reduce grading efforts, instructors should review answers to identify common student errors.

Closing the loop in minimally supervised human-robot interaction: formative and summative feedback.

Human instructors fluidly communicate with hand gestures, head and body movements, and facial expressions, but robots rarely leverage these complementary cues. A minimally supervised social robot with such skills could help people exercise and learn new activities. Thus, we investigated how nonverbal feedback from a humanoid robot affects human behavior. Inspired by the education literature, we evaluated formative feedback (real-time corrections) and summative feedback (post-task scores) for three distinct tasks: positioning in the room, mimicking the robot's arm pose, and contacting the robot's hands. Twenty-eight adults completed seventy-five 30-s-long trials with no explicit instructions or experimenter help. Motion-capture data analysis shows that both formative and summative feedback from the robot significantly aided user performance. Additionally, formative feedback improved task understanding. These results show the power of nonverbal cues based on human movement and the utility of viewing feedback through formative and summative lenses.

Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education.

INTRODUCTION: Since 2022, all Canadian post-graduate medical programs have transitioned to a Competence by Design (CBD) model within a Competency-Based Medical Education (CBME) framework. The CBME model emphasized more frequent, formative assessment of residents to evaluate their progress towards predefined competencies in comparison to traditional medical education models. Faculty members therefore have increased responsibility for providing assessments to residents on a more regular basis, which has associated challenges. Our study explores faculty assessment behaviours within the CBD framework and assesses their openness to opportunities aimed at improving the quality of written feedback. Specifically, we explore faculty's receptiveness to routine metric performance reports that offer comprehensive feedback on their assessment patterns. METHODS: Online surveys were distributed to all 28 radiology faculty at Queen's University. Data were collected on demographics, feedback practices, motivations for improving the teacher-learner feedback exchange, and openness to metric performance reports and quality improvement measures. Following descriptive statistics, unpaired t-tests and one-way analysis of variance were conducted to compare groups based on experience and subspecialty. RESULTS: The response rate was 89% (25/28 faculty). 56% of faculty were likely to complete evaluations after working with a resident. Regarding the degree to which faculty felt written feedback is important, 62% found it at least moderately important. A majority (67%) believed that performance reports could influence their evaluation approach, with volume of written feedback being the most likely to change. Faculty expressed interest in feedback-focused development opportunities (67%), favouring Grand Rounds and workshops. CONCLUSION: Assessment of preceptor perceptions reveals that faculty recognize the importance of offering high-quality written feedback to learners. Faculty openness to quality improvement interventions for curricular reform relies on having sufficient time, knowledge, and skills for effective assessments. This suggests that integrating routine performance metrics into faculty assessments could serve as a catalyst for enhancing future feedback quality.

Exploring the content and delivery of feedback facilitation co-interventions: a systematic review.

BACKGROUND: Policymakers and researchers recommend supporting the capabilities of feedback recipients to increase the quality of care. There are different ways to support capabilities. We aimed to describe the content and delivery of feedback facilitation interventions delivered alongside audit and feedback within randomised controlled trials. METHODS: We included papers describing feedback facilitation identified by the latest Cochrane review of audit and feedback. The piloted extraction proforma was based upon a framework to describe intervention content, with additional prompts relating to the identification of influences, selection of improvement actions and consideration of priorities and implications. We describe the content and delivery graphically, statistically and narratively. RESULTS: We reviewed 146 papers describing 104 feedback facilitation interventions. Across included studies, feedback facilitation contained 26 different implementation strategies. There was a median of three implementation strategies per intervention and evidence that the number of strategies per intervention is increasing. Theory was used in 35 trials, although the precise role of theory was poorly described. Ten studies provided a logic model and six of these described their mechanisms of action. Both the exploration of influences and the selection of improvement actions were described in 46 of the feedback facilitation interventions; we describe who undertook this tailoring work. Exploring dose, there was large variation in duration (15-1800 min), frequency (1 to 42 times) and number of recipients per site (1 to 135). There were important gaps in reporting, but some evidence that reporting is improving over time. CONCLUSIONS: Heterogeneity in the design of feedback facilitation needs to be considered when assessing the intervention's effectiveness. We describe explicit feedback facilitation choices for future intervention developers based upon choices made to date. We found the Expert Recommendations for Implementing Change to be valuable when describing intervention components, with the potential for some minor clarifications in terms and for greater specificity by intervention providers. Reporting demonstrated extensive gaps which hinder both replication and learning. Feedback facilitation providers are recommended to close reporting gaps that hinder replication. Future work should seek to address the 'opportunity' for improvement activity, defined as factors that lie outside the individual that make care or improvement behaviour possible. REVIEW REGISTRATION: The study protocol was published at: https://www.protocols.io/private/4DA5DE33B68E11ED9EF70A58A9FEAC02 .

Pneumonia diagnosis performance in the emergency department: a mixed-methods study about clinicians' experiences and exploration of individual differences and response to diagnostic performance feedback.

OBJECTIVES: We sought to (1) characterize the process of diagnosing pneumonia in an emergency department (ED) and (2) examine clinician reactions to a clinician-facing diagnostic discordance feedback tool. MATERIALS AND METHODS: We designed a diagnostic feedback tool, using electronic health record data from ED clinicians' patients to establish concordance or discordance between ED diagnosis, radiology reports, and hospital discharge diagnosis for pneumonia. We conducted semistructured interviews with 11 ED clinicians about pneumonia diagnosis and reactions to the feedback tool. We administered surveys measuring individual differences in mindset beliefs, comfort with feedback, and feedback tool usability. We qualitatively analyzed interview transcripts and descriptively analyzed survey data. RESULTS: Thematic results revealed: (1) the diagnostic process for pneumonia in the ED is characterized by diagnostic uncertainty and may be secondary to goals to treat and dispose the patient; (2) clinician diagnostic self-evaluation is a fragmented, inconsistent process of case review and follow-up that a feedback tool could fill; (3) the feedback tool was described favorably, with task and normative feedback harnessing clinician values of high-quality patient care and personal excellence; and (4) strong reactions to diagnostic feedback varied from implicit trust to profound skepticism about the validity of the concordance metric. Survey results suggested a relationship between clinicians' individual differences in learning and failure beliefs, feedback experience, and usability ratings. DISCUSSION AND CONCLUSION: Clinicians value feedback on pneumonia diagnoses. Our results highlight the importance of feedback about diagnostic performance and suggest directions for considering individual differences in feedback tool design and implementation.