Closing the loop in minimally supervised human-robot interaction: formative and summative feedback.

Human instructors fluidly communicate with hand gestures, head and body movements, and facial expressions, but robots rarely leverage these complementary cues. A minimally supervised social robot with such skills could help people exercise and learn new activities. Thus, we investigated how nonverbal feedback from a humanoid robot affects human behavior. Inspired by the education literature, we evaluated formative feedback (real-time corrections) and summative feedback (post-task scores) for three distinct tasks: positioning in the room, mimicking the robot's arm pose, and contacting the robot's hands. Twenty-eight adults completed seventy-five 30-s-long trials with no explicit instructions or experimenter help. Motion-capture data analysis shows that both formative and summative feedback from the robot significantly aided user performance. Additionally, formative feedback improved task understanding. These results show the power of nonverbal cues based on human movement and the utility of viewing feedback through formative and summative lenses.

Creation and Implementation of an Online Tool for Feedback on Resident Teaching: A Pilot Study.

OBJECTIVE: There are few assessments of the competence and growth of surgical residents as educators. We developed and piloted an observation-based feedback tool (FT) to provide residents direct feedback during a specific teaching session, as perceived by medical students (MS). We hypothesized that residents' performance would improve with frequent, low stakes, observation-based feedback. SETTING: This prospective study took place at an academic general surgery program. PARTICIPANTS: Focus groups of MS, surgical residents, and faculty informed FT development. MS completed the FT regarding resident teaching. DESIGN: The FT utilized 5 slider-bar ratings (0 to 100) about the teaching encounter and a checklist of 16 desirable teaching behaviors. QR codes and weekly email links were distributed for 12 months (6 clerkship blocks) to promote use. Residents were sent their results after each block. A survey after each block assessed motivation for use and gathered feedback on the FT. Descriptive statistics were used for analysis (medians, IQRs). Primary measures of performance were median of the slider-bar scores and the number of teaching behaviors. RESULTS: The FT was used 111 times; 37 of 46 residents were rated by up to 65 MS. The median rating on the slider-bars was 100 and the median number of desirable teaching behaviors was 12; there were no differences based on gender or PGY level. 10 residents had 5 or more FT observations during the year. Four residents had evaluations completed in 4 or more blocks and 19 residents had evaluations completed in at least 2 blocks. Over time, 13 residents had consistent slider-bar scores, 1 resident had higher scores, and 5 residents had lower scores (defined as a more than 5-point change from initial rating). Frequency of use of the FT decreased over time (38, 32, 9, 21, 7, 5 uses per block). The post-use survey was completed by 24 MS and 19 residents. Most common reasons for usage were interest in improving surgical learning environment, giving positive feedback (MS), and improving teaching skills (residents). Most common reasons for lack of usage from residents were "I did not think I taught enough to ask for feedback," "I forgot it existed," and "I did not know it existed." CONCLUSIONS: The FT did not lead to any meaningful improvement in resident scores over the course of the year. This may be due to overall high scores, suggesting that the components of the FT may require reevaluation. Additionally, decreased utilization of the instrument over time made it challenging to assess change in performance of specific residents, likely due to lack of awareness of the FT despite frequent reminders. Successful implementation of observation-based teaching assessments may require better integration with residency or clerkship objectives.

Impact of a coaching program on resident perceptions of communication confidence and feedback quality.

BACKGROUND: While communication is an essential skill for providing effective medical care, it is infrequently taught or directly assessed, limiting targeted feedback and behavior change. We sought to evaluate the impact of a multi-departmental longitudinal residency communication coaching program. We hypothesized that program implementation would result in improved confidence in residents' communication skills and higher-quality faculty feedback. METHODS: The program was implemented over a 3-year period (2019-2022) for surgery and neurology residents at a single institution. Trained faculty coaches met with assigned residents for coaching sessions. Each session included an observed clinical encounter, self-reflection, feedback, and goal setting. Eligible residents completed baseline and follow-up surveys regarding their perceptions of feedback and communication. Quantitative responses were analyzed using paired t-tests; qualitative responses were analyzed using content analysis. RESULTS: The baseline and follow-up survey response rates were 90.0% (126/140) and 50.5% (46/91), respectively. In a paired analysis of 40 respondents, residents reported greater confidence in their ability to communicate with patients (inpatient: 3.7 vs. 4.3, p < 0.001; outpatient: 3.5 vs. 4.2, p < 0.001), self-reflect (3.3 vs. 4.3, p < 0.001), and set goals (3.6 vs. 4.3, p < 0.001), as measured on a 5-point scale. Residents also reported greater usefulness of faculty feedback (3.3 vs. 4.2, p = 0.001). The content analysis revealed helpful elements of the program, challenges, and opportunities for improvement. Receiving mentorship, among others, was indicated as a core program strength, whereas solving session coordination and scheduling issues, as well as lowering the coach-resident ratio, were suggested as some of the improvement areas. CONCLUSIONS: These findings suggest that direct observation of communication in clinical encounters by trained faculty coaches can facilitate long-term trainee growth across multiple core competencies. Future studies should evaluate the impact on patient outcomes and workplace-based assessments.

Critical reflections on formal teaching observations.

Formal teaching observations not only have an essential role in the visibility of teaching and learning but also allow for constructive, critical feedback for improvement. In both nursing and teaching, there is an emphasis on the need for efficient, regular reflective practice; this not to identify negative aspects but to transform professional practice. This article provides a personal insight into formal teaching observations, and themes arising from them of motivation, the teacher's voice, the use of abbreviations and acronyms and the use of exemplars, using a critical reflective approach.

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review.

BACKGROUND: Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback. METHODS: A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework. RESULTS: The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes. CONCLUSIONS: This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education.

OBJECTIVES: The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency. DESIGN: A participatory action research model with a mixed methods approach. SETTING: This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department. PARTICIPANTS: Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention. RESULTS: The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4. CONCLUSION: Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Risky business: medical students' feedback-seeking behaviours: a mixed methods study.

There are differing views on how learners' feedback-seeking behaviours (FSB) develop during training. With globalisation has come medical student migration and programme internationalisation. Western-derived educational practices may prove challenging for diverse learner populations. Exploring undergraduate activity using a model of FSB may give insight into how FSB evolves and the influence of situational factors, such as nationality and site of study. Our findings seek to inform medical school processes that support feedback literacy. Using a mixed methods approach, we collected questionnaire and interview data from final-year medical students in Ireland, Bahrain, and Malaysia. A validated questionnaire investigated relationships with FSB and goal orientation, leadership style preference, and perceived costs and benefits. Interviews with the same student population explored their FSB experiences in clinical practice, qualitatively, enriching this data. The data were integrated using the 'following the thread' technique. Three hundred and twenty-five of a total of 514 completed questionnaires and 57 interviews were analysed. Learning goal orientation (LGO), instrumental leadership and supportive leadership related positively to perceived feedback benefits (0.23, 0.2, and 0.31, respectively, p < 0.05). Perceived feedback benefits are related positively to feedback monitoring and inquiry (0.13 and 0.38, respectively, p < 0.05). The personal cost of feedback is unsupported in quantitative data, but was a strong theme in interviews, as was feedback avoidance, peer feedback, and unsupportive learning environment. No differences were observed across sub-groups based on gender, study site, or student nationality. Integrated analysis describes FSB: avoiding 'unsafe' feedback (first, do no harm) and overcoming barriers (beat the system) and goal-centred curation (shop around) to optimise benefits. Diverse medical students across three continents undertake FSB with careful navigation, as a valued but risky business, that is highly contextualised. Promoting a constructive FSB is complex. Overcoming outdated theory and practices on the wards remains a challenge to psychologically safe, learner-centred feedback.

Can peer feedback substitute for faculty feedback in predoctoral dental education?

PURPOSE/OBJECTIVES: The ability to give and receive feedback is a key skill to develop during predoctoral dental education, and the use of peer feedback specifically offers distinct benefits including a different understanding of material due to peers' proximity of knowledge development and assisting with overburdened instructors. However, it is unclear if peer feedback offers similar quality to instructor feedback. METHODS: Dental students in two different graduation years provided quantitative and qualitative peer feedback on a case-based oral and maxillofacial pathology simulation. The data from these exercises were aggregated and analyzed to compare the quality of qualitative feedback to course examination scores. Student perceptions of peer feedback were also recorded. RESULTS: The mean quality of feedback was not correlated with course examination scores, though the number of times students gave high-quality feedback and received high-quality feedback was correlated with course examination scores. Student feedback overall had a lower quality than instructor feedback, though there was no significant difference between instructor feedback quality and the maximum student feedback quality received. Student perceptions of the utility of feedback were positive. CONCLUSION: While instructor feedback is more reliable and consistent, our findings suggest that in most instances, at least one peer in moderate-sized groups is able to approximate the quality of instructor feedback on case-based assignments.

Una revisión sistemática del uso de la inteligencia artificial en la educación / A systematic review of the use of artificial intelligence in education

Introducción. El uso de la inteligencia artificial (IA) en la educación ha sido objeto de una creciente atención en los últimos años. La IA se ha utilizado para mejorar la personalización del aprendizaje, la retroalimentación y la evaluación de los estudiantes. Sin embargo, también hay desafíos y limitaciones asociados. El objetivo de este trabajo fue identificar las principales tendencias y áreas de aplicación de la inteligencia artificial en la educación, así como analizar los beneficios y limitaciones de su uso en este ámbito. Métodos. Se llevó a cabo una revisión sistemática que exploró el empleo de la inteligencia artificial en el ámbito educativo. Esta revisión siguió una metodología de investigación basada en la búsqueda de literatura, compuesta por cinco etapas. La investigación se realizó utilizando Scopus como fuente de consulta primaria y se empleó la herramienta VOSviewer para analizar los resultados obtenidos. Resultados. Se encontraron numerosos estudios que investigan el uso de la IA en la educación. Los resultados sugieren que la IA puede mejorar significativamente la personalización del aprendizaje, proporcionando recomendaciones de actividades y retroalimentación adaptadas a las necesidades individuales de cada estudiante. Conclusiones. A pesar de las ventajas del uso de la IA en la educación, también hay desafíos y limitaciones que deben abordarse, como la calidad de los datos utilizados por la IA, la necesidad de capacitación para educadores y estudiantes, y las preocupaciones sobre la privacidad y la seguridad de los datos de los estudiantes. Es importante seguir evaluando los efectos del uso de la IA en la educación para garantizar su uso efectivo y responsable.

Introduction. The use of artificial intelligence (AI) in education has been the subject of increasing attention in recent years. AI has been used to improve personalized learning, feedback, and student assessment. However, there are also challenges and limitations. The aim of this study was to identify the main trends and areas of application of artificial intelligence in education, as well as to analyze the benefits and limitations of its use in this field. Methods. A systematic review was carried out on the use of artificial intelligence in education, using a literature search research methodology with five stages, based on the Scopus query and the tool for analyzing results with VOSviewer. Results. Numerous studies investigating the use of AI in education were found. The results suggest that AI can significantly improve personalized learning by providing activity recommendations and feedback tailored to the individual needs of each student. Conclusions. Despite the advantages of using AI in education, there are also challenges and limitations that need to be addressed, such as the quality of data used by AI, the need for training for educators and students, and concerns about the privacy and security of student data. It is important to continue evaluating the effects of AI use in education to ensure its effective and responsible use.

Aprendizaje en investigación clínica: experiencia de enseñanza híbrida con estudiantes de medicina / Learning in Clinical Research: Hybrid Teaching Experience with Medical Students

Introducción: la pandemia de COVID-19 indujo un cambio en nuestro sistema de salud y de educación. Los programas formativos también tuvieron que adaptarse y exigieron un cambio rápido. Objetivos: describir una experiencia educativa de enseñanza virtual/híbrida en investigación clínica, entre docentes del Servicio de Clínica de un hospital universitario y estudiantes de Medicina de una institución privada, que participaron del Programa ESIN (EStudiantes en INvestigación). Metodología: los contenidos y las estrategias educativas incluyeron las clases teóricas audiograbadas o videograbadas (asincrónicas y autoadministradas), el aprendizaje basado en proyectos, los talleres prácticos (encuentros sincrónicos virtuales y grupales), mediante la adopción de modelos de aprendizaje como el aula invertida, y la tutoría individual entre docente-estudiante. Los datos se recopilaron mediante la observación en contextos académicos, y basándonos en elementos de encuestas anónimas de satisfacción, previo consentimiento informado de los participantes. Resultados: participaron 14 estudiantes, 6 durante el año 2021 y 8 durante 2022. Todas mujeres y estudiantes de medicina (50% de cuarto año, 35% de sexto año y 15% de quinto año). Las técnicas implementadas favorecieron la participación y promovieron el aprendizaje activo, basado en proyectos. Mencionaron aspectos positivos como el enfoque académico práctico, la disponibilidad del equipo docente para atender cualquier duda, el tiempo y el entusiasmo por enseñar y fomentar la participación. Los videos teóricos resultaron útiles como herramientas de repaso, y los encuentros grupales fueron especialmente valorados, si bien los encuentros individuales fueron destacados como ayuda y apoyo previo a los congresos científicos. En general, manifestaron que fue una experiencia enriquecedora que demostró que se puede lograr lo que se creía imposible. Todas participaron activamente de al menos un congreso científico, y el 50% resultó coautora de una publicación académica. Conclusión: los estudiantes asumieron compromisos y responsabilidades, e incorporaron competencias y habilidades en la implementación y en la difusión de los proyectos. Esta experiencia educativa facilitó que el tiempo de clase pudiera optimizarse para intercambio, discusión y dudas. Los recursos producidos, las actividades desarrolladas y los contenidos abordados quedan disponibles a nivel institución. (AU)

Introduction: the COVID-19 pandemic brought about a change in our health and education system. Training programs also had to adapt and required rapid change. Objectives: to describe an educational experience of virtual/hybrid teaching in clinical research between teachers of the Clinical Service of a university hospital and medical students of a private institution who participated in the ESIN Program (Students in Research). Methodology: the contents and educational strategies included audio or videotaped lectures (asynchronous and self-administered), project-based learning, practical workshops (virtual and group synchronous meetings) by adopting learning models such as the inverted classroom, and individual tutoring between teacher and student. We gathered the data through observation in academic contexts and based on elements of anonymous satisfaction surveys, with prior informed consent of participants. Results: fourteen students participated, six in 2021 and eight in 2022. All were women and medical students (50% fourth year, 35% sixth year, and 15% fifth year). The techniques implemented favored participation and promoted active, project-based learning. They mentioned positive aspects such as the practical academic approach, the availability of the teaching team for any doubts, the time and enthusiasm for teaching, and encouraging participation. The theory videos were a valuable review tool, and team meetings received high praise even if the one-on-one meetings received much attention as help and support before the scientific congresses. In general, they stated that it was an enriching experience that showed that you can achieve what you thought impossible. All of them actively participated in at least one scientific congress, and 50% were co-authors of an academic publication. Conclusion: the students assumed commitments and responsibilities and incorporated competencies and skills in project implementation and dissemination. This educational experience helped to optimize class time for exchange, discussion, and doubts. The resources produced, the activities developed, and the contents addressed are now available at the institutional level. (AU)

When Feedback is Not Perceived as Feedback: Challenges for Regulatory Body-Mandated Peer Review.

PURPOSE: Safe and competent patient care depends on physicians recognizing and correcting performance deficiencies. Generating effective insight depends on feedback from credible sources. Unfortunately, physicians often have limited access to meaningful guidance. To facilitate quality improvement, many regulatory authorities have designed peer-facilitated practice enhancement programs. Their mandate to ensure practice quality, however, can create tension between formative intentions and risk (perceived or otherwise) of summative repercussions. This study explored how physicians engage with feedback when required to undergo review. METHOD: Between October 2018 and May 2020, 30 physicians representing various specialties and career stages were interviewed about their experiences with peer review in the context of regulatory body-mandated programs. Twenty had been reviewees and reviewers and, hence, spoke from both vantage points. Interview transcripts were analyzed using a 3-stage coding process informed by constructivist grounded theory. RESULTS: Perceptions about the learning value of mandated peer review were mixed. Most saw value but felt anxiety about being selected due to being wary of regulatory bodies. Recognizing barriers such perceptions could create, reviewers described techniques for optimizing the value of interactions with reviewees. Their strategies aligned well with the R2C2 feedback and coaching model with which they had been trained but did not always overcome reviewees' concerns. Reasons included that most feedback was "validating," aimed at "tweaks" rather than substantial change. CONCLUSIONS: This study establishes an intriguing and challenging paradox: feedback appears often to not be recognized as feedback when it poses no threat, yet feedback that carries such threat is known to be suboptimal for inducing performance improvement. In efforts to reconcile that tension, the authors suggest that peer review for individuals with a high likelihood of strong performance may be more effective if expectations are managed through feedforward rather than feedback.

Feedback Culture Perceived by Trainees in an Academic Institution: A Mixed Methods Study.

OBJECTIVES: Lack of a well-functioning institutional feedback culture can undermine acquisition of skills essential for high quality patient care. The objective of this study was to assess feedback culture perceived by resident and fellow trainees, utilizing a mixed methods design. METHODS: Pediatric fellows and residents completed an anonymous feedback environment survey consisting of 7 constructs: source credibility, feedback quality, feedback delivery, reinforcing feedback, constructive feedback, source availability, and promotion of feedback seeking, using a 7-point Likert scale. Trainee ratings were compared using two-sided Fisher's exact tests. Multivariable analyses used a linear regression model. For the qualitative study, semistructured interviews of residents were conducted. The constant comparative method was used to incrementally code, categorize data, and derive themes. RESULTS: Fifty-two residents and 21 fellows completed the survey (response rates 65% and 47%, respectively). Scores were more favorable for fellows compared with residents in 6 of 7 feedback constructs (P < .05), including on multivariate analysis. Hispanic ethnicity and female gender were associated with lower scores on source credibility (P = .04) and constructive feedback (P = .03), respectively. Two qualitative themes were identified: expectation of efficiency in patient care compromises the quality and quantity of feedback, and a culture that prioritizes courtesy over candor negatively impacts feedback quality. These themes were more pronounced when residents worked with pediatric subspecialists compared with hospitalists. CONCLUSIONS: We described the feedback culture, which was less favorable in the residency program. The need for efficient patient care and a culture of courtesy adversely impacted the quality of feedback, especially among subspecialists.

Application of the R2C2 Model to In-the-Moment Feedback and Coaching.

PURPOSE: The R2C2 (relationship, reaction, content, coaching) model is an iterative, evidence-based, theory-informed approach to feedback and coaching that enables preceptors and learners to build relationships, explore reactions and reflections, confirm content, and coach for change and cocreate an action plan. This study explored application of the R2C2 model for in-the-moment feedback conversations between preceptors and learners and the factors that influence its use. METHOD: A qualitative study using framework analysis through the lens of experiential learning was undertaken with 15 trained preceptor-learner dyads. Data were collected during feedback sessions and follow-up interviews between March 2021 and July 2022. The research team familiarized themselves with the data, used a coding template to document examples of the model's application, reviewed the initial framework and revised the coding template, indexed and summarized the data, created a summary document, examined the transcripts for alignment with each model phase, and identified illustrative quotations and overarching themes. RESULTS: Fifteen dyads were recruited from 8 disciplines (11 preceptors were paired with a single resident [n = 9] or a single medical student [n = 2]; 2 preceptors each had 2 residents). All dyads were able to apply the R2C2 phases of building relationships, exploring reactions and reflections, and confirming content. Many struggled with the coaching components, specifically in creating an action plan and follow-up arrangements. Preceptor skill in applying the model, time available for feedback conversations, and the nature of the relationship impacted how the model was applied. CONCLUSIONS: The R2C2 model can be adapted to contexts where in-the-moment feedback conversations occur shortly after a clinical encounter. Experiential learning approaches applying the R2C2 model are critical. Skillful application of the model requires that learners and preceptors go beyond confirming an area of change and deliberately engage in coaching and cocreating an action plan.

Unintended consequences of technology in competency-based education: a qualitative study of lessons learned in an OtoHNS program.

BACKGROUND: Formative feedback and entrustment ratings on assessments of entrustable professional activities (EPAs) are intended to support learner self-regulation and inform entrustment decisions in competency-based medical education. Technology platforms have been developed to facilitate these goals, but little is known about their effects on these new assessment practices. This study investigates how users interacted with an e-portfolio in an OtoHNS surgery program transitioning to a Canadian approach to competency-based assessment, Competence by Design. METHODS: We employed a sociomaterial perspective on technology and grounded theory methods of iterative data collection and analysis to study this OtoHNS program's use of an e-portfolio for assessment purposes. All residents (n = 14) and competency committee members (n = 7) participated in the study; data included feedback in resident portfolios, observation of use of the e-portfolio in a competency committee meeting, and a focus group with residents to explore how they used the e-portfolio and visualize interfaces that would better meet their needs. RESULTS: Use of the e-portfolio to document, access, and interpret assessment data was problematic for both residents and faculty, but the residents faced more challenges. While faculty were slowed in making entrustment decisions, formative assessments were not actionable for residents. Workarounds to these barriers resulted in a "numbers game" residents played to acquire EPAs. Themes prioritized needs for searchable, contextual, visual, and mobile aspects of technology design to support use of assessment data for resident learning. CONCLUSION: Best practices of technology design begin by understanding user needs. Insights from this study support recommendations for improved technology design centred on learner needs to provide OtoHNS residents a more formative experience of competency-based training.

Plain Language vs Standard Format for Youth Understanding of COVID-19 Recommendations: A Randomized Clinical Trial.

Importance: To ensure that youths can make informed decisions about their health, it is important that health recommendations be presented for understanding by youths. Objective: To compare understanding, accessibility, usability, satisfaction, intention to implement, and preference of youths provided with a digital plain language recommendation (PLR) format vs the original standard language version (SLV) of a health recommendation. Design, Setting, and Participants: This pragmatic, allocation-concealed, blinded, superiority randomized clinical trial included individuals from any country who were 15 to 24 years of age, had internet access, and could read and understand English. The trial was conducted from May 27 to July 6, 2022, and included a qualitative component. Interventions: An online platform was used to randomize youths in a 1:1 ratio to an optimized digital PLR or SLV format of 1 of 2 health recommendations related to the COVID-19 vaccine; youth-friendly PLRs were developed in collaboration with youth partners and advisors. Main Outcomes and Measures: The primary outcome was understanding, measured as the proportion of correct responses to 7 comprehension questions. Secondary outcomes were accessibility, usability, satisfaction, preference, and intended behavior. After completion of the survey, participants indicated their interest in completing a 1-on-1 semistructured interview to reflect on their preferred digital format (PLR or SLV) and their outcome assessment survey response. Results: Of the 268 participants included in the final analysis, 137 were in the PLR group (48.4% female) and 131 were in the SLV group (53.4% female). Most participants (233 [86.9%]) were from North and South America. No significant difference was found in understanding scores between the PLR and SLV groups (mean difference, 5.2%; 95% CI, -1.2% to 11.6%; P = .11). Participants found the PLR to be more accessible and usable (mean difference, 0.34; 95% CI, 0.05-0.63) and satisfying (mean difference, 0.39; 95% CI, 0.06-0.73) and had a stronger preference toward the PLR (mean difference, 4.8; 95% CI, 4.5-5.1 [4.0 indicated a neutral response]) compared with the SLV. No significant difference was found in intended behavior (mean difference, 0.22 (95% CI, -0.20 to 0.74). Interviewees (n = 14) agreed that the PLR was easier to understand and generated constructive feedback to further improve the digital PLR. Conclusions and Relevance: In this randomized clinical trial, compared with the SLV, the PLR did not produce statistically significant findings in terms of understanding scores. Youths ranked it higher in terms of accessibility, usability, and satisfaction, suggesting that the PLR may be preferred for communicating health recommendations to youths. The interviews provided suggestions for further improving PLR formats. Trial Registration: ClinicalTrials.gov Identifier: NCT05358990.

A qualitative study examining the critical differences in the experience of and response to formative feedback by undergraduate medical students in Japan and the UK.

BACKGROUND: Formative feedback plays a critical role in guiding learners to gain competence, serving as an opportunity for reflection and feedback on their learning progress and needs. Medical education in Japan has historically been dominated by a summative paradigm within assessment, as opposed to countries such as the UK where there are greater opportunities for formative feedback. How this difference affects students' interaction with feedback has not been studied. We aim to explore the difference in students' perception of feedback in Japan and the UK. METHODS: The study is designed and analysed with a constructivist grounded theory lens. Medical students in Japan and the UK were interviewed on the topic of formative assessment and feedback they received during clinical placements. We undertook purposeful sampling and concurrent data collection. Data analysis through open and axial coding with iterative discussion among research group members was conducted to develop a theoretical framework. RESULTS: Japanese students perceived feedback as a model answer provided by tutors which they should not critically question, which contrasted with the views of UK students. Japanese students viewed formative assessment as an opportunity to gauge whether they are achieving the pass mark, while UK students used the experience for reflective learning. CONCLUSIONS: The Japanese student experience of formative assessment and feedback supports the view that medical education and examination systems in Japan are focused on summative assessment, which operates alongside culturally derived social pressures including the expectation to correct mistakes. These findings provide new insights in supporting students to learn from formative feedback in both Japanese and UK contexts.

Medical Students' Integration of Formative Feedback During Simulation: A Grounded Theory Study.

INTRODUCTION: Formative feedback is critical for trainees' growth and development. However, there is a gap in the professional literature regarding the ways in which formative feedback affects student performance during simulation. This grounded theory study addresses this gap by exploring the ways in which medical students received and integrated ongoing formative feedback throughout a multiday, high-fidelity military medical simulation, Operation Bushmaster. MATERIALS AND METHODS: Our research team interviewed 18 fourth-year medical students in order to investigate how they processed formative feedback during the simulation. Guided by the grounded theory tradition of qualitative research, our research team used open coding and axial coding to categorize the data. We then used selective coding to determine the casual relationships between each of the categories that emerged from the data. These relationships determined our grounded theory framework. RESULTS: Four phases emerged from the data and provided a framework to delineate the process in which students received and integrated formative feedback throughout the simulation: (1) ability to self-assess, (2) self-efficacy, (3) leadership and teamwork, and (4) appreciation of feedback for personal and professional growth. The participants first focused on feedback related to their individual performance but then shifted to a teamwork and leadership mindset. Once they adapted this new mindset, they began to intentionally provide feedback to their peers, increasing their team's performance. At the end of the simulation, the participants recognized the benefits of formative feedback and peer feedback for ongoing professional development throughout their careers, signifying a growth mindset. CONCLUSIONS: This grounded theory study provided a framework for determining how medical students integrated formative feedback during a high-fidelity, multiday medical simulation. Medical educators can use this framework to intentionally guide their formative feedback in order to maximize student learning during simulation.

Student and Teacher Perspectives on Equity in Clinical Feedback: A Qualitative Study Using a Critical Race Theory Lens.

PURPOSE: The authors aimed to gain a better understanding of students' and teachers' perspectives about whether clinical clerkship feedback is provided equitably irrespective of a student's race/ethnicity. METHOD: A secondary analysis of existing interview data was conducted, focusing on racial/ethnic disparities in clinical grading. Data had been acquired from 29 students and 30 teachers at 3 U.S. medical schools. The authors performed secondary coding on all 59 transcripts, writing memos focused on statements related to aspects of feedback equity and developing a template for coding students' and teachers' observations and descriptions specific to clinical feedback. Using the template, memos were coded, and thematic categories emerged describing perspectives on clinical feedback. RESULTS: Forty-eight (22 teachers and 26 students) participants' transcripts provided narratives about feedback. Both student and teacher narratives described how students who are racially/ethnically underrepresented in medicine may receive less helpful formative clinical feedback needed for professional development. Thematic analysis of narratives yielded 3 themes related to feedback inequities: 1) teachers' racial/ethnic biases influence the feedback they provide students, 2) teachers have limited skill sets to provide equitable feedback, and 3) racial/ethnic inequities in the clinical learning environment shape clinical and feedback experiences. CONCLUSIONS: Narratives indicated that both students and teachers perceived racial/ethnic inequities in clinical feedback. Teacher- and learning environment-related factors influenced these racial/ethnic inequities. These results can inform medical education's efforts to mitigate biases in the learning environment and provide equitable feedback to ensure every student has what they need to develop into the competent physician they aspire to be.

Testing the Impact of an Asynchronous Online Training Program With Repeated Feedback.

BACKGROUND: Learning to effectively debrief with student learners can be a challenging task. Currently, there is little evidence to support the best way to train and evaluate a debriefer's competence with a particular debriefing method. PURPOSE: The purpose of this study was to develop and test an asynchronous online distributed modular training program with repeated doses of formative feedback to teach debriefers how to implement Debriefing for Meaningful Learning (DML). METHODS: Following the completion of an asynchronous distributed modular training program, debriefers self-evaluated their debriefing and submitted a recorded debriefing for expert evaluation and feedback using the DML Evaluation Scale (DMLES). RESULTS: Most debriefers were competent in DML debriefing after completing the modular training at time A, with DMLES scores increasing with each debriefing submission. CONCLUSION: The results of this study support the use of an asynchronous distributed modular training program for teaching debriefers how to implement DML.