Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review.

BACKGROUND: Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback. METHODS: A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework. RESULTS: The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes. CONCLUSIONS: This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

The Balancing Act of Assessment Validity in Interprofessional Healthcare Education: A Qualitative Evaluation Study.

CONSTRUCT & BACKGROUND: In order to determine students' level of interprofessional competencies, there is a need for well-considered and thoroughly designed interprofessional assessments. Current literature about interprofessional assessments focuses largely on the development and validation of assessment instruments such as self-assessments or questionnaires to assess students' knowledge or attitudes. Less is known about the design and validity of integral types of assessment in interprofessional education, such as case-based assessments, or performance assessments. The aim of this study is to evaluate the evidence for and threats to the validity of the decisions about students' interprofessional performances based on such integral assessment task. We investigated whether the assessment prototype is a precursor to practice (authenticity) and whether the assessment provides valid information to determine the level of interprofessional competence (scoring). APPROACH: We used a design-based qualitative research design in which we conducted three group interviews with teachers, students, and interprofessional assessment experts. In semi-structured group interviews, participants evaluated the evidence for and threats to the validity of an interprofessional assessment task, which were analyzed using deductive and inductive content analysis. FINDINGS: Although both evidence for and threats to validity were mentioned, the threats refuting the assessment's validity prevailed. Evidence for the authenticity aspect was that the assessment task, conducting a team meeting, is common in practice. However, its validity was questioned because the assessment task appeared more structured as compared to practice. The most frequently mentioned threat to the scoring aspect was that the process of interprofessional collaboration between the students could not be evaluated sufficiently by means of this assessment task. CONCLUSIONS: This study showed that establishing interprofessional assessment validity requires three major balancing acts. The first is the balance between authenticity and complexity. As interprofessional practice and competencies are complex, interprofessional tasks require build-up or guidance toward this complexity and chaotic practice. The second is that between authenticity and scoring, in which optimal authenticity might lead to threats to scoring and vice versa. Simultaneous optimal authenticity and scoring seems impossible, requiring ongoing evaluation and monitoring of interprofessional assessment validity to ensure authentic yet fair assessments for all participating professions. The third balancing act is between team scoring and individual scoring. As interprofessional practice requires collaboration and synthesis of diverse professions, the team process is at the heart of solving interprofessional tasks. However, to stimulate individual accountability, the individual performance should not be neglected.

Putting self-regulated learning in context: Integrating self-, co-, and socially shared regulation of learning.

Processes involved in the regulation of learning have been researched for decades, because of its impact on academic and workplace performance. In fact, self-regulated learning is the focus of countless studies in health professions education and higher education in general. While we will always need competent individuals who are able to regulate their own learning, developments in healthcare require a shift from a focus on the individual to the collective: collaboration within and between healthcare teams is at the heart of high-quality patient care. Concepts of collaborative learning and collective competence challenge commonly held conceptualisations of regulatory learning and call for a focus on the social embeddedness of regulatory learning and processes regulating the learning of the collective. Therefore, this article questions the alignment of current conceptualisations of regulation of learning with demands for collaboration in current healthcare. We explore different conceptualisations of regulation of learning (self-, co-, and socially shared regulation of learning), and elaborate on how the integration of these conceptualisations adds to our understanding of regulatory learning in healthcare settings. Building on these insights, we furthermore suggest ways forward for research and educational practice.

How medical students co-regulate their learning in clinical clerkships: a social network study.

BACKGROUND: Self-regulated learning is a key competence to engage in lifelong learning. Research increasingly acknowledges that medical students in clerkships need others to regulate their learning. The concept of "co-regulated learning" captures this act of regulating one's learning by interacting with others. To effectively cultivate such skills in students, we need to increase our understanding of co-regulated learning. This study aimed to identify the purposes for which students in different phases of clinical training engage others in their networks to regulate their learning. METHODS: In this social network study, we administered a questionnaire to 403 medical students during clinical clerkships (65.5% response rate). The questionnaire probed into the composition of students' co-regulatory networks and the purpose for which they engaged others in specified self-regulated learning activities. We calculated the proportion of students that engaged others in their networks for each regulatory activity. Additionally, we conducted ANOVAs to examine whether first-, second-, and third-year students differed in how they used their networks to support self-regulation. RESULTS: Students used others within their co-regulatory networks to support a range of self-regulated learning activities. Whom students engaged, and the purpose of engagement, seemed to shift as students progressed through clinical training. Over time, the proportion of students engaging workplace supervisors to discuss learning goals, learning strategies, self-reflections and self-evaluations increased, whereas the proportion of students engaging peers to discuss learning strategies and how to work on learning goals in the workplace decreased. Of all purposes for which students engaged others measured, discussing self-reflections and self-evaluations were consistently among the ones most frequently mentioned. CONCLUSIONS: Results reinforce the notion that medical students' regulation of learning is grounded in social interactions within co-regulatory networks students construct during clerkships. Findings elucidate the extent to which students enact self-regulatory learning within their co-regulatory networks and how their co-regulatory learning behaviors develop over time. Explicating the relevance of interactions within co-regulatory networks might help students and supervisors to purposefully engage in meaningful co-regulatory interactions. Additionally, co-regulatory interactions may assist students in regulating their learning in clinical workplaces as well as in honing their self-regulated learning skills.

Beyond the self: The role of co-regulation in medical students' self-regulated learning.

CONTEXT: Medical students are expected to self-regulate their learning within complex and unpredictable clinical learning environments. Research increasingly focuses on the effects of social interactions on the development of self-regulation in workplace settings, a notion embodied within the concept of co-regulated learning (CRL). Creating workplace learning environments that effectively foster lifelong self-regulated learning (SRL) requires a deeper understanding of the relationship between CRL and SRL. The aim of this study was therefore to explore medical students' perceptions of CRL in clinical clerkships and its perceived impact on the development of their SRL. METHODS: We conducted semi-structured interviews with 11 purposively sampled medical students enrolled in clinical clerkships at one undergraduate competency-based medical school. Data collection and analysis were conducted iteratively, informed by principles of constructivist grounded theory. Data analysis followed stages of open, axial and selective coding, which enabled us to conceptualise how co-regulation influences the development of students' self-regulation. RESULTS: Data revealed three interrelated shifts in CRL and SRL as students progressed through clerkships. First, students' CRL shifted from a focus on peers to co-regulation with clinician role models. Second, self-regulated behaviour shifted from being externally driven to being internally driven. Last, self-regulation shifted from a task-oriented approach towards a more comprehensive approach focusing on professional competence and identity formation. Students indicated that if they felt able to confidently and proactively self-regulate their learning, the threshold for engaging others in meaningful CRL seemed to be lowered, enhancing further development of SRL skills. CONCLUSIONS: Findings from the current study emphasise the notion that SRL and its development are grounded in CRL in clinical settings. To optimally support the development of students' SRL, we need to focus on facilitating and organising learners' engagement in CRL from the start of the medical curriculum.

Toward a better judgment of item relevance in progress testing.

BACKGROUND: Items must be relevant to ensure item quality and test validity. Since "item relevance" has not been operationalized yet, we developed a rubric to define it. This study explores the influence of this rubric on the assessment of item relevance and on inter-rater agreement. METHODS: Members of the item review committee (RC) and students, teachers, and alumni (STA) reassessed the relevance of 50 previously used progress test (PT) items and decided about their inclusion using a 5-criteria rubric. Data were analyzed at item level using paired samples t-tests, Intraclass Correlation Coefficients (ICC), and linear regression analysis, and at rater level in a generalizability analysis per group. RESULTS: The proportion of items that the RC judged relevant enough to be included decreased substantially from 1.00 to 0.72 (p < 0.001). Agreement between the RC and STA was high, with an ICC of >0.7 across items. The relation between inclusion and relevance was strong (correlation = 0.89, p < 0.001), and did not differ between RC and STA. To achieve an acceptable inter-rater reliability for relevance and inclusion, 6 members must serve on the RC. CONCLUSIONS: Use of the rubric results in a stricter evaluation of items' appropriateness for inclusion in the PT and facilitates agreement between the RC and other stakeholders. Hence, it may help increase the acceptability and validity of the PT.

Why peer assessment helps to improve clinical performance in undergraduate physical therapy education: a mixed methods design.

BACKGROUND: Peer Assessment (PA) in health professions education encourages students to develop a critical attitude towards their own and their peers' performance. We designed a PA task to assess students' clinical skills (including reasoning, communication, physical examination and treatment skills) in a role-play that simulated physical therapy (PT) practice. Students alternately performed in the role of PT, assessor, and patient. Oral face-to-face feedback was provided as well as written feedback and scores.This study aims to explore the impact of PA on the improvement of clinical performance of undergraduate PT students. METHODS: The PA task was analyzed and decomposed into task elements. A qualitative approach was used to explore students' perceptions of the task and the task elements. Semi-structured interviews with second year students were conducted to explore the perceived impact of these task elements on performance improvement. Students were asked to select the elements perceived valuable, to rank them from highest to lowest learning value, and to motivate their choices. Interviews were transcribed verbatim and analyzed, using a phenomenographical approach and following template analysis guidelines. A quantitative approach was used to describe the ranking results. RESULTS: Quantitative analyses showed that the perceived impact on learning varied widely. Performing the clinical task in the PT role, was assigned to the first place (1), followed by receiving expert feedback (2), and observing peer performance (3). Receiving peer feedback was not perceived the most powerful task element.Qualitative analyses resulted in three emerging themes: pre-performance, true-performance, and post-performance triggers for improvement. Each theme contained three categories: learning activities, outcomes, and conditions for learning.Intended learning activities were reported, such as transferring prior learning to a new application context and unintended learning activities, such as modelling a peer's performance. Outcomes related to increased self-confidence, insight in performance standards and awareness of improvement areas. Conditions for learning referred to the quality of peer feedback. CONCLUSIONS: PA may be a powerful tool to improve clinical performance, although peer feedback is not perceived the most powerful element. Peer assessors in undergraduate PT education use idiosyncratic strategies to assess their peers' performance.

Design guidelines for assessing students' interprofessional competencies in healthcare education: a consensus study.

INTRODUCTION: Healthcare systems require healthcare professionals and students educated in an interprofessional (IP) context. Well-designed assessments are needed to evaluate whether students have developed IP competencies, but we currently lack evidence-informed guidelines to create them. This study aims to provide guidelines for the assessment of IP competencies in healthcare education. METHODS: A qualitative consensus study was conducted to establish guidelines for the design of IP assessments using the nominal group technique. First, five expert groups (IP experts, patients, educational scientists, teachers, and students) were asked to discuss design guidelines for IP assessment and reach intra-group consensus. Second, one heterogeneous inter-group meeting was organized to reach a consensus among the expert groups on IP assessment guidelines. RESULTS: This study yielded a comprehensive set of 26 guidelines to help design performance assessments for IP education: ten guidelines for both the IP assessment tasks and the IP assessors and six guidelines for the IP assessment procedures. DISCUSSION: The results showed that IP assessment is complex and, compared to mono-professional assessment, high-quality IP assessments require additional elements such as multiple IP products and processes to be assessed, an IP pool of assessors, and assessment procedures in which standards are included for the IP collaboration process as well as individual contributions. The guidelines are based on expert knowledge and experience, but an important next step is to test these design guidelines in educational practice.

Relationships between medical students' co-regulatory network characteristics and self-regulated learning: a social network study.

INTRODUCTION: Recent conceptualizations of self-regulated learning acknowledge the importance of co-regulation, i.e., students' interactions with others in their networks to support self-regulation. Using a social network approach, the aim of this study is to explore relationships between characteristics of medical students' co-regulatory networks, perceived learning opportunities, and self-regulated learning. METHODS: The authors surveyed 403 undergraduate medical students during their clinical clerkships (response rate 65.5%). Using multiple regression analysis, structural equation modelling techniques, and analysis of variance, the authors explored relationships between co-regulatory network characteristics (network size, network diversity, and interaction frequency), students' perceptions of learning opportunities in the workplace setting, and self-reported self-regulated learning. RESULTS: Across all clerkships, data showed positive relationships between tie strength and self-regulated learning (ß = 0.095, p < 0.05) and between network size and tie strength (ß = 0.530, p < 0.001), and a negative relationship between network diversity and tie strength (ß = -0.474, p < 0.001). Students' perceptions of learning opportunities showed positive relationships with both self-regulated learning (ß = 0.295, p < 0.001) and co-regulatory network size (ß = 0.134, p < 0.01). Characteristics of clerkship contexts influenced both co-regulatory network characteristics (size and tie strength) and relationships between network characteristics, self-regulated learning, and students' perceptions of learning opportunities. DISCUSSION: The present study reinforces the importance of co-regulatory networks for medical students' self-regulated learning during clinical clerkships. Findings imply that supporting development of strong networks aimed at frequent co-regulatory interactions may enhance medical students' self-regulated learning in challenging clinical learning environments. Social network approaches offer promising ways of further understanding and conceptualising self- and co-regulated learning in clinical workplaces.

Feedback for general practitioners in training: quality, styles, and preferences.

A General Practitioner (GP) is no longer a loner, but a team player in either a group practice or a care centre. This change has led to a concomitant growth in curricular interest in skills essential for successful collaboration and for enhancing critical reflection towards colleagues' performance. Giving and receiving constructive feedback are examples of these skills. The aim of this study was to gain insight in the style and quality of feedback reports on consultation skills written by GPs-in-Training (GPiTs) and by their GP-trainers. Furthermore, the preferences of the GPiTs concerning feedback style were examined. Results show significant differences between GP-trainers and GPiTs in feedback style and quality. A ranking task indicated that GPiTs have a preference for reports characterised by a large number of reflective remarks. Questionnaire results indicate the added value of the use of peer feedback. Implications for the integration of peer feedback activities in the curriculum of GPiTs are discussed.