Integrated and implicit: how residents learn CanMEDS roles by participating in practice.

CONTEXT: Learning outcomes for residency training are defined in competency frameworks such as the CanMEDS framework, which ultimately aim to better prepare residents for their future tasks. Although residents' training relies heavily on learning through participation in the workplace under the supervision of a specialist, it remains unclear how the CanMEDS framework informs practice-based learning and daily interactions between residents and supervisors. OBJECTIVES: This study aimed to explore how the CanMEDS framework informs residents' practice-based training and interactions with supervisors. METHODS: Constructivist grounded theory guided iterative data collection and analyses. Data were collected by direct observations of residents and supervisors, combined with formal and field interviews. We progressively arrived at an explanatory theory by coding and interpreting the data, building provisional theories and through continuous conversations. Data analysis drew on sensitising insights from communities of practice theory, which provided this study with a social learning perspective. RESULTS: CanMEDS roles occurred in an integrated fashion and usually remained implicit during interactions. The language of CanMEDS was not adopted in clinical practice, which seemed to impede explicit learning interactions. The CanMEDS framework seemed only one of many factors of influence in practice-based training: patient records and other documents were highly influential in daily activities and did not always correspond with CanMEDS roles. Additionally, the position of residents seemed too peripheral to allow them to learn certain aspects of the Health Advocate and Leader roles. CONCLUSIONS: The CanMEDS framework did not really guide supervisors' and residents' practice or interactions. It was not explicitly used as a common language in which to talk about resident performance and roles. Therefore, the extent to which CanMEDS actually helps improve residents' learning trajectories and conversations between residents and supervisors about residents' progress remains questionable. This study highlights the fact that the reification of competency frameworks into the complexity of practice-based learning is not a straightforward exercise.

Student distress in clinical workplace learning: differences in social comparison behaviours.

In medical education, student distress is known to hamper learning and professional development. To address this problem, recent studies aimed at helping students cope with stressful situations. Undergraduate students in clinical practice frequently use experiences of surrounding peers to estimate their abilities to master such challenging situations. This use of the experiences of others, known as social comparison, may affect student distress both positively and negatively. To find characteristics of a beneficial use of social comparison, we examined differences in comparison behaviours between students expressing low and high levels of distress. The participants in our study, response rate 93% (N = 301/321), were all medical students in their first year in clinical practice. They completed the General Health Questionnaire (GHQ-12) to measure distress, and three separate questionnaires to measure: (1) orientation to comparison, (2) motive for comparison, and (3) interpretation of comparison. Differences were analysed using multivariate analysis of variance. Although all students were oriented towards social comparison, the analyses showed that this orientation was less apparent among low-distress students. Besides, the low-distress students were less inclined to use motives indicative for comparisons with peers perceived as performing worse and were less negative in the interpretations of their comparisons. As social comparison is frequently used among all students, we recommend to make them aware of their comparison behaviours and inform them about the pros and cons of the distinguished aspects of the comparison process.

Peer influence on students' estimates of performance: social comparison in clinical rotations.

CONTEXT: During clinical rotations, students move from one clinical situation to another. Questions exist about students' strategies for coping with these transitions. These strategies may include a process of social comparison because in this context it offers the student an opportunity to estimate his or her abilities to master a novel rotation. These estimates are relevant for learning and performance because they are related to self-efficacy. We investigated whether student estimates of their own future performance are influenced by the performance level and gender of the peer with whom the student compares him- or herself. METHODS: We designed an experimental study in which participating students (n = 321) were divided into groups assigned to 12 different conditions. Each condition entailed a written comparison situation in which a peer student had completed the rotation the participant was required to undertake next. Differences between conditions were determined by the performance level (worse, similar or better) and gender of the comparison peer. The overall grade achieved by the comparison peer remained the same in all conditions. We asked participants to estimate their own future performance in that novel rotation. Differences between their estimates were analysed using analysis of variance (ANOVA). RESULTS: Students' estimates of their future performance were highest when the comparison peer was presented as performing less well and lowest when the comparison peer was presented as performing better (p < 0.001). Estimates of male and female students in same-gender comparison conditions did not differ. In two of three opposite-gender conditions, male students' estimates were higher than those of females (p < 0.001 and p < 0.05, respectively). CONCLUSIONS: Social comparison influences students' estimates of their future performance in a novel rotation. The effect depends on the performance level and gender of the comparison peer. This indicates that comparisons against particular peers may strengthen or diminish a student's self-efficacy, which, in turn, may ease or hamper the student's learning during clinical rotations. The study is limited by its experimental design. Future research should focus on students' comparison behaviour in real transitions.

The reliability of in-training assessment when performance improvement is taken into account.

During in-training assessment students are frequently assessed over a longer period of time and therefore it can be expected that their performance will improve. We studied whether there really is a measurable performance improvement when students are assessed over an extended period of time and how this improvement affects the reliability of the overall judgement. In-training assessment results were obtained from 104 students on rotation at our university hospital or at one of the six affiliated hospitals. Generalisability theory was used in combination with multilevel analysis to obtain reliability coefficients and to estimate the number of assessments needed for reliable overall judgement, both including and excluding performance improvement. Students' clinical performance ratings improved significantly from a mean of 7.6 at the start to a mean of 7.8 at the end of their clerkship. When taking performance improvement into account, reliability coefficients were higher. The number of assessments needed to achieve a reliability of 0.80 or higher decreased from 17 to 11. Therefore, when studying reliability of in-training assessment, performance improvement should be considered.

Learning strategies during clerkships and their effects on clinical performance.

BACKGROUND: Previous research revealed relationships between learning strategies and knowledge acquisition. During clerkships, however, students' focus widens beyond mere knowledge acquisition as they further develop overall competence. This shift in focus can influence learning strategy use. AIM: We explored which learning strategies were used during clerkships and their relationship to clinical performance. METHODS: Participants were 113 (78%) clerks at the university hospital or one of six affiliated hospitals. Learning strategies were assessed using the 'Approaches to Learning at Work Questionnaire' (deep, surface-rational and surface-disorganised learning). Clinical performance was calculated by taking the mean of clinical assessment marks. The relationship between learning strategies and clinical performance was explored using regression analysis. RESULTS: Most students (89%) did not clearly prefer a single learning strategy. No relationship was found between learning strategies and clinical performance. DISCUSSION: Since overall competence comprises integration of knowledge, skills and professional behaviour, we assume that students without a clear preference use more than one learning strategy. Finding no relationship between learning strategies and clinical performance reflects the complexity of clinical learning. Depending on circumstances it may be important to obtain relevant information quickly (surface-rational) or understand material thoroughly (deep). In future research we will examine when and why students use different learning strategies.

Instructiveness of feedback during clerkships: influence of supervisor, observation and student initiative.

BACKGROUND: Several authors assume that the supervisor's role, observation of behaviour and students' active participation are important factors in the instructiveness of feedback. AIM: This study aims to provide empirical evidence for these expectations. METHODS: For two weeks, 142 clerks from eight hospitals recorded for each individual feedback event: who provided the feedback, whether the feedback was based on observation of behaviour, who initiated the feedback moment and the perceived instructiveness of the feedback. Data were analysed with multilevel techniques. RESULTS: The perceived instructiveness of feedback provided by specialists and residents did not differ significantly. However, both were perceived to be more instructive than feedback from nursing and paramedical staff (beta(specialists) = 0.862, p < 0.01; beta(residents) = 0.853, p < 0.01). Feedback on behaviour that had been directly observed was reported to be more instructive than feedback on behaviour that had not been observed (beta(observed) = 0.314, p < 0.001). Feedback which stemmed from student initiative or a joint initiative was experienced to be more instructive than feedback which ensued from the supervisor's initiative (beta(student) = 0.441, p < 0.01; beta(joint) = 0.392, p < 0.01). CONCLUSIONS: The expectations concerning the influence of observation and student initiative on the instructiveness of feedback were confirmed in this empirical study. Expected differences in instructiveness between feedback from specialists and residents were not confirmed.

Development and Validation of the Scan of Postgraduate Educational Environment Domains (SPEED): A Brief Instrument to Assess the Educational Environment in Postgraduate Medical Education.

INTRODUCTION: Current instruments to evaluate the postgraduate medical educational environment lack theoretical frameworks and are relatively long, which may reduce response rates. We aimed to develop and validate a brief instrument that, based on a solid theoretical framework for educational environments, solicits resident feedback to screen the postgraduate medical educational environment quality. METHODS: Stepwise, we developed a screening instrument, using existing instruments to assess educational environment quality and adopting a theoretical framework that defines three educational environment domains: content, atmosphere and organization. First, items from relevant existing instruments were collected and, after deleting duplicates and items not specifically addressing educational environment, grouped into the three domains. In a Delphi procedure, the item list was reduced to a set of items considered most important and comprehensively covering the three domains. These items were triangulated against the results of semi-structured interviews with 26 residents from three teaching hospitals to achieve face validity. This draft version of the Scan of Postgraduate Educational Environment Domains (SPEED) was administered to residents in a general and university hospital and further reduced and validated based on the data collected. RESULTS: Two hundred twenty-three residents completed the 43-item draft SPEED. We used half of the dataset for item reduction, and the other half for validating the resulting SPEED (15 items, 5 per domain). Internal consistencies were high. Correlations between domain scores in the draft and brief versions of SPEED were high (>0.85) and highly significant (p<0.001). Domain score variance of the draft instrument was explained for ≥80% by the items representing the domains in the final SPEED. CONCLUSIONS: The SPEED comprehensively covers the three educational environment domains defined in the theoretical framework. Because of its validity and brevity, the SPEED is promising as useful and easily applicable tool to regularly screen educational environment quality in postgraduate medical education.