Impact of familiarity with the format of the exam on performance in the OSCE of undergraduate medical students - an interventional study.

BACKGROUND: Assessments, such as summative structured examinations, aim to verify whether students have acquired the necessary competencies. It is important to familiarize students with the examination format prior to the assessment to ensure that true competency is measured. However, it is unclear whether students can demonstrate their true potential or possibly perform less effectively due to the unfamiliar examination format. Hence, we questioned whether a 10-min active familiarization in the form of simulation improved medical students´ OSCE performance. Next, we wanted to elucidate whether the effect depends on whether the familiarization procedure is active or passive. METHODS: We implemented an intervention consisting of a 10-min active simulation to prepare the students for the OSCE setting. We compared the impact of this intervention on performance to no intervention in 5th-year medical students (n = 1284) from 2018 until 2022. Recently, a passive lecture, in which the OSCE setting is explained without active participation of the students, was introduced as a comparator group. Students who participated in neither the intervention nor the passive lecture group formed the control group. The OSCE performance between the groups and the impact of gender was assessed using X2, nonparametric tests and regression analysis (total n = 362). RESULTS: We found that active familiarization of students (n = 188) yields significantly better performance compared to the passive comparator (Cohen´s d = 0.857, p < 0.001, n = 52) and control group (Cohen´s d = 0.473, p < 0.001, n = 122). In multivariate regression analysis, active intervention remained the only significant variable with a 2.945-fold increase in the probability of passing the exam (p = 0.018). CONCLUSIONS: A short 10-min active intervention to familiarize students with the OSCE setting significantly improved student performance. We suggest that curricula should include simulations on the exam setting in addition to courses that increase knowledge or skills to mitigate the negative effect of nonfamiliarity with the OSCE exam setting on the students.

Entrustment versus performance scale in high-stakes OSCEs: Rater insights and psychometric properties.

BACKGROUND: Although entrustment scales are increasingly applied in workplace-based assessments, their role in OSCEs remains unclear. We investigated raters' perceptions using an entrustment scale and psychometric analyses. METHOD: A mixed-methods design was used. OSCE raters' (n = 162) perceptions were explored via questionnaire and four focus groups (n = 14). Psychometric OSCE properties were analyzed statistically. RESULTS: Raters (n = 53, response rate = 41%) considered the entrustment scale comprehensible (89%) and applicable (60%). A total of 43% preferred the entrustment scale, 21% preferred the global performance scale, and 36% were undecided. Raters' written comments indicated that while they appreciated the authenticity of entrustment levels, they considered them subjective. The focus groups highlighted three main themes: (1) recollections of the clinical workplace as a cognitive reference triggered by entrustment scales; (2) factors influencing entrustment decisions; and (3) cognitive load is reduced at the perceived cost of objectivity. Psychometric analyses (n = 480 students) revealed improvements in some OSCE metrics when entrustment and global performance scales were combined. CONCLUSION: Entrustment scales are beneficial for high-stakes OSCEs and have greater clinical relevance from the raters' perspective. Our findings support the use of entrustment and global performance scales in combination.

Challenges in the transition from resident to attending physician in general internal medicine: a multicenter qualitative study.

BACKGROUND: The attending physician in general internal medicine (GIM) guarantees comprehensive care for persons with complex and/or multiple diseases. Attendings from other medical specialties often report that transitioning from resident to attending is burdensome and stressful. We set out to identify the specific challenges of newly appointed attendings in GIM and identify measures that help residents better prepare to meet these challenges. METHODS: We explored the perceptions of 35 residents, attendings, and department heads in GIM through focus group discussions and semi-structured interviews. We took a thematic approach to qualitatively analyze this data. RESULTS: Our analysis revealed four key challenges: 1) Embracing a holistic, patient centered perspective in a multidisciplinary environment; 2) Decision making under conditions of uncertainty; 3) Balancing the need for patient safety with the need to foster a learning environment for residents; and 4) Taking on a leader's role and orchestrating an interprofessional team of health care professionals. Newly appointed attendings required extensive practical experience to adapt to their new roles. Most attendings did not receive regular, structured, professional coaching during their transition, but those who did found it very helpful. CONCLUSIONS: Newly appointed attending physician in GIM face a number of critical challenges that are in part specific to the field of GIM. Further studies should investigate whether the availability of a mentor as well as conscious assignment of a series of increasingly complex tasks during residency by clinical supervisors will facilitate the transition from resident to attending.

The introduction of a standardised national licensing exam as a driver of change in medical education: A qualitative study from Switzerland.

INTRODUCTION: Only a few studies have described the impacts, strengths and needs for further development of national licensing exams (NLE). To gain such insights regarding the Swiss NLE, which includes a multiple-choice and a standardised clinical skills exam, we explored the perceptions of involved experts and stakeholders. METHODS: We explored participants' perceptions in four focus group discussions. The interviews were recorded, transcribed verbatim and qualitatively analysed using a thematic analysis approach. RESULTS: The analysis resulted in five perceived impacts, two strengths and two needs for further developments of the NLE. Perceived impacts were (1) steering students' learning behaviour, (2) supporting teachers and assessors to align teaching to competencies, (3) elevating the importance of the Swiss Catalogue of Learning Objectives, (4) setting incentives for the further development of curricula, and (5) fostering the collaboration between the faculties of medicine. Perceived strengths were the blend of assessment formats, including their competency-based orientation, and the collaborative development approach. Perceived needs lay in the NLE's further development to sustain its fit for purpose and in incentives for people involved. CONCLUSION: According to our study, this NLE had, and has, notable impacts on medical education in Switzerland. Our insights can be useful for others planning a similar undertaking.

Simulated patient-based teaching of medical students improves pre-anaesthetic assessment: A rater-blinded randomised controlled trial.

BACKGROUND: Pre-anaesthetic assessment of patients is a complex competency that needs to be taught during anaesthesia clerkships. OBJECTIVES: We aimed to improve student teaching and investigated the effectiveness of trained 'simulated patients' (lay persons or actors trained to portray specific roles or symptoms) in the teaching of medical students to perform routine pre-anaesthetic assessments. We hypothesised that the intervention of one 30-min teaching sequence with a simulated patient will improve the performance of year 4 medical students in pre-anaesthesia assessment of elective surgical patients, compared with the control of standard apprentice-based teaching. DESIGN: Randomised controlled trial. SETTING/PARTICIPANTS: One hundred and forty-four year 4 medical students at the University of Bern. INTERVENTION: These students were randomised to either the standard clinician-supervised learning in the operating theatre (n=71; control group) or a single teaching session with a simulated patient (nonhealthcare provider, as a trained layperson) (n=73; intervention group). Both groups of students then performed pre-anaesthetic patient visits. The student performances during these visits were assessed according to the mini-Clinical Evaluation Exercise tool by trained anaesthesiologists blinded to randomisation. The 71 students in the standard clinical supervision group also underwent the simulated patient teaching session on the day following the assessments. RESULTS: The students in the intervention group of simulated patient teaching scored significantly higher in both their mini-Clinical Evaluation Exercise overall impression scores (8.8 ±â€Š0.8 vs. 8.3 ±â€Š0.9; P = 0.004) and mean domain scores (8.7 ±â€Š0.8 vs. 8.3 ±â€Š0.9; P = 0.01), compared with those of the control group with the standard clinical supervision. CONCLUSION: The current single teaching encounter with a trained layperson acting as a simulated patient improved medical student performances in their pre-anaesthetic clinical assessment of surgical patients. This might be a suitable alternative to reduce the teaching burden for busy and costly clinicians.

Identity matters - perceptions of inter-professional feedback in the context of workplace-based assessment in Diabetology training: a qualitative study.

BACKGROUND: Inter-professional collaboration is acknowledged as essential for quality patient-care. However, little is known about receptiveness to inter-professional feedback in the postgraduate training. This study explores, in light of social identity theory, the perceptions of residents, supervising physicians and allied health care professionals regarding inter-professional feedback in the context of workplace-based assessment. METHODS: For 6 months, residents in Diabetology at the University Hospital of Bern performed formative workplace-based assessments under direct observation of a supervising physician and an allied health care professional. Feedback from both observers was given to the resident after every assessment. Subsequently, focus group discussions were conducted to collect the participants' perceptions of inter- and intra-professional feedback. Transcripts were analyzed qualitatively using a thematic analysis approach. RESULTS: We identified four main themes: (1) Identity and hierarchy; (2) Interdependence of feedback source and feedback content; (3) Impact on collaboration and patient-care; (4) Logistical and organizational requirements. While different social identities are the source of inter-professional hierarchies, they did not impede the receptiveness to feedback. Perceived trustworthiness of the feedback was attributed with more importance than professional affiliations, whereas intra-professional hierarchies between physicians led to the perception of a more summative nature of the feedback and rather impeded receptiveness. According to the participants, inter-professional feedback raised awareness of the working reality of other team members and had a positive impact on communication between the different professional groups. Moreover, participants reported positive response from patients regarding the inter-professional collaboration they experienced. Considerable organizational effort is required to enable the parallel observation of a resident's consultation by a supervising physician and an allied health care professional. CONCLUSIONS: Feedback from allied health care professionals can be a valuable learning resource for residents, given its role outside the sometimes conflicting area of intra-professional hierarchies. Inter-professional feedback in the context of workplace-based assessment carries the potential to strengthen collaboration between the different professional groups.

A german-language competency-based multisource feedback instrument for residents: development and validity evidence.

BACKGROUND: In medical settings, multisource feedback (MSF) is a recognised method of formative assessment. It collects feedback on a doctor's performance from several perspectives in the form of questionnaires. Yet, no validated MSF questionnaire has been publicly available in German. Thus, we aimed to develop a German MSF questionnaire based on the CanMEDS roles and to investigate the evidence of its validity. METHODS: We developed a competency-based MSF questionnaire in German, informed by the literature and expert input. Four sources of validity evidence were investigated: (i) Content was examined based on MSF literature, blueprints of competency, and expert-team discussions. (ii) The response process was supported by analysis of a think-aloud study, narrative comments, "unable to comment" ratings and evaluation data. (iii) The internal structure was assessed by exploratory factor analysis, and inter-rater reliability by generalisability analysis. Data were collected during two runs of MSF, in which 47 residents were evaluated once (first run) or several times (second and third run) on 81 occasions of MSF. (iv) To investigate consequences, we analysed the residents' learning goals and the progress as reported via MSF. RESULTS: Our resulting MSF questionnaire (MSF-RG) consists of 15 items and one global rating, which are each rated on a scale and accompanied by a field for narrative comments and cover a construct of a physician's competence. Additionally, there are five open questions for further suggestions. Investigation of validity evidence revealed that: (i) The expert group agreed that the content comprehensively addresses clinical competence; (ii) The response processes indicated that the questions are understood as intended and supported the acceptance and usability; (iii) For the second run, factor analysis showed a one-factor solution, a Cronbach's alpha of 0.951 and an inter-rater reliability of 0.797 with 12 raters; (iv) There are indications that residents benefitted, considering their individual learning goals and based on their ratings reported via MSF itself. CONCLUSIONS: To support residency training with multisource feedback, we developed a German MSF questionnaire (MSF-RG), which is supported by four sources of validity evidence. This MSF questionnaire may be useful to implement MSF in residency training in German-speaking regions.

Factors influencing the educational impact of Mini-CEX and DOPS: A qualitative synthesis.

INTRODUCTION: The educational impact of Mini-CEX and DOPS varies greatly and can be influenced by several factors. However, there is no comprehensive analysis and synthesis of the described influencing factors. METHODS: To fill this gap, we chose a two-step approach. First, we performed a systematic literature review and selected articles describing influencing factors on the educational impact of Mini-CEX and DOPS. Second, we performed a qualitative synthesis of these factors. RESULTS: Twelve articles were included, which revealed a model consisting of four themes and nine subthemes as influencing factors. The theme context comprises "time for Mini-CEX/DOPS" and "usability of the tools", and influences the users. The theme users comprises "supervisors' knowledge about how to use Mini-CEX/DOPS", "supervisors' attitude to Mini-CEX/DOPS", "trainees' knowledge about Mini-CEX/DOPS", and "trainees' perception of Mini-CEX/DOPS". These influence the implementation of Mini-CEX and DOPS, including "observation" and "feedback". The theme implementation directly influences the theme outcome, which, in addition to the educational impact, encompasses "trainees' appraisal of feedback". CONCLUSIONS: Our model of influencing factors might help to further improve the use of Mini-CEX and DOPS and serve as basis for future research.

Variability and dimensionality of students' and supervisors' mini-CEX scores in undergraduate medical clerkships - a multilevel factor analysis.

BACKGROUND: The mini clinical evaluation exercise (mini-CEX)-a tool used to assess student-patient encounters-is increasingly being applied as a learning device to foster clinical competencies. Although the importance of eliciting self-assessment for learning is widely acknowledged, little is known about the validity of self-assessed mini-CEX scores. The aims of this study were (1) to explore the variability of medical students' self-assessed mini-CEX scores, and to compare them with the scores obtained from their clinical supervisors, and (2) to ascertain whether learners' self-assessed mini-CEX scores represent a global dimension of clinical competence or discrete clinical skills. METHODS: In year 4, medical students conducted one to three mini-CEX per clerkship in gynaecology, internal medicine, paediatrics, psychiatry and surgery. Students and clinical supervisors rated the students' performance on a 10-point scale (1 = great need for improvement; 10 = little need for improvement) in the six domains history taking, physical examination, counselling, clinical judgement, organisation/efficiency and professionalism as well as in overall performance. Correlations between students' self-ratings and ratings from clinical supervisors were calculated (Pearson's correlation coefficient) based on averaged scores per domain and overall. To investigate the dimensionality of the mini-CEX domain scores, we performed factor analyses using linear mixed models that accounted for the multilevel structure of the data. RESULTS: A total of 1773 mini-CEX from 164 students were analysed. Mean scores for the six domains ranged from 7.5 to 8.3 (student ratings) and from 8.8 to 9.3 (supervisor ratings). Correlations between the ratings of students and supervisors for the different domains varied between r = 0.29 and 0.51 (all p < 0.0001). Mini-CEX domain scores revealed a single-factor solution for both students' and supervisors' ratings, with high loadings of all six domains between 0.58 and 0.83 (students) and 0.58 and 0.84 (supervisors). CONCLUSIONS: These findings put a question mark on the validity of mini-CEX domain scores for formative purposes, as neither the scores obtained from students nor those obtained from clinical supervisors unravelled specific strengths and weaknesses of individual students' clinical competence.

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills--a controlled study.

BACKGROUND: The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS: All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS: At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS: Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships--a multilevel analysis.

BACKGROUND: In contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students' mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics. METHODS: Medical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students' mean scores of 9 clinical skills OSCE stations and mean 'overall' and 'domain' mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments. RESULTS: A total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87%). Most influential for the prediction of 'overall' mini-CEX scores was the trainers' clinical position with a regression coefficient of 0.55 (95%-CI: 0.26-0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced 'overall' mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase 'overall' mini-CEX scores. CONCLUSION: In our study, Mini-CEX scores depended rather on context characteristics than on students' clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores' validity or to use narrative comments only.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES: The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer-student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS: Retrospective analyses of adapted mini-CEX forms (trainers' and students' assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS: A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains of 'clinical reasoning' (23% of 1167 complete forms), 'organisation/efficiency' (20%) and 'physical examination' (20%). At least one 'defined' LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were 'physical examination' (49%) and 'history taking' (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS: Although trainer-student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students' LNs raise questions about how the full potential of the mini-CEX as not only a 'diagnostic' but also an 'educational' tool can be exploited.

Expertise in performance assessment: assessors' perspectives.

The recent rise of interest among the medical education community in individual faculty making subjective judgments about medical trainee performance appears to be directly related to the introduction of notions of integrated competency-based education and assessment for learning. Although it is known that assessor expertise plays an important role in performance assessment, the roles played by different factors remain to be unraveled. We therefore conducted an exploratory study with the aim of building a preliminary model to gain a better understanding of assessor expertise. Using a grounded theory approach, we conducted seventeen semi-structured interviews with individual faculty members who differed in professional background and assessment experience. The interviews focused on participants' perceptions of how they arrived at judgments about student performance. The analysis resulted in three categories and three recurring themes within these categories: the categories assessor characteristics, assessors' perceptions of the assessment tasks, and the assessment context, and the themes perceived challenges, coping strategies, and personal development. Central to understanding the key processes in performance assessment appear to be the dynamic interrelatedness of the different factors and the developmental nature of the processes. The results are supported by literature from the field of expertise development and in line with findings from social cognition research. The conceptual framework has implications for faculty development and the design of programs of assessment.

Exploring the Impact of Entrustable Professional Activities on Feedback Culture: A Qualitative Study of Anesthesiology Residents and Attendings.

PURPOSE: Entrustable professional activities (EPAs) were introduced as a potential way to optimize workplace-based assessments. Yet, recent studies suggest that EPAs have not yet overcome all of the challenges to implementing meaningful feedback. The aim of this study was to explore the extent to which the introduction of EPAs via mobile app impacts feedback culture as experienced by anesthesiology residents and attending physicians. METHOD: Using a constructivist grounded theory approach, the authors interviewed a purposive and theoretical sample of residents (n = 11) and attendings (n = 11) at the Institute of Anaesthesiology, University Hospital of Zurich, where EPAs had recently been implemented. Interviews took place between February and December 2021. Data collection and analysis were conducted iteratively. The authors used open, axial, and selective coding to gain knowledge and understanding on the interplay of EPAs and feedback culture. RESULTS: Participants reflected on a number of changes in their day-to-day experience of feedback culture with the implementation of EPAs. Three main mechanisms were instrumental in this process: lowering the feedback threshold, change in feedback focus, and gamification. Participants felt a lower threshold to feedback seeking and giving and that the frequency of feedback conversations increased and tended to be more focused on a specific topic and shorter, while feedback content tended to focus more on technical skills and more attention was given to average performances. Residents indicated that the app-based approach fostered a game-like motivation to "climb levels," while attendings did not perceive a game-like experience. CONCLUSIONS: EPAs may offer a solution to problems of infrequent occurrence of feedback and invite attention to average performances and technical competencies, but may come at the expense of feedback on nontechnical skills. This study suggests that feedback culture and feedback instruments have a mutually interacting influence on each other.

From threat to challenge-Improving medical students' stress response and communication skills performance through the combination of stress arousal reappraisal and preparatory worked example-based learning when breaking bad news to simulated patients: study protocol for a randomized controlled trial.

BACKGROUND: Breaking bad news (BBN; e.g., delivering a cancer diagnosis) is perceived as one of the most demanding communication tasks in the medical field and associated with high levels of stress. Physicians' increased stress in BBN encounters can negatively impact their communication performance, and in the long term, patient-related health outcomes. Although a growing body of literature acknowledges the stressful nature of BBN, little has been done to address this issue. Therefore, there is a need for appropriate tools to help physicians cope with their stress response, so that they can perform BBN at their best. In the present study, we implement the biopsychosocial model of challenge and threat as theoretical framework. According to this model, the balance between perceived situational demands and perceived coping resources determines whether a stressful performance situation, such as BBN, is experienced as challenge (resources > demands) or threat (resources < demands). Using two interventions, we aim to support medical students in shifting towards challenge-oriented stress responses and improved communication performance: (1) stress arousal reappraisal (SAR), which guides individuals to reinterpret their stress arousal as an adaptive and beneficial response for task performance; (2) worked examples (WE), which demonstrate how to BBN in a step-by-step manner, offering structure and promoting skill acquisition. METHODS: In a randomized controlled trial with a 2 (SAR vs. control) x 2 (WE vs. control) between-subjects design, we will determine the effects of both interventions on stress response and BBN skills performance in N = 200 third-year medical students during a simulated BBN encounter. To identify students' stress responses, we will assess their perceived coping resources and task demands, record their cardiovascular activity, and measure salivary parameters before, during, and after BBN encounters. Three trained raters will independently score students' BBN skills performances. DISCUSSION: Findings will provide unique insights into the psychophysiology of medical students who are tasked with BBN. Parameters can be understood more comprehensively from the challenge and threat perspective and linked to performance outcomes. If proven effective, the evaluated interventions could be incorporated into the curriculum of medical students and facilitate BBN skills acquisition. TRIAL REGISTRATION: ClinicalTrials.gov (NCT05037318), September 8, 2021.

Refreshing medical students' intravenous-cannulation skills: a blinded observer three-arm randomised comparison of mental imagery, part-task trainer simulation and written instructions.

INTRODUCTION: Intravenous cannulation is a core competence in medicine, but is considered challenging to learn. This study investigates the effectiveness of three educational strategies used to refresh the intravenous cannulation skills of first-year medical students: mental imagery, part-task trainer simulation and written instructions. MATERIALS AND METHODS: In this single-centre randomised controlled trial, first-year medical students were assigned to one of three different refresher tutorials on intravenous cannulation. Six months after their compulsory 4 hour instructor-led intravenous-cannulation course, each student was randomised to a 6 min self-learning tutorial: a mental imagery audioguide session, hands-on intravenous cannulation on a part-task trainer or reading written instructions.Immediately after the refresher tutorials, trained evaluators who were blinded to the randomised group assessed the students' performance. Each evaluator completed a 15-item standardised checklist in an Objective Structured Clinical Examination (OSCE) station for intravenous cannulation. We performed a descriptive analysis of the data and a one-way analysis of variance. Additionally, we investigated the influence of previous intravenous cannulation experience on the total OSCE score. RESULTS AND DISCUSSION: On analysing the 309 students' results, we did not find differences in the total rating of the performance (in percentage) between the three groups at the OSCE station (mental imagery group: 72.0%±17.9%; part-task trainer group: 74.4%±15.6%; written instructions group: 69.9%±16.6%, p=0.158). Multiple linear regression showed a small but statistically significant effect of students' previous intravenous cannulation experience on OSCE performance. With the same outcome, written instructions and mental imagery had a better return on effort, compared with resource-intensive hands-on training with part-task trainers. CONCLUSION: A single, short refresher seems to have a limited effect on intravenous-cannulation skills in first-year medical students. Less resource-intensive interventions, such as written instructions or mental imagery, are effective compared with hands-on part-task trainer simulation for refreshing this simple but important skill. TRIAL REGISTRATION NUMBER: AEARCTR-0008043.

Swiss general internal medicine board examination: quantitative effects of publicly available and unavailable questions on question difficulty and test performance.

BACKGROUND: Formerly, a substantial number of the 120 multiple-choice questions of the Swiss Society of General Internal Medicine (SSGIM) board examination were derived from publicly available MKSAP questions (Medical Knowledge Self-Assessment Program®). The possibility to memorise publicly available questions may unduly influence the candidates' examination performance. Therefore, the examination board raised concerns that the examination did not meet the objective of evaluating the application of knowledge. The society decided to develop new, "Helvetic" questions to improve the examination. The aim of the present study was to quantitatively assess the degree of difficulty of the Helvetic questions (HQ) compared with publicly available and unavailable MKSAP questions and to investigate whether the degree of difficulty of MKSAP questions changed over time as their status changed from publicly available to unavailable. METHODS: The November 2019 examination consisted of 40 Helvetic questions, 40 publicly available questions from MKSAP edition 17 (MKSAP-17) and 40 questions from MKSAP-15/16, which were no longer publicly available at the time of the examination. An one factorial univariate analysis of variance (ANOVA) examined question difficulty (lower values mean higher difficulty) between these three question sets. A repeated ANOVA compared the difficulty of MKSAP-15/16 questions in the November 2019 examination with the difficulty of the exact same questions from former examinations, when these questions belonged to the publicly available MKSAP edition. The publicly available MKSAP-17 and the publicly unavailable Helvetic questions served as control. RESULTS: The analysis of the November 2019 exam showed a significant difference in average item difficulty between Helvetic and MKSAP-17 questions (71% vs 86%, p <0.001) and between MKSAP-15/16 and MKSAP-17 questions (70% vs 86%, p <0.001). There was no significant difference in item difficulty between Helvetic and MKSAP-15/16 questions (71% vs 70%, p = 0.993). The repeated measures ANOVA on question use and the three question categories showed a significant interaction (p <0.001, partial eta-squared = 0.422). The change in the availability of MKSAP-15/16 questions had a strong effect on difficulty. Questions became on average 21.9% more difficult when they were no longer publicly available. In contrast, the difficulty of the MKSAP-17 and Helvetic questions did not change significantly across administrations. DISCUSSION: This study provides the quantitative evidence that the public availability of questions has a decisive influence on question difficulty and thus on SSGIM board examination performance. Reducing the number of publicly available questions in the examination by introducing confidential, high-quality Helvetic questions contributes to the validity of the board examination by addressing higher order cognitive skills and making rote-learning strategies less effective.

Nationwide introduction of a new competency framework for undergraduate medical curricula: a collaborative approach.

Switzerland recently introduced PROFILES, a revised version of its national outcomes reference framework for the undergraduate medical curriculum. PROFILES is based on a set of competencies adapted from the CanMEDS framework and nine entrustable professional activities (EPAs) that students have to be able to perform autonomously in the context of a predefined list of clinical situations. The nationwide implementation of such a competency- and EPA-based approach to medical education is a complex process that represents an important change to the organisation of undergraduate training in the various medical schools. At the same time, the concepts underlying PROFILES also have to be reflected at the level of the Federal Licencing Examination (FLE) and the national accreditation process. The vice-deans for education mandated a Swiss Working Group for PROFILES Implementation (SWGPI) to elaborate a guide presenting the principles and best practices based on the current scientific literature, to ensure the coherence between the future developments of the medical curricula and the evolution of the FLE, and to propose a coordinated research agenda to evaluate the implementation process. On the basis of the literature and analysis of our national context, we determined the key elements important for a successful implementation. They can be grouped into several areas including curricular design and governance, the assessment system and entrustment process, faculty development and change management. We also identified two dimensions that will be of particular importance to create synergies and facilitate exchange between the medical schools: a systematic approach to curriculum mapping and the longitudinal integration of an e-portfolio to support the student learning process. The nationwide collaborative approach to define strategies and conditions for the implementation of a new reference framework has allowed to develop a shared understanding of the implications of PROFILES, to promote the establishment of Swiss mapping and e-portfolio communities, and to establish the conditions necessary for ensuring the continuous alignment of the FLE with the evolving medical curricula.

Predictive power of high school educational attainment and the medical aptitude test for performance during the Bachelor program in human medicine at the University of Bern: a cohort study.

OBJECTIVE: Prior high school educational attainment and medical aptitude tests are two of the most frequently used selection procedures for admission to medical school. Both of these have been shown to correlate with future performance. However, there is a need for further analysis of the combined impact of these two admissions tools and comparison of their predictive value for future performance. At present, successful completion of high school (Matura) and an aptitude test (Eignungstest Medizinstudium Schweiz, EMS; Swiss Aptitude Test in Medicine) are used for admission to all medical schools in the German-speaking part of Switzerland. The purpose of this study was to explore the predictors that are most decisive for performance in undergraduate medical education. More precisely, we were interested in the contributions of the Matura grade and the EMS score to explanations of performance in the Bachelor program of Medicine at the University of Bern. METHODS: Matura grades, EMS score and performance in the Bachelor program of Medicine were collected for 730 students from four cohorts. Of these, 277 graduated from high school with a biology-chemistry major. Hierarchical regression analysis was conducted for each study year and type of examination to determine which predictors affected performance during undergraduate medical education. RESULTS: These data show that Matura grades are an important predictor for performance in undergraduate medical education. The EMS score had no impact when the Matura grades were part of the analysis. The biology-chemistry major grade was a predictor for performance in the first year of undergraduate study. From the second academic year onwards, past performance in the bachelor’s program was the best predictor for future performance during undergraduate medical education. CONCLUSIONS: Students’ Matura grades predicted their subsequent performance in undergraduate medical education in the bachelor’s program of the University of Bern. In contrast, EMS scores do not explain any additional variance in students’ performance throughout the entire bachelor’s program. These findings suggest a need for rethinking the admission process.