Essential steps in the development, implementation, evaluation and quality assurance of the written part of the Swiss federal licensing examination for human medicine.

Purpose: This report describes the essential steps in the development, implementation, evaluation and quality assurance of the written part of the Swiss Federal Licensing Examination for Human Medicine (FLE) and the insights gained since its introduction in 2011. Methods: Based on existing scientific evidence, international expertise, and experience gained from previous examinations, the FLE is developed by experts from all five medical faculties in Switzerland with the support of the Institute for Medical Education and is held simultaneously at five locations. The exam organisers document and review every examination held and continuously optimise the processes; they have summarised the results in this report. Results: The essential steps comprise the development, revision and translation of questions; construction of the exam and production of materials; candidate preparation; implementation and analysis. The quality assurance measures consist of guideline coherence in the development of the questions and implementation of the exam, revision processes, construction of the exam based on the national blueprint, multiphase review of the translations and exam material, and statistical analysis of the exam and the comments from candidates. The intensive collaboration, especially on the part of representatives from all the participating faculties and a central coordination unit, which provides methodological support throughout and oversees the analysis of the exam, has proven successful. Successfully completed examinations and reliable results in the eleven examinations so far implemented represent the outcomes of the quality assurance measures. Significant insights in recent years are the importance of appreciating the work of those involved and the central organisation of exam development, thus ensuring the long-term success of the process. Conclusion: Common guidelines and workshops, quality assurance measures accompanied by the continuous improvement of all processes, and appreciation of everyone involved, are essential to carrying out such an examination at a high-quality level in the long term.

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills--a controlled study.

BACKGROUND: The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS: All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS: At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS: Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.

The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships--a multilevel analysis.

BACKGROUND: In contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students' mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics. METHODS: Medical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students' mean scores of 9 clinical skills OSCE stations and mean 'overall' and 'domain' mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments. RESULTS: A total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87%). Most influential for the prediction of 'overall' mini-CEX scores was the trainers' clinical position with a regression coefficient of 0.55 (95%-CI: 0.26-0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced 'overall' mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase 'overall' mini-CEX scores. CONCLUSION: In our study, Mini-CEX scores depended rather on context characteristics than on students' clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores' validity or to use narrative comments only.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group.

OBJECTIVE: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. METHODS: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. RESULTS: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. CONCLUSION: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking.

Diffuse idiopathic skeletal hyperostosis (DISH) of the elbow: a controlled radiological study.

BACKGROUND: Extraspinal manifestations of diffuse idiopathic skeletal hyperostosis (DISH) have been described previously. We aimed to assess the prevalence of elbow hyperostotic spurs, to search for sites discriminating for elbow DISH and to analyze the effect of physical activities, handedness and sex. METHODS: Out of 284 patients hospitalized for extraskeletal disorders, 85 patients (33 with and 52 without thoracospinal DISH) agreed to bilateral elbow X-rays in two projections. Clinical information was collected by a standardized questionnaire and X-rays were graded blindly. RESULTS: A total of 400 hyperostotic spurs (210 unilateral, 95 bilateral) were present at 11 predefined sites. The most frequent sites affected were the olecranon (20.8%), lateral epicondyle (17.8%) and medial epicondyle (15.5%). In carriers of thoracospinal DISH significantly more hyperostotic spurs were present at the lateral and medial epicondyle compared to non-DISH carriers (OR 4.01 [95% CI 1.35-12.34] and 2.88 [1.03-8.24], respectively). The olecranon, lateral and medial epicondyle contributed significantly to the classification of elbow DISH (OR 22.2 [4.1-144.7], 9.6 [1.9-61.2] and 10.1 [2.2-52.1], respectively). The prevalence of elbow hyperostotic spurs was higher in 45 patients with a history of heavy physical activities (24.4% versus 18.0%, OR 1.48 [1.17-1.86]), at the right elbow (24.2% versus 18.6%, OR 1.39 [1.11-1.75]) and in 62 males (22.8% versus 17.6%, OR 1.38 [1.06-1.81]). CONCLUSIONS: Hyperostotic spurs at the olecranon, lateral and medial epicondyle had the highest prevalence and disclosed the most pronounced discrimination for elbow DISH. Mechanical factors such as physical activities and handedness, and sex influenced the formation of these spurs.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES: The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer-student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS: Retrospective analyses of adapted mini-CEX forms (trainers' and students' assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS: A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains of 'clinical reasoning' (23% of 1167 complete forms), 'organisation/efficiency' (20%) and 'physical examination' (20%). At least one 'defined' LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were 'physical examination' (49%) and 'history taking' (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS: Although trainer-student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students' LNs raise questions about how the full potential of the mini-CEX as not only a 'diagnostic' but also an 'educational' tool can be exploited.

The new licencing examination for human medicine: from concept to implementation.

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical "clinical skills" test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

Retrospective adjustment of self-assessed medical competencies - noteworthy in the evaluation of postgraduate practical training courses.

AIM: The efficacy of postgraduate practical training courses is frequently evaluated by self-assessment instruments. The present study analyses the effect of a basic course in laparoscopic surgery on self-assessed medical competencies. METHODS: The 3-day course included teaching of knowledge and training of practical skills. In relation to course evaluation, a questionnaire for self-assessment was applied at the beginning of the course ('pre-course'), at the end of the course ('post-course') and at the end of the course to reassess pre-course competencies ('retrospective pre-course'). RESULTS: 89 out of 110 participants (81%) attending 10 courses completed all the questionnaires; 83% were postgraduate trainees in surgery and 82% were inexperienced as an independent surgeon. At the beginning of the course most trainees rated themselves as 'moderately competent' or 'fully competent' with respect to the various task levels as well as to specific areas of medical competencies. At the end of the course however pronounced retrospective revisions of self-assessment to lower ratings became apparent. Statistically significant differences were seen for the task 'performing surgical procedures under supervision' and for most of the practical skills trained during the course (p <0.01). In contrast, no significant differences were observed for knowledge taught during the course as well as for 'ability to work in a team' and 'ability to concentrate', which were not foci of the course. CONCLUSIONS: Surgeons with little experience change their self-assessment of pre-course competencies to a lower level after participation in a practical postgraduate training course. Evaluations comparing 'pre-course' and 'post-course' ratings only - without 'retrospective pre-course' ratings - may underestimate the training effects. This phenomenon needs to be taken into account when evaluations are dependent exclusively on self-assessment instruments.

Postgraduate training program.

BACKGROUND: Faculties face the permanent challenge to design training programs with well-balanced educational outcomes, and to offer various organised and individual learning opportunities. AIM: To apply our original model to a postgraduate training program in rheumatology in general, and to various learning experiences in particular, in order to analyse the balance between different educational objectives. METHODS: Learning times of various educational activities were reported by the junior staff as targeted learners. The suitability of different learning experiences to achieve cognitive, affective and psychomotor learning objectives was estimated. Learning points with respect to efficacy were calculated by multiplication of the estimated learning times by the perceived appropriateness of the educational strategies. RESULTS: Out of 780 hours of professional learning per year (17.7 hours/week), 37.7% of the time was spent under individual supervision of senior staff, 24.4% in organised structured learning, 22.6% in self-studies, and 15.3% in organised patient-oriented learning. The balance between the different types of learning objectives was appropriate for the overall program, but not for each particular learning experience. Acquisition of factual knowledge and problem solving was readily aimed for during organised teaching sessions of different formats, and by personal targeted reading. Attitudes, skills and competencies, as well as behavioural and performance changes were mostly learned during caring for patients under interactive supervision by experts. CONCLUSION: We encourage other faculties to apply this approach to any other curriculum of undergraduate education, postgraduate training or continuous professional development in order to foster the development of well-balanced learning experiences.