Entrustment versus performance scale in high-stakes OSCEs: Rater insights and psychometric properties.

BACKGROUND: Although entrustment scales are increasingly applied in workplace-based assessments, their role in OSCEs remains unclear. We investigated raters' perceptions using an entrustment scale and psychometric analyses. METHOD: A mixed-methods design was used. OSCE raters' (n = 162) perceptions were explored via questionnaire and four focus groups (n = 14). Psychometric OSCE properties were analyzed statistically. RESULTS: Raters (n = 53, response rate = 41%) considered the entrustment scale comprehensible (89%) and applicable (60%). A total of 43% preferred the entrustment scale, 21% preferred the global performance scale, and 36% were undecided. Raters' written comments indicated that while they appreciated the authenticity of entrustment levels, they considered them subjective. The focus groups highlighted three main themes: (1) recollections of the clinical workplace as a cognitive reference triggered by entrustment scales; (2) factors influencing entrustment decisions; and (3) cognitive load is reduced at the perceived cost of objectivity. Psychometric analyses (n = 480 students) revealed improvements in some OSCE metrics when entrustment and global performance scales were combined. CONCLUSION: Entrustment scales are beneficial for high-stakes OSCEs and have greater clinical relevance from the raters' perspective. Our findings support the use of entrustment and global performance scales in combination.

Elementary school children as standardized patients in a summative OSCE - A mixed-method study according to the Ottawa criteria for good assessment.

BACKGROUND: The goal of the present study was to evaluate whether children as standardized patients (SPs) for a summative OSCE station fulfills the Ottawa criteria for good assessment. METHODS: Data for participating students (n = 124), raters (n = 11), and children (n = 127) were collected using questionnaires (students, raters), focus groups (raters), and interviews (children, raters). The psychometrics (including reproducibility) of this OSCE station were also analyzed according to quality indices. RESULTS: Validity, reproducibility, feasibility, educational effect, and acceptability were attained for this OSCE station. Students and raters accepted the new approach and evaluated it as fair and acceptable, with realism and validity. This OSCE station had a perceived positive educational effect for ∼30% of the students, who expected child SPs. Raters stated it as feasible to involve children. The children enjoyed their participation, which was acceptable and feasible. Statistical measures were in the recommended ranges and showed reproducibility. CONCLUSION: Children as SPs for a summative OSCE station was perceived as fair, acceptable, realistic, and feasible by all stakeholders, and with perceived positive educational impact. This pediatric OSCE station fulfilled five of the seven Ottawa criteria for good assessment. Further studies are needed to investigate different pediatric stations.

Improving the assessment of communication competencies in a national licensing OSCE: lessons learned from an experts' symposium.

BACKGROUND: As the communication competencies of physicians are crucial for providing optimal patient care, their assessment in the context of the high-stakes Objective Structured Clinical Examination (OSCE) is of paramount importance. Despite abundant literature on the topic, evidence-based recommendations for the assessment of communication competencies in high stakes OSCEs are scarce. As part of a national project to improve communication-competencies assessments in the Swiss licensing exam, we held a symposium with national and international experts to derive corresponding guidelines. METHODS: Experts were invited on account of their recognized expertise either in teaching or assessing communication competencies, or in conducting national high-stakes OSCEs. They were asked to propose concrete solutions related to four potential areas for improvement: the station design, the rating tool, the raters' training, and the role of standardized patients. Data gene.rated in the symposium was available for analysis and consisted of video recordings of plenary sessions, of the written summaries of group work, and the cards with participants' personal take-home messages. Data were analyzed using a thematic analysis approach. RESULTS: Nine major suggestions for improving communication-competencies assessments emerged from the analysis and were classified into four categories, namely, the roles of the OSCE scenarios, rating tool, raters' training, and simulated patients. CONCLUSION: In the absence of established evidence-based guidelines, an experts' symposium facilitated the identification of nine practical suggestions for improving the assessment of communication competencies in the context of high-stakes OSCEs. Further research is needed to test effectiveness of the suggestions and how they contribute to improvements in the quality of high-stakes communication-competencies assessment.

Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam.

Background: Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine. Methods: All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded. Results: The data from these examiners (n=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, n=60, 2015: 89.8%, n=230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists. Conclusions: Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.

[Determination of sex steroids in shed skin of the beaded lizard Heloderma suspectum ("Gila Monster")]. / Bestimmung von Sexualsteroiden in abgestoßener Haut der Gila-Krustenechse (Heloderma suspectum).

OBJECTIVE: Measurement of steroid hormones in skin appendages such as mammalian hair or claws and in avian feathers represents a recognized non-invasive method for the determination of these parameters. The aim of this pilot study in the Gila Monster was to investigate whether the measurement of sex steroids in shed skin may be employed for the monitoring of endocrine gonadal function or sex determination in reptiles. MATERIAL AND METHODS: Shed skins were available from 11 female and 7 male adult and sexually mature animals. Large pieces of skin were initially cut into smaller pieces with scissors. The resultant dermal fragments were finely ground under liquid nitrogen and finally extracted with organic solvents. The following parameters were determined radioimmunologically in the dried and re-dissolved extracts: progesterone (P4), estradiol-17ß (E2), testosterone (T), free total estrogens (fGÖ) and free plus conjugated total estrogens (fkGÖ). RESULTS: For P4 (p = 0.0052) and E2 (p = 0.0079) significant sex differences were found with higher concentrations in females compared to males. Unexpectedly, the measured values for T were also significantly higher in females (p = 0.0232) than in males, with the concentrations overall only slightly above the detection limit. Compared to fGÖ, the concentrations of fkGÖ were only slightly higher, with no significant differences between both sexes. CONCLUSION AND CLINICAL RELEVANCE: In this pilot study, the methods employed did not allow for reliable sex determination in individual animals, neither alone nor in combination, due to an overlap between the sexes. In principle, however, the measurement of sex steroids in shed skins could represent a useful method for non-invasive sex determination or endocrine gonadal function assessment in certain reptile species.

Occurrence of sulfonated steroids and ovarian expression of steroid sulfatase and SULT1E1 in cyclic cows.

Historically sulfonated steroids were primarily considered as inactive metabolites destined for elimination. However, more recently they have been increasingly recognized as precursors for the production of bioactive steroids in target tissues and as functional molecules without preceding hydrolysis. In order to comprehensively characterize their occurrence in cyclic cows and their formation and hydrolysis in bovine ovarian steroidogenesis, ovaries from cyclic cows were screened for the expression of oestrogen sulfotransferase (SULTE1) and steroid sulfatase (STS) by Western blot and immunohistochemistry. Moreover, a broad spectrum of 13 sulfonated steroids was measured applying liquid chromatography-tandem mass spectrometry (LC-MS/MS) in blood samples collected from three cycling heifers during defined stages of the ovarian cycle and in fluid obtained from ovarian follicles of different size. SULT1E1 was undetectable in ovarian tissues. For STS only a weak immunostaining was found predominantly in granulosa cells of larger follicles. However, no specific band occurred in Western blot. In blood, concentrations of all sulfonated steroids investigated were below the limit of quantification (LOQ). In follicular fluid, only cholesterol sulfate was measured in considerable concentrations (328.3 ±â€¯63.8 ng/ml). However, the role of cholesterol sulfate in bovine follicular steroidogenesis remains unclear as concentrations were obviously unrelated to follicular size. The remaining sulfonated steroids investigated were undetectable or only slightly exceeded LOQ in a minor proportion of samples. The results are clearly contrary to a role of sulfonated steroids as important precursors, intermediates or products of bovine ovarian steroidogenesis.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group.

OBJECTIVE: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. METHODS: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. RESULTS: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. CONCLUSION: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking.

The role of strategy and redundancy in diagnostic reasoning.

BACKGROUND: Diagnostic reasoning is a key competence of physicians. We explored the effects of knowledge, practice and additional clinical information on strategy, redundancy and accuracy of diagnosing a peripheral neurological defect in the hand based on sensory examination. METHOD: Using an interactive computer simulation that includes 21 unique cases with seven sensory loss patterns and either concordant, neutral or discordant textual information, 21 3rd year medical students, 21 6th year and 21 senior neurology residents each examined 15 cases over the course of one session. An additional 23 psychology students examined 24 cases over two sessions, 12 cases per session. Subjects also took a seven-item MCQ exam of seven classical patterns presented visually. RESULTS: Knowledge of sensory patterns and diagnostic accuracy are highly correlated within groups (R2 = 0.64). The total amount of information gathered for incorrect diagnoses is no lower than that for correct diagnoses. Residents require significantly fewer tests than either psychology or 6th year students, who in turn require fewer than the 3rd year students (p < 0.001). The diagnostic accuracy of subjects is affected both by level of training (p < 0.001) and concordance of clinical information (p < 0.001). For discordant cases, refutation testing occurs significantly in 6th year students (p < 0.001) and residents (p < 0.01), but not in psychology or 3rd year students. Conversely, there is a stable 55% excess of confirmatory testing, independent of training or concordance. CONCLUSIONS: Knowledge and practice are both important for diagnostic success. For complex diagnostic situations reasoning components employing redundancy seem more essential than those using strategy.

The new licencing examination for human medicine: from concept to implementation.

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical "clinical skills" test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.