Variance due to the examination conditions and factors associated with success in objective structured clinical examinations (OSCEs): first experiences at Paris-Saclay medical school.

BACKGROUND: We aimed to measure the variance due to examination conditions during the first sessions of objective structured clinical examinations (OSCEs) performed at a French medical school and identify factors associated with student success. METHODS: We conducted a retrospective, observational study using data from the first three OSCEs sessions performed at Paris-Saclay medical school in 2021 and 2022. For all sessions (each organized in 5 parallel circuits), we tested a circuit effect using a linear mixed-effects model adjusted for sex and the average academic level of students (according to written tests). Then, we studied the factors associated with student success at one station using a multivariate linear mixed-effects model, including the characteristics of students, assessors, and standardized patients. RESULTS: The study included three OSCEs sessions, with 122, 175, and 197 students and a mean (± SD) session score of 13.7(± 1.5)/20, 12.7(± 1.7)/20 and 12.7(± 1.9)/20, respectively. The percentage of variance due to the circuit was 6.5%, 18.2% (statistically significant), and 3.8%, respectively. For all sessions, the student's average level and station scenario were significantly associated with the score obtained in a station. Still, specific characteristics of assessors or standardized patients were only associated with the student's score in April 2021 (first session). CONCLUSION: The percentage of the variance of students' performance due to the examination conditions was significant in one out of three of the first OSCE sessions performed at Paris-Saclay medical school. This result seems more related to individual behaviors rather than specific characteristics of assessors or standardized patients, highlighting the need to continue training teaching teams. NATIONAL CLINICAL TRIAL NUMBER: Not applicable.

Entrustable professional activities-based objective structured clinical examinations in a pharmacy curriculum.

BACKGROUND: The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice. METHODS: In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration. RESULTS: The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria. CONCLUSIONS: This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.

Evaluation of the effectiveness of using flipped classroom in puncture skills teaching.

BACKGROUND: The effectiveness of flipped classroom (FC) on puncture skills in medical education is still uncertain. This study aimed to assess the role of the FC model in puncture skills and investigate the acceptance and approval of FC among medical students and instructors. METHODS: A mixed research approach of quasi-experimental research design and descriptive qualitative research was conducted in September 2022 for one month, using an FC teaching method that combined instructional videos and group learning. The study participants were 71 fifth-year medical students from two classes at a Chinese medical school and four instructors. The medical students were randomly divided into two groups: the traditional classroom (TC) group (Group A) and the FC group (Group B). For teaching, Group B used FC, and Group A used PowerPoint-based TC. The effectiveness of the two teaching models was assessed with Objective Structured Clinical Examination (OSCE), and questionnaires were distributed to the medical students and instructors after the assessment. Two independent sample t-tests were used to analyse the differences in demographic data and the OSCE scores of the two groups of medical students. RESULTS: Group B scored higher in puncture skills than Group A, especially regarding abdominal puncture (p = 0.03), thoracentesis (p < 0.001), bone marrow puncture (p < 0.001) and average performance of puncture skills (p < 0.001). For lumbar puncture, no difference in skill scores was observed between groups A and B (p > 0.409). The medical students thought that the FC improved their self-learning ability and helped them acquire knowledge. Regarding the OSCE of their skills, most medical students thought that it was more innovative and objective than traditional examinations and that it was better for assessing their overall abilities. Both the FC and OSCE were supported by the medical students. The instructors were also satisfied with the students' performance in the FC and supported the teaching model, agreeing to continue using it. CONCLUSIONS: This study shows that FC teaching that combines instructional videos and group learning is a reliable and well-received teaching method for puncture skills, which supplements and expands existing teaching methods in the medical field.

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping.

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Prior participation as a standardized patient improves OSCE scores of third-year medical students: A pilot comparative study at Université Paris Cité Medical School.

OBJECTIVE: Objective structured clinical examinations (OSCE) are one of the main modalities of skills' assessment of medical students. We aimed to evaluate the educational value of the participation of third-year medical students in OSCE as standardized patients. METHODS: We conducted a pilot OSCE session where third-year students participated in sixth-year students' OSCE as standardized patients (cases). Their scores in their own subsequent OSCE exams were compared with third-year students who had not participated (controls). Students' perceptions (stress, preparedness, ease) regarding their OSCE were compared with self-administered questionnaires. RESULTS: A total of 42 students were included (9 cases and 33 controls). Median [IQR] overall score (out of 20 points) obtained by the cases was 17 [16.3-18] versus 14.5 [12.7-16.3] for controls (p < 0.001). Students' perception of their evaluation (difficulty, stress, communication) was not significantly different between cases and controls. Most cases agreed that their participation was beneficial in reducing their stress (67%), increasing their preparedness (78%) and improving their communication skills (100%). All cases agreed that this participation should be offered more widely. CONCLUSION: Students' participation in OSCE as standardized patients led to better performance on their own OSCE and were considered beneficial. This approach could be more broadly generalized to improve student performance.

Practical Tips for Setting Up and Running OSCEs.

Objective structured clinical examinations (OSCEs) are used to assess students' skills on a variety of tasks using live animals, models, cadaver tissue, and simulated clients. OSCEs can be used to provide formative feedback, or they can be summative, impacting progression decisions. OSCEs can also drive student motivation to engage with clinical skill development and mastery in preparation for clinical placements and rotations. This teaching tip discusses top tips for running an OSCE for veterinary and veterinary nursing/technician students as written by an international group of authors experienced with running OSCEs at a diverse set of institutions. These tips include tasks to perform prior to the OSCE, on the day of the examination, and after the examination and provide a comprehensive review of the requirements that OSCEs place on faculty, staff, students, facilities, and animals. These tips are meant to assist those who are already running OSCEs and wish to reassess their existing OSCE processes or intend to increase the number of OSCEs used across the curriculum, and for those who are planning to start using OSCEs at their institution. Incorporating OSCEs into a curriculum involves a significant commitment of resources, and this teaching tip aims to assist those responsible for delivering these assessments with improving their implementation and delivery.

Understanding and developing procedures for video-based assessment in medical education.

INTRODUCTION: Novel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We aimed to understand examiners' and students' interaction with video whilst developing procedures to promote its optimal use. METHODS: Using design-based research we developed theory and procedures for video use in assessment, iteratively adapting conditions across simulated OSCE stations. We explored examiners' and students' perceptions using think-aloud, interviews and focus group. Data were analysed using constructivist grounded-theory methods. RESULTS: Video-based assessment produced detachment and reduced volitional control for examiners. Examiners ability to make valid video-based judgements was mediated by the interaction of station content and specifically selected filming parameters. Examiners displayed several judgemental tendencies which helped them manage videos' limitations but could also bias judgements in some circumstances. Students rarely found carefully-placed cameras intrusive and considered filming acceptable if adequately justified. DISCUSSION: Successful use of video-based assessment relies on balancing the need to ensure station-specific information adequacy; avoiding disruptive intrusion; and the degree of justification provided by video's educational purpose. Video has the potential to enhance assessment validity and students' learning when an appropriate balance is achieved.

Strengthening the planning and design of Objective Structured Clinical Examinations.

Background: Although Objective Structured Clinical Examinations (OSCEs) offer innovative, objective, and fair methods of clinical assessment, their quality is compromised by poor planning and design. Aim: This study aimed to describe the development and present evidence-based recommendations on strengthening the planning and design of OSCEs for a South African public College of Nursing. Setting: A South African public College of Nursing. Methods: Recommendations were developed based on synthesising two sets of qualitative data. Set 1 included two main themes with each of the four sub-themes related to barriers and facilitators towards quality in OSCE designs from 14 nurse educator interviews and 15 external moderator reports. Set 2 included 12 quality measures to be adopted in the quality design of OSCEs derived from an integrative literature review. The draft recommendations were reviewed by eight experts to be finalised. Results: Seven recommendations were developed for strengthening OSCEs' planning and design, related to: (1) policy framework, standard operating procedures and stakeholder code of conduct; (2) blueprinting and mapping of the OSCE content; (3) developing a bank of OSCE stations; (4) scoring rubric and standard-setting method selection; (5) examiners and standardised patients' recruitment and training; (6) venue selection; and (7) station piloting. Conclusion: The seven developed recommendations can strengthen the quality of OSCEs in the South African public College of Nursing context. Contribution: The developed recommendations can assist nurse educators in planning and designing to conduct quality OSCEs following piloting and implementation.

Using Kane's Validity Framework to Compare an Integrated and Single-Skill Objective Structured Clinical Examination.

OBJECTIVE: The aim of this study was to compare the validity of an integrated objective structured clinical examination (OSCE) station assessing both oral and written components with that of an OSCE station assessing 1 single skill (oral only), both targeted at assessing taking a best possible medication history. METHODS: A convergent mixed-methods design that used the 4 inferences of Kane's validity framework (scoring, generalization, extrapolation, and implications) as a scaffold to integrate qualitative data (post-OSCE reflections) and quantitative data (assessment grades and categories of medication errors) was applied. RESULTS: In 2022, 216 students completed the OSCE station with the oral component alone, while in 2023, 254 students completed the integrated (oral and written) OSCE station. Students in 2023 performed significantly better, with a median score of 88% vs 80% in 2022. There was a greater proportion of commission errors in the integrated assessment (20.4% vs 15.3%), but fewer omission errors (29.9% vs 31.8%) and patient profile errors (5.1% vs 69.4%). Student reflections revealed that conversations were rushed in the integrated assessment, with a greater focus on written formatting, but an appreciation for the authenticity and structured format of the integrated OSCE compared with the single-skill OSCE alone. CONCLUSION: Students completing the integrated OSCE (with oral and written components) had fewer patient profile and medication omission errors than students who completed the oral-only OSCE. Considering Kane's validity framework, there was a stronger argument for the more authentic integrated OSCE in terms of the inferences of extrapolation and implications.

Investigating the Use of Simulated Patient Ratings in Objective Structured Clinical Examinations to Enhance Undergraduate Medical Assessments in Riyadh, Saudi Arabia.

BACKGROUND: Objective structured clinical examinations (OSCE) are the gold standard of clinical assessment, and are used to conduct undergraduate family medicine clinical assessment at King Faisal Specialist Hospital and Research Centre (KFSHRC). Some studies have suggested that simulated patient (SP) ratings could provide a better measure of empathy and communication skills than physician scores. The objective of this study is to further explore the effectiveness of simulated patient (SP) ratings in undergraduate OSCE assessments. METHODS: The research employed a mixed-method approach. Three OSCE assessments for final-year students were selected. Both physicians and SPs evaluated each student, providing global ratings across four domains. The quantitative aspect involved comparing physician and SP scores and assessing correlation. The qualitative aspect involved interviewing SPs to establish what student behaviours led to higher or lower scores. RESULTS: Moderate correlation was found between physician ratings and SP ratings (r=0.53, p<0.01). Internal consistency of the SP ratings was lower than physician scores. SPs considered themselves to be patient advocates and were keen to give formative feedback. The ability of the trainee to truly listen was a major concern. Scoring for SPs was relatively holistic in nature. CONCLUSIONS: The results demonstrate that SP scores have slightly weaker reliability but are still relevant and offer a completely different perspective, enriching the assessment data. Assessment should take patient or SP perspectives into account, and not rely solely on the expert physician. Changing the assessment methods will lead to necessary changes in student approach to the OSCE and improve authenticity and validity.

Dental hygiene educators' knowledge and implementation of objective structured clinical examination in United States dental hygiene programs.

PURPOSE: In 2021, the American Dental Association (ADA) announced the intention to develop a standardized dental hygiene licensure objective structured clinical examination (DHLOSCE). The purpose of this study was to measure the United States (US) dental hygiene (DH) educators' foundational knowledge of OSCE development and delivery in light of the impending development of the DHLOSCE by the ADA's Testing Services. METHODS: The study was determined to be exempt from Institutional Review Board oversight. A 21-question survey was developed, pilot tested, and electronically disseminated through Qualtrics. The survey recruitment was emailed to the directors of all entry-level DH education programs in the US (n = 328), asking them to participate in the survey and to forward it to the clinical faculty in their institutions. Descriptive and inferential statistics were utilized to analyze the data. RESULTS: There were 143 study participants, for a completion rate of 45%. Over two-thirds of respondents (64%) were unaware of the plans to develop the DHLOSCE, while 13% reported utilizing OSCEs to meet accreditation standards. Only 3% reported receiving a formal education in OSCE development compared to 29% who learned through a colleague or peer. Nearly half reported a lack of faculty experience as a barrier to OSCE implementation. Over three-quarters, 76% reported a lack of OCSE development committees within their program and only 14% had experience developing an OSCE station. CONCLUSION: The study results suggest an urgent need for the development of OSCE training resources specific to DH education, as programs across the US prepare for the impending DHLOSCE.

Assessment of cardiopulmonary point-of-care ultrasound objective structured clinical examinations in graduating anesthesiology residents across multiple residency programs.

STUDY OBJECTIVE: To implement and assess a cardiopulmonary point-of-care ultrasound (POCUS) objective structured clinical examination (OSCE) in a large cohort of graduating anesthesia residents. DESIGN: Observational cohort study. SETTING: University-affiliated hospitals. SUBJECTS: 150 graduating anesthesia residents in their last nine months of training. INTERVENTIONS: A standardized cardiopulmonary OSCE was administered to each resident. MEASUREMENTS: The cardiac views evaluated were parasternal long axis (PLAX), apical 4 chamber (A4C), and parasternal short axis (PSAX). The pulmonary views evaluated were pleural effusion (PLE) and pneumothorax (PTX). In addition, a pre- and post-exam survey scored on a 5-point Likert scale was administered to each resident. MAIN RESULTS: A4C view (mean 0.7 ± 0.3) scored a lower mean, compared to PSAX (mean 0.8 ± 0.3) and PLAX (mean 0.8 ± 0.4). Residents performed well on the PTX exam (mean 0.9 ± 0.3) but more poorly on the PLE exam (mean 0.6 ± 0.4). Structural identification across cardiac and pulmonary views were mostly high (means >0.7), but advanced interpretive skills and maneuvers had lower mean scores. Pre- and post- OSCE survey results were positive with almost all questions scoring >4 on the Likert scale. CONCLUSION: Our study demonstrates that a cardiopulmonary POCUS OSCE can be successfully implemented across multiple anesthesia training programs. While most residents were able to perform basic ultrasound views and identify structures, advanced interpretive skills and maneuvers performed lower.

A Comparison of the Checklist Scoring Systems, Global Rating Systems, and Borderline Regression Method for an Objective Structured Clinical Examination for a Small Cohort in a Saudi Medical School.

BACKGROUND: This study aims to compare the effectiveness of using the checklist and global rating scores to evaluate the clinical competency of medical students in Objective Structured Clinical Examinations (OSCEs). Additionally, the study assesses the appropriateness of using the borderline regression method to set standards for small-scale OSCE exams and determines if the estimated passing marks differ significantly from the university's prefixed passing score of 70%. The study also examines whether the university should utilize the borderline regression method to determine passing scores for each OSCE exam instead of a set passing score. METHODS: The study analyzed medical students' grades in 11 OSCE exams in the 2022-2023 academic year at Alfaisal University, Riyadh, Saudi Arabia. Students received family medicine clerkship rotations, and after each rotation, they took an OSCE exam consisting of three stations that family medicine consultants graded. The exam included a checklist of 30 tasks and a five-level global rank scale. The study collected all the checklist marks and global rank grades and analyzed them using IBM® Statistical Package for Social Sciences (SPSS® Statistics) software. The statistical tests used were descriptive statistics, the T-test, chi-square tests, Fisher's exact test, and Pearson correlation. RESULTS: The study showed that students were more likely to pass when using the global rating system than the checklist scoring system. Additionally, students had a significantly lower passing rate when using the higher cut-off passing score estimated using the borderline regression method compared to the pre-set passing score of 70% established by the university (with a p-value of 0.00). CONCLUSION: Each scoring system has advantages and disadvantages, but they complement each other. Combining scoring systems can produce a more comprehensive and precise evaluation of a candidate's performance. The study also emphasizes the importance of carefully selecting and validating cut-off points in OSCE exams to ensure fairness and consistency in assessment.

Comparison of student pharmacists' performance on in-person vs. virtual OSCEs in a pre-APPE capstone course.

INTRODUCTION: Objective structured clinical examinations (OSCEs,) provided in-person or virtually, assess student pharmacist readiness for advanced pharmacy practice experiences (APPEs). During the COVID-19 pandemic in 2020, it was necessary for many educators to design and implement virtual OSCEs (vOSCEs). Impact on student performance utilizing in-person vs. vOSCE has not been well evaluated. The objective of this study was to determine if a difference existed in student performance when comparing in-person vs. vOSCE in a third year (P3) pharmacy pre-APPE capstone course. METHODS: In winter 2019, four in-person OSCE stations were designed and implemented in a pre-APPE P3 capstone course. In winter 2021, the same four stations were transitioned into vOSCE stations. Assessment (summative) data from similar student cohorts from OSCE 2019 were compared vOSCE 2021 stations using Mann-Whitney U test. RESULTS: There was no meaningful difference found when comparing student performance on in-person OSCE vs. vOSCE. There was no significant difference for the number of students offered remediation. For most stations, performance improved from formative to summative assessments. CONCLUSIONS: Providing vOSCEs to assess students' skills in a pre-APPE P3 capstone course is a reasonable alternative to in-person OSCEs.

Quality management of objective structured clinical examinations: An integrative literature review.

BACKGROUND: Objective Structured Clinical Examinations are widely adopted as a clinical assessment method in nurse education. Quality is an integral aspect of the design and implementation of OSCEs, facilitating their credibility and rigour. However, there is a dearth of literature regarding the management of the quality of Objective Structured Clinical Examinations. OBJECTIVE: To summarise existing literature regarding best practices on the management of the quality of Objective Structured Clinical Examinations in health science education. DESIGN: An integrative literature review. DATA SOURCES: EBSCOhost, including CINAHL, eBook Collection, E-journals, ERIC, Health Source-Consumer Edition, Health Source-Nursing/Academic Edition, Humanities International Complete and MEDLINE, together with Cochrane Online, PubMed, Taylor & Francis Online and ScienceDirect, were searched, followed by hand searching of references as well as a manual search for grey literature, using Google. REVIEW METHODS: Using predetermined inclusion and exclusion criteria, research documents (experimental, non-experimental, descriptive and qualitative studies), non-research documents (editorials, opinion letters), including grey literature, published between January 2010 and March 2021. RESULTS: A total of thirteen (n = 13) studies were included in the review. A total of 22 quality measures were identified, which should be applied in the preparation and planning, implementation and evaluation phases of Objective Structured Clinical Examinations. The preparation and planning phase is crucial for the quality of Objective Structured Clinical Examinations, as more than half (n = 12) of the 22 quality measures were identified in this phase. CONCLUSIONS: There is limited available recorded evidence that defines and describes the management of the quality of Objective Structured Clinical Examinations. Future research should focus on the context-specific use of psychometric analysis to ensure generation of valid, objective and consistent assessment during Objective Structured Clinical Examinations. More rigorous large-scale studies (such as quantitative or randomised controlled trials) regarding management of variances amongst examiner scores, impact of moderation on Objective Structured Clinical Examinations, and virtual Objective Structured Clinical Examinations should be conducted.

Objective structured clinical examinations (OSCEs) for students' training and assessment in the French respiratory medicine departments in 2021: An overview.

BACKGROUND: Medical professional performances can be assessed by objective structured clinical examinations (OSCEs) where medical trainees go through a series of simulated clinical situations. OSCEs are now the gold standard for the assessment of medical students' competence during their training. In France, the first national OSCEs will take place in May 2024 and respiratory teachers will be involved in this reform and will use OSCEs for students' training and assessment in respiratory medicine. Students training regarding this final OSCE may vary across medical faculties and may impact students' results. Therefore, we aimed to provide a national overview of OSCE's training performed by respiratory teachers and their interest in developing a common French databank of OSCEs. METHODS: We conducted a national anonymous online survey among the members of the French college of respiratory teachers (CEP), from 2021 February the 15th to 2021 June the 15th. The survey consisted of 32 questions. RESULTS: Among 118 French pulmonologists teachers, 52 (45%) responded to the survey. We received a response from at least one of each of the French Medical Universities. Twenty-two (42%) had received specific training on how to conduct an OSCE. Twenty-eight (54%) of respondents used OSCEs for training purposes and 24 (46%) for assessment purposes, for less than 1 year in more than half of the participants. The average satisfaction scores out of 10 about OSCEs was 7.3 ± 1.7 for training and 7.4 ± 1.5 for students' assessment. Respondents were willing (8.9 ± 1.8 out of ten) to develop a common databank to share OSCEs subjects in respiratory medicine in France. CONCLUSIONS: This survey confirms heterogeneity in the training and the use of OSCEs among French respiratory teachers. However, a common national databank could be a useful tool to reduce these disparities.

Nailed it! Doctor of pharmacy learners' self-awareness of performance on authentic assessments.

INTRODUCTION: The primary objective of this pilot study was to determine the reliability and validity of an objective structured clinical examination (OSCE) self-awareness survey. A secondary objective was to assess learner self-awareness in OSCE performance. METHODS: A retrospective review of OSCE data from a single cohort in a three-year accelerated doctor of pharmacy program occurred. The cohort completed four OSCEs with a total of 23 discrete cases. At the end of each case, before receiving feedback, learners completed an evaluation of their skills, communication, overall performance, and identified if they believed the patient/provider would return ("nailed or failed"). These self-evaluations were then compared with respective learner performance scores for each case. Content validity was assessed for the self-awareness survey via the Lawshe method. Reliability of the OSCE cases and self-awareness survey were conducted utilizing Cronbach's alpha and the Spearman-Brown formula, respectively. RESULTS: The OSCE self-awareness survey was found to be face valid and reliable for the overall survey. The subsections of return to pharmacist and overall learner performance were also found to be valid and reliable. A statistical relationship was found between learner self-assessment of pass/fail and their actual grade for overall performance and return to pharmacist. CONCLUSIONS: Results from this study may guide remediation plans related to OSCE performance and development of other self-awareness activities in practice settings. Further studies are needed to evaluate the broader application and use of self-awareness tools.

"Could You Work in My Team?": Exploring How Professional Clinical Role Expectations Influence Decision-Making of Assessors During Exit-Level Medical School OSCEs.

Decision-making in clinical assessment, such as exit-level medical school Objective Structured Clinical Examinations (OSCEs), is complex. This study utilized an empirical phenomenological qualitative approach with thematic analysis to explore OSCE assessors' perceptions of the concept of a "prototypical intern" expressed during focus group discussions. Topics discussed included the concept of a prototypical intern, qualities to be assessed, and approaches to clinical assessment decision-making. The thematic analysis was then applied to a theoretical framework (Cultural Historical Activity Theory-CHAT) that explored the complexity of making assessment decisions amidst potentially contradicting pressures from academic and clinical perspectives. Ten Australasian medical schools were involved with 15 experienced and five less experienced assessors participating. Thematic analysis of the data revealed four major themes in relation to how the prototypical intern concept influences clinical assessors' judgements: (a) Suitability of marking rubric based on assessor characteristics and expectations; (b) Competence as final year student vs. performance as a prototypical intern; (c) Safety, trustworthiness and reliability as constructs requiring assessment and (d) Contradictions in decision making process due to assessor differences. These themes mapped well within the interaction between two proposed activity systems in the CHAT model: academic and clinical. More clinically engaged and more experienced assessors tend to fall back on a heuristic, mental construct of a "prototypical intern," to calibrate judgements, particularly, in difficult situations. Further research is needed to explore whether consensus on desirable intern qualities and their inclusion into OSCE marksheets decreases the cognitive load and increases the validity of assessor decision making.

Simple moulage for improving simulation for objective structured clinical examinations in undergraduate dermatology courses.

BACKGROUND: Objective structured clinical examination (OSCE) is being increasingly used as an assessment tool for undergraduate dermatology courses. One of the practical difficulties in conducting OSCEs in dermatology is getting patients with typical skin lesions which can be used for the whole group to ensure uniformity of assessment. We present a study on the use of simple moulage techniques to create uniform and standardized skin lesions for OSCEs in dermatology. METHODS: As a first step, the dermatology faculty in our department chose the clinical conditions which could be covered by using moulages. The main criteria considered were the importance of the condition to the exam blueprint, ease of making and resistance to handling (should not require frequent retouching). Moulages were created on volunteers after taking consent and the same were used in OSCEs s for a group of 5th-year students (N = 102). Difficulty and discrimination indices were compared between the stations using the moulage and the other stations. Qualitative feedback was obtained regarding the same from both the faculty and the students. RESULTS: There was consensus among the faculty and the majority of the students that the lesions were clearly recognizable. As far as other psychometrics were concerned, average difficulty and discrimination of the stations using the moulage were good (average difficulty index-0.78 and average discrimination index-0.68) and compared favorably with the other stations (average difficulty index-0.77 and average discrimination index-0.57). LIMITATIONS: Limited number of stations included, lack of detailed item analysis and lack of feedback from the simulated patients were the main limitations in this study. CONCLUSION: For most common skin conditions creating moulages to simulate the corresponding lesions is an easy procedure and can be an effective tool to standardize dermatology OSCEs for undergraduates, especially in resource-poor settings.

Utilization of Objective Structured Clinical Examinations (OSCE) as an Assessment for Clinical Competency and Licensure: A survey of dental hygiene director's knowledge and attitudes.

Purpose: Objective Structured Clinical Examinations (OSCEs) have been established as a gold standard assessment for determining clinical competence. The Coalition for Dental Licensure Reform called for the acceptance of the Dental Licensure Objective Structured Clinical Examination (DLOSCE) to replace the live-patient examinations (LPE) for dental licensure, which are often viewed as biased, unreliable, and in some cases unethical. The purpose of this study was to assess dental hygiene program directors' awareness of and attitudes toward the DLOSCE, whether their curricula included OSCEs, and perceived barriers to implementing OSCEs.Methods: A nine-question electronic survey was developed, and pilot tested by five-dental hygiene program directors across three-dental hygiene institutions. The survey was emailed to the directors of all dental hygiene program directors in the United States (n=332). Descriptive statistics were used to analyze the data.Results: A response rate of 36% (n=121) was achieved. Nearly 30% of respondents were unaware of the developing DLOSCE, however, the majority (80%) were in favor of the acceptance of the examination. Nearly three-quarters of the respondents considered OSCEs as valid assessments of clinical competence, however, over half of the respondents reported not currently utilizing OSCEs in their curricula. Barriers reported were time (22%), perceived lack of best practices (21%), and lack of resources (18%). Respondents who currently employed OSCEs were more likely to agree they were both valid and reliable assessments (p=0.05).Conclusion: The majority of dental hygiene program directors were in favor of eliminating the single-encounter LPE in favor of an OSCE for licensure. However, more than half do not currently utilize OSCEs for clinical assessments. Further studies are needed to explore implementation of OSCEs in dental hygiene education, and how a potential dental hygiene licensure OSCE might impact the current curricula and licensure of dental hygienists in the United States.