Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping.

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Implementation of a hepatitis-themed virtual escape room in pharmacy education: A pilot study.

As we enter a world of blended learning in higher education, an increased need for adaptation of teaching strategies to enhance engagement has been recognised to amplify learning outcomes online. Gamification has been identified as a creative tool to engage the current cohort of learners who are also characteristically tech-savvy. To this end, escape room games have gained considerable traction in medical and pharmacy education to promote learning, critical thinking and teamwork. In this pilot study we describe the implementation of a 60-minute, web-based hepatitis-themed escape room game within a Year 3 Pharmacotherapy unit at Monash University. A total of 418 students participated in this activity. Students' knowledge gain on the topic was assessed through a pre- and post-intervention assessment, whereby a statistically significant improvement was seen in the knowledge score following implementation of the gaming activity (58.66% pre-intervention vs. 72.05% post-intervention, p < 0.05). The innovative learning activity was also well perceived by the students. Virtual escape room game is a viable pedagogical approach to teach and reinforce clinical concepts among pharmacy students. With the evolving landscape of education and learner demographics, investment in technology- enhanced game-based learning is a promising trajectory to support students' growth in a learner-centered environment. A comparison between virtual escape room game and traditional teaching will further inform effectiveness of the gamification on long term knowledge retention.

A teamwork OSCE station - Encompassing shared decision making between a doctor, pharmacist and patient.

BACKGROUND AND PURPOSE: Teamwork with a focus on incorporating interprofessional education (IPE) is a core skill taught in pharmacy curriculum. Including the patient in shared decision making has become a more empathetic and holistic approach to care. Teamwork skills and approaches are usually assessed by peers via group work. EDUCATIONAL ACTIVITY AND SETTING: A teamwork Objective Structured Clinical Examination (OSCE) station was implemented in 2022 in two countries, conducted in final year pharmacy students where students had to solve a clinical scenario with a doctor and patient in the room and manage any resistances to therapy recommendations. Students were marked on their collaborative skills and how they articulated clinical evidence to justify their therapy recommendations to reach an optimal goal to suit the team. Perception data from students simulated patients and doctors were also collected. FINDINGS: Across both countries (n = 414), the overall mean score was 79.1% (24.0%SD) across six different OSCE cases. Students generally struggled with the concept of engaging in a two-way conversation, and often presented their full recommendations without any breaks to confirm understanding or agreement. Simulated patients appreciated the OSCE showing a patient role in decision making. Simulated doctors valued the OSCE for its assessment of clinical justification in times of disagreement. Overall, 86.2% of students (n = 354) agreed or strongly agreed the station assessed their teamwork skills adequately. SUMMARY: A teamwork OSCE station is a valuable assessment for assessing shared decision making skills and more work around prepare students to engage in two-way conversations is needed.

Do students' self-reflections of performance align with their graded performance in objective structured clinical exams?

INTRODUCTION: Self-awareness of strengths and weaknesses through self-reflection are important for life-long learning and development. The aim of this study was to assess the alignment in third-year undergraduate pharmacy students' self-reflections of their objective structured clinical exam (OSCE) performance to their actual OSCE scores and explore the most common aspects students reflected on as markers of perceived performance. METHODS: Students completed a three-station OSCE and a written self-reflection about their performance. These reflections were coded using a latent pattern content analysis, with categories defined as "doing well (≥ 50% on exam)" and "not doing well (< 50% on exam)" and compared to their actual OSCE exam scores, to determine the degree of alignment. RESULTS: Two hundred sixty-nine students completed the OSCE and reflection. Students had a low degree of alignment between their self-reflections and actual OSCE performance. Low alignment was overwhelmingly prevalent and significant in high-achieving students with OSCE scores of ≥90%. Most common aspects students reflected on as indicators of performance were finishing on time and communicating effectively. High-achieving students reflected on aspects such as empathy, systematic questioning, and patient teach-back as aspects of their performance. CONCLUSIONS: Student reflections on exam performance do not align with their actual performance, particularly amongst the high-achieving students. High-achieving students were more aware of the different aspects that affected their performance. To ensure that high-achieving students are aware of their strengths, educators should provide more targeted feedback mechanisms and positive reassurances to help these students become more confident in their decision-making skills.

Assessment of Antimicrobial Stewardship through objective structured clinical examination in pharmacy education.

OBJECTIVES: To describe the implementation and assess whether an objective structured clinical examination (OSCE) is a viable assessment tool for testing Antimicrobial Stewardship (AMS) principles. METHODS: A three-station OSCE set in a hospital and community pharmacy was designed and mapped to the World Health Organisation's AMS intervention practical guide. This OSCE comprised 39 unique cases and was implemented across two campuses (Malaysia and Australia) at one institute. Stations were 8 min long and consisted of problem-solving and applying AMS principles to drug therapy management (Station 1), counselling on key antimicrobials (Station 2) or managing infectious diseases in primary care (Station 3). Primary outcome measure to assess viability was the proportion of students who were able to pass each case. KEY FINDINGS: Other than three cases with pass rates of 50, 52.8 and 66. 7%, all cases had pass rates of 75% or more. Students were most confident with referral to medical practitioner cases and switching from intravenous to oral or empirical to directed therapy. CONCLUSIONS: An AMS-based OSCE is a viable assessment tool in pharmacy education. Further research should explore whether similar assessments can help improve students' confidence at recognising opportunities for AMS intervention in the workplace.

Students' and Examiners' Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic.

Objective Structured Clinical Examinations (OSCEs) are routinely used in healthcare education programs. Traditionally, students undertake OSCEs as face-to-face interactions to assess competency in soft skills. Due to physical distancing restrictions during COVID-19, alternative methods were required. This study utilized a mixed-method design (online survey and interviews) to evaluate second-year pharmacy students' and examiners' experiences of their first virtual OSCEs in Australia. A total of 196 students completed their first virtual OSCE in June 2020 of which 190 students completed the online survey. However, out of the 190 students, only 88% (n = 167) consented to the use of the data from their online survey. A further 10 students and 12 examiners were interviewed. Fifty-five students (33%) who participated in the online survey strongly agreed or agreed that they preferred the virtual experience to face-to-face OSCEs while 44% (n = 73) neither agreed nor disagreed. Only 20% (n = 33) felt more anxious with the virtual OSCEs. Additionally, thematic analysis found non-verbal communication as a barrier during the OSCE. Positive aspects about virtual OSCEs included flexibility, decreased levels of anxiety and relevance with emerging telehealth practice. The need for remote online delivery of assessments saw innovative ways of undertaking OSCEs and an opportunity to mimic telehealth. While students and examiners embraced the virtual OSCE process, face-to-face OSCEs were still considered important and irreplaceable. Future opportunities for OSCEs to be delivered both face-to-face and virtually should be considered.

Investigating the prevalence of diabetic complications in overweight/obese patients: a study in a tertiary hospital in Malaysia.

Background: In Malaysia, although diabetes accounts for more than 70% of all deaths, it is unclear how it relates to BMI and diabetic complications. This study aimed to investigate the prevalence of obesity and diabetic complications among diabetic patients in Malaysia. Materials and methods: A cross-sectional study using an existing clinical registry was performed from 1 January 2020 to 31 December 2020 at Hospital Serdang, Malaysia. Adult patients with type 2 diabetes mellitus had their medical records examined for disease complications, as reported by the patient at first contact with the DMTAC pharmacist. Results: The study comprised a total of 495 participants with an average HbA1c of 10.5%. About 91% (n = 451) of the 495 patients were obese/overweight. Around 37.8% (n = 187) of diabetic patients are between the ages of 50 and 59, and 59% (n = 292) have had diabetes for less than 10 years. A total of 8.5% (n = 42) and 9.7% (n = 48) consume alcohol and smoke, respectively. Around 29.9% (n = 148) had one other comorbidity (hypertension or dyslipidemia), and 63.4% (n = 314) had two comorbidities. Regarding the prevalence of complications, there were 18.9% (n = 94) who had myocardial infarction, 11.1% (n = 55) who had stroke, and 9% (n = 45) who had CKD. Age (adjusted OR = 1.03; 95% CI 1.00 to 1.07; p = 0.041) and hypertension (adjusted OR = 4.06; 95% CI 1.21 to 13.60; p = 0.023) were significantly related with the prevalence of complications in patients with diabetes. Conclusion: In our study, a BMI of more than 23 kg/m2 (obese/overweight) does not seem to be associated with the prevalence of complications. Age and hypertension, on the other hand, appear to be strong risk predictors of the incidence of complications. With the understanding of the recent outlook on diabetes, it is recommended that public education on the targeted population should be encouraged to negate these complications. Supplementary Information: The online version contains supplementary material available at 10.1007/s13410-022-01131-x.

Using Simulation to Teach Methods for Improving Patient Literacy about Medicines.

Pharmacists have a role in educating patients on the self-management of their medications, using accurate medicines lists. Thus, pharmacy students need to be adequately trained and assessed in health-literacy skills to be competent for future patient-education consultations using medicines lists. Performance-based assessments using patient simulation are often utilized to examine students' competence in clinical knowledge and communication skills. Due to COVID-19, education systems changed to remote online delivery utilizing video conferencing platforms (i.e., ZoomTM), which proved challenging for performance-based assessments. These challenges include difficulty in observing non-verbal cues over ZoomTM and not having adequate internet access. Adaptations to reduce reliance on internet access were made where students submitted a video-recording task wherein they educated a simulated patient on a medicines list, under lockdown restrictions during the pandemic. A total of 304 submissions were received where students performed the role-play with a simulated patient, such as their family members, housemates or peers either at home in person or via ZoomTM. Although it was not an original goal of the task, the collaborative effort between pharmacy instructors, students and the public helped increase awareness of medicines lists through this task.