Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping.

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Preceptor Perceptions of Pharmacy Student Performance Before and After a Curriculum Transformation.

Objective. To explore preceptors' perceptions about the performance of undergraduate pharmacy students during experiential placements in Australia, before and after curricular transformation.Methods. Using a semi-structured approach, we interviewed 26 preceptors who had recently supervised students who took part in the transformed curriculum and students from the previous curriculum. A directed content analysis approach was used to analyze the transcripts.Results. Preceptors described students from the transformed curriculum as having improved professional skills, behaviors, and attitudes and as having an increased ability to perform clinical activities compared to students of the previous curriculum. Preceptors also perceived that students in the transformed curriculum had improved clinical knowledge and knowledge application. They less frequently expressed that students in the transformed curriculum had lower-than-expected knowledge levels.Conclusion. The results of this study suggest that curricular transformation with a focus on skill-based and active learning can improve the performance of pharmacy students in terms of their professional behaviors and attitudes, skills, knowledge, and clinical abilities, as perceived by preceptors.

Experiential Education in the Time of COVID-19: An Australian Experience.

The pharmacy degree at Monash University is a four year undergraduate Bachelor of Pharmacy with Honours (BPharm(Hons)), followed by a one year Master of Pharmacy incorporating a preregistration year of supervised practice, known in Australia as internship. The first cohort of the BPharm(Hons) will graduate in 2020. A key principle of the new curriculum was that clinical experiential placements should commence early in the curriculum, be fully integrated with the didactic curriculum, and prepare students for internship by developing their competence in performing useful and relevant tasks in the workplace. Entrustable Professional Activities (EPA) are used as the framework for their experiences.