The use of mixed reality technology for the objective assessment of clinical skills: a validation study.

BACKGROUND: Mixed Reality technology may provide many advantages over traditional teaching methods. Despite its potential, the technology has yet to be used for the formal assessment of clinical competency. This study sought to collect validity evidence and assess the feasibility of using the HoloLens 2 mixed reality headset for the conduct and augmentation of Objective Structured Clinical Examinations (OSCEs). METHODS: A prospective cohort study was conducted to compare the assessment of undergraduate medical students undertaking OSCEs via HoloLens 2 live (HLL) and recorded (HLR), and gold-standard in-person (IP) methods. An augmented mixed reality scenario was also assessed. RESULTS: Thirteen undergraduate participants completed a total of 65 OSCE stations. Overall inter-modality correlation was 0.81 (p = 0.01), 0.98 (p = 0.01) and 0.82 (p = 0.01) for IP vs. HLL, HLL vs. HLR and IP vs. HLR respectively. Skill based correlations for IP vs. HLR were assessed for history taking (0.82, p = 0.01), clinical examination (0.81, p = 0.01), procedural (0.88, p = 0.01) and clinical skills (0.92, p = 0.01), and assessment of a virtual mixed reality patient (0.74, p = 0.01). The HoloLens device was deemed to be usable and practical (Standard Usability Scale (SUS) score = 51.5), and the technology was thought to deliver greater flexibility and convenience, and have the potential to expand and enhance assessment opportunities. CONCLUSIONS: HoloLens 2 is comparable to traditional in-person examination of undergraduate medical students for both live and recorded assessments, and therefore is a valid and robust method for objectively assessing performance. The technology is in its infancy, and users need to develop confidence in its usability and reliability as an assessment tool. However, the potential to integrate additional functionality including holographic content, automated tracking and data analysis, and to facilitate remote assessment may allow the technology to enhance, expand and standardise examinations across a range of educational contexts.

Mixed Reality Enhanced Otolaryngology Case-Based Learning: A Randomized Educational Study.

INTRODUCTION: The COVID-19 pandemic has caused unprecedented disruptions to medical education. Education in medical specialties, such as otolaryngology faces multiple challenges, including reduced bedside and "hands-on" training opportunities at all levels. Educators are turning to technological advancements to deliver effective remote medical education. This study investigated the value of enhancing traditional remote case-based teaching with the HoloLens2™. METHODS: We present a randomized educational design study. All educational content, media, and learning outcomes were identical. Primary outcome measures included student performance as measured with pre- and post-intervention quizzes and student engagement as measured by a tally-mark system. Secondary outcome measures, collected using feedback questionnaires, included perceived enjoyment, engagement, and opinions regarding the educational role of this technology. RESULTS: The undergraduate medical students were randomized to either conventional or HoloLens2™ enhanced remote case-based teaching (n = 56). HoloLens2™ enhanced teaching improved student performance by an average of 3 marks of 15% (p < 0.001). It was engaging and encouraged questions 4-fold per session (p < 0.05) when compared to conventional remote case-based teaching. There was no significant difference in overall objective measurements of engagement. Students taught using HoloLens2™ agreed that the teaching was enjoyable, effective in concept demonstration, and encouraged engagement. CONCLUSIONS: Remote teaching has allowed for the continuation of medical education in uncertain times. Beyond COVID-19, we predict that there will be a paradigm shift toward remote learning as new technological advancements emerges. These novel technologies may prove invaluable in the future potentially enabling education to be delivered between different hospitals, universities, and even overseas. LEVEL OF EVIDENCE: NA Laryngoscope, 133:1606-1613, 2023.