RESUMO
An Objective Structured Practical Examination (OSPE) is an effective and robust, but resource-intensive, means of evaluating anatomical knowledge. Since most OSPEs employ short answer or fill-in-the-blank style questions, the format requires many people familiar with the content to mark the examinations. However, the increasing prevalence of online delivery for anatomy and physiology courses could result in students losing the OSPE practice that they would receive in face-to-face learning sessions. The purpose of this study was to test the accuracy of Decision Trees (DTs) in marking OSPE questions as a first step to creating an intelligent, online OSPE tutoring system. The study used the results of the winter 2020 semester final OSPE from McMaster University's anatomy and physiology course in the Faculty of Health Sciences (HTHSCI 2FF3/2LL3/1D06) as the data set. Ninety percent of the data set was used in a 10-fold validation algorithm to train a DT for each of the 54 questions. Each DT was comprised of unique words that appeared in correct, student-written answers. The remaining 10% of the data set was marked by the generated DTs. When the answers marked by the DT were compared to the answers marked by staff and faculty, the DT achieved an average accuracy of 94.49% across all 54 questions. This suggests that machine learning algorithms such as DTs are a highly effective option for OSPE grading and are suitable for the development of an intelligent, online OSPE tutoring system.
RESUMO
As post-secondary education migrates online, developing and evaluating new avenues for assessment in anatomy is paramount. Three-dimensional (3D) visualization technology is one area with the potential to augment or even replace resource-intensive cadaver use in anatomical education. This manuscript details the development of a smartphone application, entitled "Virtual Reality Bell-Ringer (VRBR)," capable of displaying monoscopic two-dimensional (2D) or stereoscopic 3D images with the use of an inexpensive cardboard headset for use in spot examinations. Cadaveric image use, creation, and pinning processes are explained, and the source code is provided. To validate this tool, this paper compares traditional laboratory-based spot examination assessment stations against those administered using the VRBR application to test anatomical knowledge. Participants (undergraduate, n = 38; graduate, n = 13) completed three spot examinations specific to their level of study, one in each of the modalities (2D, 3D, laboratory) as well as a mental rotation test (MRT), Stereo Fly stereotest, and cybersickness survey. Repeated measures ANCOVA suggested participants performed significantly better on laboratory and 3D stations compared to 2D stations. Moderate to severe cybersickness symptoms were reported by 63% of participants in at least one category while using the VRBR application. Highest reported symptoms included: eye strain, general discomfort, difficulty focusing, and difficulty concentrating. Overall, the VRBR application is a promising tool for its portability, affordability, and accessibility. Due to reported cybersickness and other technical limitations, the use of VRBR as an alternative to cadaveric specimens presents several challenges when testing anatomy knowledge that must be addressed before widespread adoption.
