Taking More Society for Academic Emergency Medicine Practice Tests Does Not Lead to Improved National EM-M4 Exam Scores.

INTRODUCTION: Emergency medicine (EM) is a required clerkship for third-year medical students, and an elective EM acting internship (AI) is available to fourth-year students at our institution. The Society for Academic Emergency Medicine's (SAEM) National Emergency Medicine M4 Examination (EM-M4) is administered to students at the end of the EM AI experience. To prepare for the exam, students gain access to 23 practice tests available from SAEM. In this study we investigate the correlation between the number of practice tests taken and EM-M4 performance. METHODS: We collected data for EM-M4 and the US Medical Licensing Exam (USMLE) Step 2 Clinical Knowledge (CK) from students completing a MS4 EM clerkship in consecutive medical school classes from 2014-2017 at a private medical school. In addition, we collected data during the clerkship on the number of practice exams taken and whether a comprehensive practice exam was taken. We analyzed the study population three ways to determine whether the number of practice tests impacted final exam results: a binary distribution (1-11 or 12-23 tests taken); quaternary distribution (1-6, 7-12, 13-18, or 19-23 tests taken); and individual test variability (1,2,3,…22,23 tests taken). Complete data for 147 students was used for data analysis. RESULTS: The EM-M4 showed moderate (r = 0.49) correlations with USMLE Step 2 CK. There was no significant difference in EM-M4 performance in the binary analysis (P ≤ 0.09), the quaternary analysis (P ≤ 0.09), or the continuous variable analysis (P ≤ 0.52). Inclusion of a comprehensive practice test also did not correlate with EM-M4 performance (P ≤ 0.78). CONCLUSION: Degree of utilization of SAEM practice tests did not seem to correlate with performance on the EM-M4 examination at our institution. This could be due to many factors including that the question bank is composed of items that had poor item discrimination, possible inadequate coverage of EM curriculum, and/or use of alternative study methods. While further investigation is needed, if our conclusions prove generalizable, then using the SAEM practice tests is an extraneous cognitive load from a modality without proven benefit.

Development of content-valid technical skill assessment instruments for athletic taping skills.

BACKGROUND AND PURPOSE: The content validity of technical skill assessment instruments (TSAI) for the skills of athletic taping has not been reported. The purpose of this paper is to outline and present the process of content validation for nine TSAIs for athletic taping. Local and national validators were selected from Canadian Athletic Therapists' Association (CATA)-accredited athletic therapy (AT) programs to serve as content validators. METHODS: The process of content validation began with the creation of a detailed task analysis via mail and simple validation by local validators. Subsequently, the detailed task analysis was committee validated by a group of 10 validators from across Canada. Validators judged the importance and difficulty of each item, and a face-to-face committee-validator meeting established consensus on the majority of checklist items. Through a modified Ebel procedure, frequency distribution was used in the formation of the final TSAIs. RESULTS: Initial consensus for pre-taping assessment and technical skill performance items was low. Upon committee discussion and lack of agreement, the decision to remove pretaping assessment items was made. Initial results of importance and difficulty for athletic taping technical skills were low prior to the committee meeting. Results of importance and difficulty improved substantially following the face-to-face committee-validators meeting. Consensus on fail points improved from initial to final committee validation. CONCLUSION: The process of simple and committee validation can be seen as effective methods to establish the content validity of instruments used for the evaluation of athletic taping.