Status of Entrustable Professional Activities (EPA) Implementation at Colleges of Osteopathic Medicine in the United States and Future Considerations.

CONTEXT: Competency-based medical education, developmental milestones for residency training, and the single graduate medical education (GME) accreditation system have emerged over the last decade, necessitating new ways to adequately prepare graduates to meet new standards in proficiency, including the 13 Core Entrustable Professional Activities (EPA) for Entering Residency. The American Association of Colleges of Osteopathic Medicine (AACOM) Entrustable Professional Activities (EPA) steering committee has implemented an information-gathering process to provide suggestions for supporting a variety of EPA-related implementation efforts at colleges of osteopathic medicine (COMs) across the country. OBJECTIVE: To review the status of EPA implementation at COMs nationally. METHODS: An explanatory mixed-methods design was used to guide information gathering and synthesis of a 41-question survey and interview feedback; the overarching premise of this design was to use qualitative data to build upon initial quantitative findings. This survey was delivered electronically through a link emailed to liaisons at each main, branch, and satellite campus of the 47 schools within the AACOM institutional database. After survey administration, follow-up structured interviews were conducted according to an 18-question script with a purposive sample of 16 institutions with EPA implementation levels ranging from "moderate implementation with reporting" to "full implementation with reporting." Post-interview, the interview notes were analyzed and results were aggregated for comparison with the original survey findings. RESULTS: Of the 47 schools surveyed, 42 responded (89.4%). To maintain uniformity in data coding and analysis, 36 of 47 (76.6%) of COMs with independently submitted survey responses were retained in the review. The majority of those respondents (23 of 36; 64%) indicated that their institution was above "somewhat knowledgeable" toward "expert" regarding knowledge of EPAs, but 23 (64%) also indicated "no confidence" or "somewhat confident" regarding EPA implementation. Postinterview results showed that the majority of schools were equally distributed across the "foundational implementation" (10; 28%), "slight implementation" (11; 31%), and "moderate implementation" (11, 31%) categories, with a few schools indicating "no implementation" (2; 5%) or "progressive implementation" (2; 5%). CONCLUSION: The results of this review indicate that most osteopathic medical schools are at the early stages of EPA implementation, with emphasis varying by program year in terms of the specific EPAs addressed. Many schools appear engaged in curricular change efforts that will support the advancement of EPA use within their institutions. Faculty development was identified as a continued critical need for a majority of institutions.

Relationship of Clinical Skills Performance in Medical School With COMLEX-USA Level 2-Performance Evaluation.

CONTEXT: Osteopathic medical students are required to pass the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Levels 1- and 2-Cognitive Evaluation and COMLEX-USA Level 2-Performance Evaluation (PE) to graduate. Predictors of COMLEX-USA cognitive exam performance are well established, but relatively few studies have explored factors associated with performance on the Level 2-PE. OBJECTIVE: To evaluate the relationship between school-based clinical competency assessments (written, simulation, and workplace evaluations) and Level 2-PE performance to ensure that these assessment efforts are effective and grounded in evidence to support student readiness. METHODS: School-based performance measures for 451 first-time takers of COMLEX-USA Level 2-PE were analyzed. A series of Mann-Whitney analyses were applied to compare 3 types of clinical performance assessments against COMLEX-USA Level 2-PE pass-fail performance: (1) internal objective structured clinical examinations (OSCE; average discipline-specific OSCE score and a comprehensive OSCE); (2) national examination performance (average clinical subject Comprehensive Osteopathic Medical Achievement Test, or COMAT, scores and Comprehensive Osteopathic Medical Self-Assessment Examination, or COMSAE, Phase 2 scores); and (3) a workplace-based clinical evaluation by preceptors. RESULTS: Students who passed the Level 2-PE had a significantly higher average discipline-specific OSCE score, COMSAE Phase 2 performance score, average COMAT score, and individual subject COMAT scores in all subjects except Psychiatry. Students who passed the Level 2-PE humanistic domain also had significantly higher scores in the communication skill component of the school's comprehensive OSCE. Similarly, students who passed the Level 2-PE biomedical domain had significantly higher scores in the data gathering and subjective, objective, assessment, and plan documentation scores on the internal OSCE. The size of these differences (η2) was relatively small. Student performance on the competency-based preceptor evaluation showed no significant relationship with pass-fail performance on the Level 2-PE. CONCLUSION: High-stakes OSCEs aligned with the level 2-PE blueprint are effective predictors of performance and are an important way to support student readiness for the Level 2-PE. Other assessments, such as subject-based COMATs and COMSAE Phase 2, add value to school-based assessments over workplace-based assessments.

Relationship of Preceptor Ad-hoc Entrustment Decisions to Students' Clinical Skills Performance.

This article was migrated. The article was marked as recommended. Using a workplace evaluation instrument that included clinical skills evaluation and supervisory EPA scale, this study provided empirical evidence of the relationship between preceptor evaluation of student clinical ability and their entrustment decisions at the medical student level, and advanced the understanding of general conditions that ad-hoc entrustment decision are based upon. A total of 4217 evaluations from 353 third-year medical students were included in the study. The analyses focused on entrustment decisions for seven EPAs (EPA1, 2, 3, 5, 6, 7, and 9) and the relationship with related clinical skill (e.g. history taking) performance. Pearson's correlations showed statistically significant and positive correlations within clerkships (r= 0.43-0.75) and across clerkships (r= 0.70-0.77). Analyses of individual entrustment level and skill rating also revealed that the lowest level of entrustment was predominantly associated with the rating of developing on corresponding skills, whereas the highest level of entrustment with the rating of approaching advanced/ advanced; the link between the three middle entrustment levels and particular skill ratings were less salient. Overall, these patterns of association between individual entrustment level and skill rating varied by EPAs as well as level of entrustment. Limitations of the current study were also discussed.

Entrustable Professional Activities for Entering Residency: Establishing Common Osteopathic Performance Standards in the Transition From Medical School to Residency.

Entrustable professional activities (EPAs) are measurable units of observable professional practice that can be entrusted to an unsupervised trainee. They were first introduced as a method of operationalizing competency-based medical education in graduate medical education. The American Association of Medical Colleges subsequently used EPAs to establish the core skills that medical students must be able to perform before they enter residency training. A recently published guide provides descriptions, guidelines, and rationale for implementing and assessing the core EPAs from an osteopathic approach. These osteopathically informed EPAs can allow schools to more appropriately assess a learner's whole-person approach to a patient, in alignment with the philosophy of the profession. As the single accreditation system for graduate medical education moves forward, it will be critical to integrate EPAs into osteopathic medical education to demonstrate entrustment of medical school graduates. The authors describe the collaborative process used to establish the osteopathic considerations added to EPAs and explores the challenges and opportunities for undergraduate osteopathic medical education.

Reliability and benefits of medical student peers in rating complex clinical skills.

BACKGROUND: Peers have been shown to be reliable raters in an objective structured clinical exam (OSCE). While the literature supports the use of senior level students in rating basic clinical skills, little is known about the reliability of peers of the same level of training in assessing complex clinical skills. AIM: To investigate the reliability of student peers of the same level of training in rating complex clinical skills in a geriatric OSCE. METHODS: Peer (n = 115) ratings were compared to faculty ratings using correlation and generalizability analysis. Paired Wilcoxon Signed-Rank test was used to establish peer learning benefits. RESULTS: Reliability of the OSCE was moderately strong (G-coefficient = 0.70) with strong correlations between peer and faculty ratings for the overall OSCE (r = 0.78, p = 0.001) and for each case (r = 0.70-0.85, p = 0.001). Generalizability analysis indicated that raters contributed minimally to score variance. Peers reported gaining learning benefits from the rating process. CONCLUSION: Peer raters of the same level of training can provide accurate ratings of complex clinical tasks and can serve as an important resource in assessing student performance in an OSCE. The peer review process can also serve an important role in enhancing student learning.

Developing osteopathic competencies in geriatrics for medical students.

CONTEXT: Minimum core competencies for allopathic medical students in the specialty area of geriatrics have been developed, comprising 26 competencies divided into 8 topical domains. These competencies are appropriate for osteopathic medical students, but they do not include competencies relating to osteopathic principles and practice (OPP) in geriatrics. There remains a need within the osteopathic profession to develop specialty-specific competencies specific to OPP. OBJECTIVE: To develop more specific and comprehensive minimum competencies in OPP for osteopathic medical students in the field of geriatric medicine. METHODS: The Delphi technique (a structured communication technique that uses a panel of experts to reach consensus) was adapted to generate new core competencies relating to OPP. Osteopathic geriatricians and members of the Educational Council on Osteopathic Principles (ECOP) of the American Association of Colleges of Osteopathic Medicine participated in a breakout session and 2 rounds of surveys. Proposed competencies with 80% of the participants ranking it as "very important and should be added as a competency" were retained. Participants were also asked if they agreed that competencies in OPP should include specific types of osteopathic manipulative treatment techniques for the elderly. RESULTS: Responses were received from 26 osteopathic physician experts: 17 ECOP members and 9 geriatricians. Fourteen proposed competencies were developed: 7 related to the existing topic domains, and 7 were placed into a new domain of osteopathic manipulative medicine (OMM). Six proposed competencies were retained, all of which were in the new OMM domain. These competencies related to using OMM for gait and balance assessment, knowing adverse events and contraindications of OMM, using OMM for pain relief and end-of-life care, using OMM in the hospital and nursing home setting, adapting OMM to fit an elderly individual, and using OMM to address limited range of motion and ability to perform activities of daily living. Thirteen of 22 participants (59%) agreed that OPP competencies should include specific osteopathic manipulative treatment techniques. CONCLUSIONS: The Delphi consensus building process was used to create 6 new minimum competencies in OMM for osteopathic medical students for the specialty area of geriatrics. Using data from this consensus, medical schools, residencies, and fellowships can create standards and expectations for osteopathic physicians regarding the best care of geriatric patients.