UGRC 2021 recommendations on GME transition: pros and cons, opportunities and limitations.

The Coalition for Physician Accountability's Undergraduate Medical Education-Graduate Medical Education (UME-GME) Review Committee (UGRC): Recommendations for Comprehensive Improvement of the UME-GME Transition final report includes a total of 34 recommendations and outlines opportunities to transform the current processes of learner transition from a US-based MD- or DO-granting medical school or international medical education pathway into residency training in the United States. This review provides a reflection on the recommendations from the authors, all members of the UGRC, describing the pros and cons and the opportunities and limitations, in the hopes that they might inspire readers to dig deeper into the report and contribute to meaningful improvements to the current transition. The UGRC Recommendations highlight the many opportunities for improvement in the UME-to-GME transition. They are built on the connection to the system of education and formation of physicians to a more just healthcare system, with attention to diversity, equity, and inclusion to improve health disparities and to the quality of care that patients receive. However, there are justifiable concerns about changes that are not fully understood or that could potentially lead to unintentional consequences. This analysis, reached through author consensus, considers the pros and cons in the potential application of the UGRC Recommendations to improve the UME-to-GME transition. Further debate and discussion are warranted, without undue delay, all with the intention to continue to improve the education of tomorrow's physicians and the care for the patients who we have the privilege to serve.

CATALYST: Piloting a Longitudinal Assessment and Learning Program for Board Recertification and Continuous Professional Development.

CONTEXT: As a proposed alternative to the traditional recertification examination, CATALYST is a longitudinal formative assessment platform created on cognitive learning principles. CATALYST was designed by the National Board of Osteopathic Medical Examiners to encourage more complex and durable practice-relevant learning and demonstration of ongoing competencies. OBJECTIVE: To investigate the value of the CATALYST platform using board diplomates' subjective feedback and comparison of performance on CATALYST questions with performance on board examinations. METHODS: Diplomates from 3 osteopathic specialty boards (the American Osteopathic Board of Internal Medicine, the American Osteopathic Board of Pediatrics, and the American Osteopathic Board of Obstetricians and Gynecologists) participated in this pilot study. Over the course of 16 weeks, participants were provided 2 questions per week via the CATALYST platform. An evaluation survey was emailed at the end of the study period to collect participants' feedback. Survey results and correlations of CATALYST performance with past or upcoming board examination scores were analyzed. RESULTS: A total of 196 diplomates completed the surveys, with 95% reporting that participation in the platform would help them stay current in their specialties and 91% reporting that participation would help them provide better care to their patients. For the AOBIM, a significant correlation was found between the number of CATALYST questions answered correctly and performance on the board examination (r=0.51, P<.001). The correlations found for the AOBP and AOBOG were not significant (r=0.197, P=.296, and r=0.370, P=.075, respectively). CONCLUSION: The CATALYST platform could offer valuable contributions to the board recertification process and to patient safety. Further investigations are being conducted on a new user-friendly platform.

CATALYST: Piloting a Longitudinal Assessment and Learning Program for Board Recertification and Continuous Professional Development.

Context: As a proposed alternative to the traditional recertification examination, CATALYST is a longitudinal formative assessment platform created on cognitive learning principles. CATALYST was designed by the National Board of Osteopathic Medical Examiners to encourage more complex and durable practice-relevant learning and demonstration of ongoing competencies. Objective: To investigate the value of the CATALYST platform using board diplomates' subjective feedback and comparison of performance on CATALYST questions with performance on board examinations. Methods: Diplomates from 3 osteopathic specialty boards (the American Osteopathic Board of Internal Medicine, the American Osteopathic Board of Pediatrics, and the American Osteopathic Board of Obstetricians and Gynecologists) participated in this pilot study. Over the course of 16 weeks, participants were provided 2 questions per week via the CATALYST platform. An evaluation survey was emailed at the end of the study period to collect participants' feedback. Survey results and correlations of CATALYST performance with past or upcoming board examination scores were analyzed. Results: A total of 196 diplomates completed the surveys, with 95% reporting that participation in the platform would help them stay current in their specialties and 91% reporting that participation would help them provide better care to their patients. For the AOBIM, a significant correlation was found between the number of CATALYST questions answered correctly and performance on the board examination (r=0.51, P<.001). The correlations found for the AOBP and AOBOG were not significant (r=0.197, P=.296, and r=0.370, P=.075, respectively). Conclusion: The CATALYST platform could offer valuable contributions to the board recertification process and to patient safety. Further investigations are being conducted on a new user-friendly platform.

2019 United States Osteopathic Medical Regulatory Summit: Consensus, Recommendations, and Next Steps in Defining Osteopathic Distinctiveness.

BACKGROUND: Osteopathic distinctiveness is a result of professional education, identity formation, training, credentialing, and qualifications. With the advancement of a single graduate medical education (GME) accreditation system and the continued growth of the osteopathic medical profession, osteopathic distinctiveness and professional identity are seen as lacking clarity and pose a challenge. SUMMIT: To achieve consensus on a succinct definition of osteopathic distinctiveness and to identify steps to more clearly define and advance that distinctiveness, particularly in professional self-regulation, a representative group of osteopathic medical students, residents, physicians, and members of the licensing, GME, and undergraduate medical education (UME) communities convened the 2019 United States Osteopathic Medical Regulatory Summit in February 2019. Key features of osteopathic distinctiveness were discussed. Growth in the profession; changes in health care delivery, technology, and demographics within the profession and patient communities; and associated challenges and opportunities for osteopathic medical practice and patients were considered. CONSENSUS: Osteopathic medicine is a distinctive practice that brings unique, added value to patients, the public, and the health care community at large. A universal definition and common understanding of that distinctiveness is lacking. Efforts to unify messaging that defines osteopathic distinctiveness, to align the distinctive elements of osteopathic medical education and professional self-regulation across a continuum, and to advance research on care and educational program outcomes are critical to the future of the osteopathic medical profession. RECOMMENDATIONS: (1) Convene a task force of groups represented at the Summit to develop a succinct and consistent message defining osteopathic distinctiveness. (2) Demonstrate uniqueness of the profession through research demonstrating efficacy of care and patient outcomes, adding to the public good. (3) Harmonize GME and UME by beginning to align entrustable professional activities with UME milestones. (4) Convene representatives from osteopathic specialty colleges and certification boards to define curricular elements across GME, certification, and osteopathic continuous certification. (5) Build on the Project in Osteopathic Medical Education and Empathy study.

An Investigation of the Relationship Between COMLEX-USA Licensure Examination Performance and State Licensing Board Disciplinary Actions.

PURPOSE: Passing the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) serves as a licensing requirement, yet there is limited understanding between this high-stakes exam and performance outcomes. This study examined the relationship between COMLEX-USA scores and disciplinary actions received by osteopathic physicians. METHOD: Data for osteopathic physicians (N = 26,383) who graduated from medical school between 2004 and 2013 were analyzed using multinomial logistic regression to assess the relationship between COMLEX-USA scores and placement into one of 3 disciplinary action categories relative to no action received, controlling for years in practice and gender. RESULTS: Less than 1% of physicians in this study (n = 187) had a disciplinary action(s). Controlling for all COMLEX-USA levels, years in practice, and gender, higher Level 3 scores were associated with significant decreased odds for all action categories: revoked licensed (odds ratio [OR] = 0.51, 95% confidence interval [CI] 0.36, 0.72; P < .001), imposed limitations to practice (OR = 0.59, 95% CI 0.41, 0.84; P < .01), and other action imposed (OR = 0.48, 95% CI 0.33, 0.69; P < .001), relative to not receiving an action. In these same models, higher Level 2 Performance Evaluation Biomedical/Biomechanical Domain scores decreased the odds for an action that revoked a license (OR = 0.75, 95% CI 0.58, 0.98; P < .05) and imposed limitations to practice (OR = 0.64, 95% CI 0.49, 0.84; P < .001). CONCLUSIONS: These findings provide evidence that the COMLEX-USA delivers useful information regarding the likelihood of a practitioner receiving state board disciplinary actions.

Association Between Performance on COMLEX-USA and the American College of Osteopathic Family Physicians In-Service Examination.

BACKGROUND: The primary goal of residency programs is to select and educate qualified candidates to become competent physicians. Program directors often use performance on licensure examinations to evaluate the ability of candidates during the resident application process. The American College of Osteopathic Family Physicians (ACOFP) administers an in-service examination (ISE) to residents annually. There are few prior studies of the relationship between the Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) series and formative assessments of residents in training. OBJECTIVE: We explored the relationship between performance on COMLEX-USA and the ACOFP in-service examination to offer support on the use of licensing examinations in resident selection. METHODS: In 2016, performance data from the COMLEX-USA and the ISE were matched for 3 resident cohorts (2011-2013, inclusive; N = 1384). Correlations were calculated to examine the relationship between COMLEX-USA and ISE scores. Multiple linear regression models were used to determine if performance on COMLEX-USA significantly predicted third-year ISE (ISE-3) scores. RESULTS: Findings indicated that correlations among performance on COMLEX-USA and ISE were statistically significant (all P < .001), and there was strong intercorrelation between COMLEX-USA Level 3 and ISE-1 performance (r = 0.57, P < .001). Performance on the COMLEX-USA Levels 1 and 2-Cognitive Examination significantly predicted performance on the ISE-3 (F(2,1381) = 228.8, P < .001). CONCLUSIONS: The results support using COMLEX-USA as a part of resident selection in family medicine. Additionally, program directors may use performance on COMLEX-USA to predict success on the ISE-3.

Evidence-Based Redesign of the COMLEX-USA Series.

To ensure that the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) reflects the evolving practice of osteopathic medicine, the National Board of Osteopathic Medical Examiners has developed new content and format specifications for an enhanced, competency-based examination program to be implemented with COMLEX-USA Level 3 in 2018. This article summarizes the evidence-based design processes that served as the foundation for blueprint development and the evidence supporting its validity. An overview is provided of the blueprint's 2 dimensions: Competency Domains and Clinical Presentations. The authors focus on the evidence that supports interpretation of test scores for the primary and intended purpose of COMLEX-USA, which is osteopathic physician licensure. Important secondary uses and the educational and catalytic effect of assessments are also described. This article concludes with the National Board of Osteopathic Medical Examiners' plans to ensure that the COMLEX-USA series remains current and meets the needs of its stakeholders-the patients who seek care from osteopathic physicians.

The Use of COMLEX-USA and USMLE for Residency Applicant Selection.

BACKGROUND: The Comprehensive Osteopathic Medical Licensing Examination (COMLEX-USA) and the United States Medical Licensing Examination (USMLE) are recognized by all state medical licensing boards in the United States, and the Federation of State Medical Boards has supported the validity of both examinations for medical licensure. Many osteopathic medical students take both examinations. OBJECTIVE: The purpose of this study was to investigate performance on COMLEX-USA Level 1 and USMLE Step 1 of students from colleges of osteopathic medicine where the majority of students took both examinations. METHODS: Data were collected on the entering classes of 2010 and 2011. Relationships between the COMLEX-USA Level 1 and the USMLE Step 1 were quantified using Pearson correlations. The correlation between outcomes on the 2 examinations was evaluated using the phi coefficient. A contingency table was constructed to look at first-attempt outcomes (pass/fail). RESULTS: Data for 2010 and 2011 were collected from 3 osteopathic medical schools, with 795 of 914 students (87%) taking both examinations. The correlation between first-attempt COMLEX-USA Level 1 and USMLE Step 1 scores was statistically significant across and within all 3 schools. The overall correlation was r(795) = 0.84 (P < .001). Pass/fail status on the 2 examinations was moderately correlated (φ = 0.39, P < .01). CONCLUSIONS: Our study found a strong association between COMLEX Level 1 and USMLE Step 1 performance. Additional studies to accurately compare scores on these examinations are warranted.

Preparation Strategies of Osteopathic Medical Students for the COMLEX-USA Level 2-PE.

CONTEXT: Since 2002, osteopathic medical schools have made curricular changes to further enhance the clinical skills of their students, to prepare them for residency training, and to pass the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE). OBJECTIVE: To report how students at osteopathic medical schools prepare for the COMLEX-USA Level 2-PE, and to investigate the effect of these techniques on examination performance. METHODS: A standardized survey was given to students before the beginning of their examination to assess the preparation of osteopathic medical students for the COMLEX-USA Level 2-PE, such as coursework, orientation materials, and standardized patient (SP) encounters. Surveys that were completed by first-time test takers during the 2013-2014 and 2014-2015 test cycles were included in this study. RESULTS: Of 9120 surveys administered, 8733 were completed, achieving a response rate of 95.8%. Of those 8733 respondents, 8706 students (99.7%) reported having SP encounters during the first and second year of medical school, and 7379 (84%) reported having at least 1 SP encounter in years 3 and 4. Of 8733 students, 6079 (70%) reported receiving feedback from an osteopathic physician on their SP encounters, and 6049 (69%) and 6253 (72%) reported having viewed the COMLEX-USA Level 2-PE orientation video online and having read the examination's orientation guide, respectively. The largest difference in preparation between students who passed the COMLEX-USA Level 2-PE and students who did not was a prerequisite SP examination at their school, with 5574 students (68.9%) who passed reporting having participated compared with 364 students (56.5%) who failed. None of the differences in clinical skills training and test preparation was associated with statistically significant differences in pass or fail status. CONCLUSION: Osteopathic medical students use a variety of methods to enhance their clinical skills in preparation for the COMLEX-USA Level 2-PE, with universal use of SP programs since the COMLEX-USA Level 2-PE was implemented in 2004. Educators should continue to foster ways to develop students' clinical skills that reflect new advances in education and assessment to ensure that future osteopathic physicians can demonstrate competency in fundamental clinical skills before beginning postgraduate training.

Relationship between COMLEX-USA scores and performance on the American Osteopathic Board of Emergency Medicine Part I certifying examination.

CONTEXT: Few studies have investigated how well scores from the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) series predict resident outcomes, such as performance on board certification examinations. OBJECTIVES: To determine how well COMLEX-USA predicts performance on the American Osteopathic Board of Emergency Medicine (AOBEM) Part I certification examination. METHODS: The target study population was first-time examinees who took AOBEM Part I in 2011 and 2012 with matched performances on COMLEX-USA Level 1, Level 2-Cognitive Evaluation (CE), and Level 3. Pearson correlations were computed between AOBEM Part I first-attempt scores and COMLEX-USA performances to measure the association between these examinations. Stepwise linear regression analysis was conducted to predict AOBEM Part I scores by the 3 COMLEX-USA scores. An independent t test was conducted to compare mean COMLEX-USA performances between candidates who passed and who failed AOBEM Part I, and a stepwise logistic regression analysis was used to predict the log-odds of passing AOBEM Part I on the basis of COMLEX-USA scores. RESULTS: Scores from AOBEM Part I had the highest correlation with COMLEX-USA Level 3 scores (.57) and slightly lower correlation with COMLEX-USA Level 2-CE scores (.53). The lowest correlation was between AOBEM Part I and COMLEX-USA Level 1 scores (.47). According to the stepwise regression model, COMLEX-USA Level 1 and Level 2-CE scores, which residency programs often use as selection criteria, together explained 30% of variance in AOBEM Part I scores. Adding Level 3 scores explained 37% of variance. The independent t test indicated that the 397 examinees passing AOBEM Part I performed significantly better than the 54 examinees failing AOBEM Part I in all 3 COMLEX-USA levels (P<.001 for all 3 levels). The logistic regression model showed that COMLEX-USA Level 1 and Level 3 scores predicted the log-odds of passing AOBEM Part I (P=.03 and P<.001, respectively). CONCLUSION: The present study empirically supported the predictive and discriminant validities of the COMLEX-USA series in relation to the AOBEM Part I certification examination. Although residency programs may use COMLEX-USA Level 1 and Level 2-CE scores as partial criteria in selecting residents, Level 3 scores, though typically not available at the time of application, are actually the most statistically related to performances on AOBEM Part I.

Using standardized patients to assess the communication skills of graduating physicians for the comprehensive osteopathic medical licensing examination (COMLEX) level 2-performance evaluation (level 2-PE).

BACKGROUND: Standardized patients can be trained to assess the communication and interpersonal skills of medical students and graduates. PURPOSE: The purpose of this study is to present data to support the psychometric adequacy of the communication ratings provided by standardized patients. METHODS: Using the data from testing of 3, 450 examinees over a 1-year period, a number of psychometric analyses were undertaken. These included a variance component analysis, the calculation of various validity coefficients, the comparison of communication ratings for select examinee cohorts and case characteristics, and the investigation of some potential sources of score invalidity. RESULTS: Communication skills scores are moderately correlated to other competencies (knowledge, skills) and may be influenced by candidate characteristics such as gender and English language proficiency. They are not dependant on the age of the examinees, the clinical case content, or the gender of the standardized patients. CONCLUSIONS: For a multistation assessment, a reasonably precise and valid estimate of a candidate's communication ability can be obtained from trained standardized patients.