The End of Step 2 CS Should Be the Beginning of a New Approach to Clinical Skills Assessment.

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills (CS) in 2020 in response to the COVID-19 pandemic marked the end of a decades-long debate about the utility and value of the exam. For all its controversy, the implementation of Step 2 CS in 2004 brought about profound changes to the landscape of medical education, altering the curriculum and assessment practices of medical schools to ensure students were prepared to take and pass this licensing exam. Its elimination, while celebrated by some, is not without potential negative consequences. As the responsibility for assessing students' clinical skills shifts back to medical schools, educators must take care not to lose the ground they have gained in advancing clinical skills education. Instead, they need to innovate, collaborate, and share resources; hold themselves accountable; and ultimately rise to the challenge of ensuring that physicians have the necessary clinical skills to safely and effectively practice medicine.

Evolution of Clinical Skills Assessment in the USMLE: Looking to the Future After Step 2 CS Discontinuation.

The COVID-19 pandemic interrupted administration of the United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) exam in March 2020 due to public health concerns. As the scope and magnitude of the pandemic became clearer, the initial plans by the USMLE program's sponsoring organizations (NBME and Federation of State Medical Boards) to resume Step 2 CS in the short-term shifted to long-range plans to relaunch an exam that could harness technology and reduce infection risk. Insights about ongoing changes in undergraduate and graduate medical education and practice environments, coupled with challenges in delivering a transformed examination during a pandemic, led to the January 2021 decision to permanently discontinue Step 2 CS. Despite this, the USMLE program considers assessment of clinical skills to be critically important. The authors believe this decision will facilitate important advances in assessing clinical skills. Factors contributing to the decision included concerns about achieving desired goals within desired time frames; a review of enhancements to clinical skills training and assessment that have occurred since the launch of Step 2 CS in 2004; an opportunity to address safety and health concerns, including those related to examinee stress and wellness during a pandemic; a review of advances in the education, training, practice, and delivery of medicine; and a commitment to pursuing innovative assessments of clinical skills. USMLE program staff continue to seek input from varied stakeholders to shape and prioritize technological and methodological enhancements to guide development of clinical skills assessment. The USMLE program's continued exploration of constructs and methods by which communication skills, clinical reasoning, and physical examination may be better assessed within the remaining components of the exam provides opportunities for examinees, educators, regulators, the public, and other stakeholders to provide input.

Demographic Trends From 2005 to 2015 Among Physicians With Accreditation Council for Graduate Medical Education-Accredited Anesthesiology Training and Active Medical Licenses.

BACKGROUND: A temporary decrease in anesthesiology residency graduates that occurred around the turn of the millennium may have workforce implications. The aims of this study are to describe, between 2005 and 2015, (1) demographic changes in the workforce of physicians trained as anesthesiologists; (2) national and state densities of these physicians, as well as temporal changes in the densities; and (3) retention of medical licenses by mid- and later-career anesthesiologists. METHODS: Using records from the American Board of Anesthesiology and state medical and osteopathic boards, the numbers of licensed physicians aged 30-59 years who had completed Accreditation Council for Graduate Medical Education-accredited anesthesiology residency training were calculated cross-sectionally for 2005, 2010, and 2015. Demographic trends were then described. Census data were used to calculate national and state densities of licensed physicians. Individual longitudinal data were used to describe retention of medical licenses among older physicians. RESULTS: The number of licensed physicians trained as anesthesiologists aged 30-59 years increased from 32,644 in 2005 to 36,543 in 2010 and 36,624 in 2015, representing a national density of 1.10, 1.18, and 1.14 per 10,000 population in those years, respectively. The density of anesthesiologists among states ranged from 0.37 to 3.10 per 10,000 population. The age distribution differed across the years. For example, anesthesiologists aged 40-49 years predominated in 2005 (47%), but by 2015, only 31% of anesthesiologists were aged 40-49 years. The proportion of female anesthesiologists grew from 22% in 2005, to 24% in 2010, and to 28% in 2015, particularly among early-career anesthesiologists. For anesthesiologists with licenses in 2005, the number who still had active licenses in 2015 decreased by 9.6% for those aged 45-49 years, by 14.1% for those aged 50-54 years, and by 19.7% for those aged 55-59 years. CONCLUSIONS: The temporary decrease in anesthesiology residency graduates around the turn of the 21st century decreased the proportion of anesthesiologists who were midcareer as of 2015. This may affect the future availability of senior leaders as well as the future overall workforce in the specialty as older anesthesiologists retire. National efforts to plan for workforce needs should recognize the geographical variability in the distribution of anesthesiologists.

Three Decades Later: A Scoping Review of the Literature Related to the United States Medical Licensing Examination.

PURPOSE: To conduct a scoping review of the timing, scope, and purpose of literature related to the United States Medical Licensing Examination (USMLE) given the recent impetus to revise USMLE scoring. METHOD: The authors searched PubMed, PsycInfo, and ERIC for relevant articles published from 1990 to 2019. Articles selected for review were labeled as research or commentaries and coded by USMLE Step level, sample characteristics (e.g., year(s), single/multiple institutions), how scores were used (e.g., predictor/outcome/descriptor), and purpose (e.g., clarification/justification/description). RESULTS: Of the 741 articles meeting inclusion criteria, 636 were research and 105 were commentaries. Publication totals in the past 5 years exceeded those of the first 20 years.Step 1 was the sole focus of 38%, and included in 84%, of all publications. Approximately half of all research articles used scores as a predictor or outcome measure related to other curricular/assessment efforts, with a marked increase in the use of scores as predictors in the past 10 years. The overwhelming majority of studies were classified as descriptive in purpose. CONCLUSIONS: Nearly 30 years after the inception of the USMLE, aspirations for its predictive utility are rising faster than evidence supporting the manner in which the scores are used. A closer look is warranted to systematically review and analyze the contexts and purposes for which USMLE scores can productively be used. Future research should explore cognitive and noncognitive factors that can be used in conjunction with constrained use of USMLE results to inform evaluation of medical students and schools and to support the residency selection process.

Effects of Moving the United States Medical Licensing Examination Step 1 After Core Clerkships on Step 2 Clinical Knowledge Performance.

PURPOSE: To investigate the effect of a change in the United States Medical Licensing Examination Step 1 timing on Step 2 Clinical Knowledge (CK) scores, the effect of lag time on Step 2 CK performance, and the relationship of incoming Medical College Admission Test (MCAT) score to Step 2 CK performance pre and post change. METHOD: Four schools that moved Step 1 after core clerkships between academic years 2008-2009 and 2017-2018 were analyzed. Standard t tests were used to examine the change in Step 2 CK scores pre and post change. Tests of differences in proportions were used to evaluate whether Step 2 CK failure rates differed between curricular change groups. Linear regressions were used to examine the relationships between Step 2 CK performance, lag time and incoming MCAT score, and curricular change group. RESULTS: Step 2 CK performance did not change significantly (P = .20). Failure rates remained highly consistent (pre change: 1.83%; post change: 1.79%). The regression indicated that lag time had a significant effect on Step 2 CK performance, with scores declining with increasing lag time, with small but significant interaction effects between MCAT and Step 2 CK scores. Students with lower incoming MCAT scores tended to perform better on Step 2 CK when Step 1 was after clerkships. CONCLUSIONS: Moving Step 1 after core clerkships appears to have had no significant impact on Step 2 CK scores or failure rates, supporting the argument that such a change is noninferior to the traditional model. Students with lower MCAT scores benefit most from the change.

Can Better Selection Tools Help Us Achieve Our Diversity Goals in Postgraduate Medical Education? Comparing Use of USMLE Step 1 Scores and Situational Judgment Tests at 7 Surgical Residencies.

PURPOSE: Use of the United States Medical Licensing Examination (USMLE) for residency selection has been criticized for its inability to predict clinical performance and potential bias against underrepresented minorities (URMs). This study explored the impact of altering traditional USMLE cutoffs and adopting more evidence-based applicant screening tools on inclusion of URMs in the surgical residency selection process. METHOD: Multimethod job analyses were conducted at 7 U.S. general surgical residency programs during the 2018-2019 application cycle to gather validity evidence for developing selection assessments. Unique situational judgment tests (SJTs) and scoring algorithms were created to assess applicant competencies and fit. Programs lowered their traditional USMLE Step 1 cutoffs and invited candidates to take their unique SJT. URM status (woman, racial/ethnic minority) of candidates who would have been considered for interview using traditional USMLE Step 1 cutoffs was compared with the candidate pool considered based on SJT performance. RESULTS: A total of 2,742 general surgery applicants were invited to take an online SJT by at least 1 of the 7 programs. Approximately 35% of applicants who were invited to take the SJT would not have met traditional USMLE Step 1 cutoffs. Comparison of USMLE-driven versus SJT-driven assessment results demonstrated statistically different percentages of URMs recommended, and including the SJT allowed an average of 8% more URMs offered an interview invitation (P < .01). CONCLUSIONS: Reliance on USMLE Step 1 as a primary screening tool precludes URMs from being considered for residency positions at higher rate than non-URMs. Developing screening tools to measure a wider array of candidate competencies can help create a more equitable surgical workforce.

The growth of teledermatology: Expanding to reach the underserved.

The regulation of telemedicine in the United States is evolving, with new legislation expanding reimbursement and cross-state licensing capabilities. As telemedicine grows, communities with limited access to traditional dermatologic care may find a solution in teledermatology. A search of the medical literature and online health care law resources published within the past decade was performed to assess the current status of telemedicine availability, health record integration and security, reimbursement policy, and licensure requirements in the United States, with a focus on teledermatology. The majority of states have implemented policies requiring private insurance coverage. Medicaid reimburses some form of telemedicine in all states but restricts which modalities can be used and by which specialties. Medicare places the heaviest limitations on telemedicine coverage. Twenty-four states and Guam are members of the Interstate Medical Licensure Compact (IMLC), and 27 states offer alternative cross-state practice options. With the advent of publicly and privately funded programs, volunteer efforts, and mobile applications, teledermatology is more readily available to rural and underserved communities.

Peer assisted feedback: a successful approach for providing feedback on United States Medical Licensing Exam-style clinical skills exam notes in the United States.

PURPOSE: Peer assisted learning (PAL) promotes the development of communication, facilitates improvement of clinical skills, and provides feedback to learners. We utilized PAL as a conceptual framework to explore the feasibility of peer-assisted feedback (PAF) to improve note writing skills without requiring faculty time. The aim was to assess if peer assisted learning was a successful method to provide feedback on USMLE-style clinical skills exam notes by using student feedback on a survey in the United States. METHODS: The University of Florida College of Medicine administers clinical skills examinations (CSEs) includes USMLE-like note-writing. PAL, where students support the learning of peers, was utilized as an alternative to faculty feedback. Second (MS2) and third (MS3) year medical students taking CSEs participated in faculty-run note-grading sessions immediately after testing, which included explanations of grading rubrics and the feedback process. Students graded an anonymized peer's notes. The graded material was then forwarded anonymously to its student author to review. Students were surveyed on perceived ability to provide feedback and benefits derived from PAF using a Likert scale (1-6) and open-ended comments during the 2017-2018 academic year. RESULTS: Students felt generally positively about the activity with mean score for items related to educational value of 4.49 for MS2 and 5.11 for MS3 out of 6. MS3s perceived peer feedback as constructive, felt benefit from evaluating other's notes, and felt the exercise would improve their future notes. While still positive, MS2 students gave comparatively lower scores than the MS2 students. CONCLUSION: PAF was a successful method to provide feedback on student CSE notes, especially for MS3s. MS2s commented that although they learned during the process, they might be more invested in improving their note writing as they approach their own USMLE exam.

Investigation of answer changes on the USMLE® Step 2 Clinical Knowledge examination.

BACKGROUND: Examinees often believe that changing answers will lower their scores; however, empirical studies suggest that allowing examinees to change responses may improve their performance in classroom assessments. To date, no studies have been able to examine answer changes during large scale professional credentialing or licensing examinations. METHODS: In this study, we expand the research on answer changes by analyzing responses from 27,830 examinees who completed the Step 2 Clinical Knowledge (CK) examination between August of 2015 and August of 2016. RESULTS: The results showed that although 68% of examinees changed at least one item, the overall average number of changes was small. Among the examinees who changed answers, approximately 45% increased their scores and approximately 28% decreased their scores. On average, examinees spent shortest time on the item changes from wrong to right and they were more likely to change their scores from wrong to right than right to wrong. CONCLUSIONS: Consistent with previous studies, these findings support the beneficial effects of answer changes in high-stakes medical examinations and suggest that examinees who are overly cautious about changing answers may put themselves at a disadvantage.