A Plea to Reassess the Role of United States Medical Licensing Examination Step 1 Scores in Residency Selection.

The three-step United States Medical Licensing Examination (USMLE) was developed by the National Board of Medical Examiners and the Federation of State Medical Boards to provide medical licensing authorities a uniform evaluation system on which to base licensure. The test results appear to be a good measure of content knowledge and a reasonable predictor of performance on subsequent in-training and certification exams. Nonetheless, it is disconcerting that the test preoccupies so much of students' attention with attendant substantial costs (in time and money) and mental and emotional anguish. There is an increasingly pervasive practice of using the USMLE score, especially the Step 1 component, to screen applicants for residency. This is despite the fact that the test was not designed to be a primary determinant of the likelihood of success in residency. Further, relying on Step 1 scores to filter large numbers of applications has unintended consequences for students and undergraduate medical education curricula. There are many other factors likely to be equally or more predictable of performance during residency. The authors strongly recommend a move away from using test scores alone in the applicant screening process and toward a more holistic evaluation of the skills, attributes, and behaviors sought in future health care providers. They urge more rigorous study of the characteristics of students that predict success in residency, better assessment tools for competencies beyond those assessed by Step 1 that are relevant to success, and nationally comparable measures from those assessments that are easy to interpret and apply.

Commentary: Balancing responsibility to patients and responsibility to aspiring physicians with disabilities.

In 2008, Congress amended the Americans with Disabilities Act (ADA) to relax court-imposed limitations on evidence required to warrant protection under the ADA. Since passage of the ADA in 1990, medicine has focused not on evaluating the types of accommodations that would best balance the interests of individuals with disabilities, institutions, and patients but, rather, on the question of whether individuals seeking protection under the law qualify for disability accommodations at all. The medical profession should refocus on the nature of accommodations provided to those with disabilities. In doing so, the intent to support disabled persons seeking careers in medicine must be balanced with ethical obligations to protect patient welfare. Medical schools, graduate medical education programs, licensing and certifying authorities, and assessment organizations should work together to establish evidence-based minimum criteria for the physical and cognitive capabilities required of every physician.

Licensing examinations in North America: is external audit valuable?

The United States and Canada both have long-standing, highly developed national systems of assessment for medical-licensure based outside the institutions of medical education. This commentary reviews those programs and explores some of the reasons for their implementation and retention for nearly a century. The North American experience may be relevant to dialog about national or European assessments for medical practice.

From defending the walls to improving global medical education: fifty years of collaboration between the ECFMG and the NBME.

The author outlines the intertwining roles of the Educational Commission for Foreign Medical Graduates (ECFMG), which is celebrating its 50th anniversary in 2006, and the National Board of Medical Examiners (NBME) in meeting needs for assessment of international medical graduates. Both organizations had early histories focused on a protective role: ensuring that only the most qualified foreign-trained doctors could train or practice in the United States. The two organizations have interacted throughout the ECFMG's 50-year history to improve the assessment of internationally trained doctors. As both the ECFMG and the NBME have matured, their missions have expanded to include improvement of medical education and assessment around the world. Much of the success of each organization in fulfilling its mission can be attributed to their close collaboration through the past 50 years.

Physician performance and assessment and their effect on continuing medical education and continuing professional development.

This article has three key points. The first proposes and illustrates a model for planning effective continuing medical education (CME) and continuing professional development (CPD) and how assessment might fit into it. The second reviews major trends in assessment, particularly with regard to regulation and CME. The third addresses challenges for CME and CPD.

[Transposition of an American-designed comprehensive medical student examination within the framework of the forthcoming French nationwide comprehensive examination. A preliminary study]. / Intérêt de transposer un examen américain dans le cadre du futur examen national classant validant français. Etude préliminaire.

Medical training is undergoing extensive revision in France. A nationwide comprehensive clinical competency examination will be administered for the first time in 2004, relying exclusively on essay-questions. Unfortunately, these questions have psychometric shortcomings, particularly their typically low reliability. High score reliability is mandatory in a high-stakes context. The National Board of Medical Examiners-designed multiple choice-questions (MCQ) are well adapted to assess clinical competency with a high reliability score. The purpose of this study was to test the hypothesis that French medical students could take an American-designed and French-adapted comprehensive clinical knowledge examination with this MCQ format. Two hundred and eighty five French students, from four Medical Schools across France, took an examination composed of 200 MCQs under standardized conditions. Their scores were compared with those of American students. This examination was found assess French students' clinical knowledge with a high level of reliability. French students' scores were slightly lower than those of American students, mostly due to a lack of familiarity with this particular item format, and a lower motivational level. Another study is being designed, with a larger group, to address some of the shortcomings of the initial study. If these preliminary results are replicated, the MCQ format might be a more defendable and sensible alternative to the proposed essay questions.

Assessing medical students' clinical sciences knowledge in France: a collaboration between the NBME and a consortium of French medical schools.

PURPOSE: The French government, as part of medical education reforms, has affirmed that an examination program for national residency selection will be implemented by 2004. The purpose of this study was to develop a French multiple-choice (MC) examination using the National Board of Medical Examiners' (NBME) expertise and materials. METHOD: The Evaluation Standardisée du Second Cycle (ESSC), a four-hour clinical sciences examination, was administered in January 2002 to 285 medical students at four university test sites in France. The ESSC had 200 translated and adapted MC items selected from the Comprehensive Clinical Sciences Examination (CCSE), an NBME subject test. RESULTS: Less than 10% of the ESSC items were rejected as inappropriate to French practice. Also, the distributions of ESSC item characteristics were similar to those reported with the CCSE. The ESSC also appeared to be very well targeted to examinees' proficiencies and yielded a reliability coefficient of.91. However, because of a higher word count, the ESSC did show evidence of speededness. Regarding overall performance, the mean proficiency estimate for French examinees was about 0.4 SD below that of a CCSE population. CONCLUSIONS: This study provides strong evidence for the usefulness of the model adopted in this first collaborative effort between the NBME and a consortium of French medical schools. Overall, the performance of French students was comparable to that of CCSE students, which was encouraging given the differences in motivation and the speeded nature of the French test. A second phase with the participation of larger numbers of French medical schools and students is being planned.