Relationships Between the ACGME Resident and Faculty Surveys and Program Pass Rates on the ABIM Internal Medicine Certification Examination.

PURPOSE: The Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents since 2003, and faculty since 2012. Surveys are designed to assess program functioning and specify areas for improvement. The purpose of this study was to assess the association of the ACGME's resident and faculty surveys with residency-program-specific performance on the American Board of Internal Medicine (ABIM) certification exam. METHOD: Data were available from residents and faculty in 375 U.S. ACGME-accredited internal medicine programs from the 2012-2013, 2013-2014, and 2014-2015 academic years. Analysis of variance and correlations were used to examine the relationship between noncompliance with ACGME program requirements as assessed by the resident and faculty surveys, and ABIM program pass rates. RESULTS: Noncompliance reported on the resident and faculty surveys was highest for programs not meeting the ACGME program requirement of an 80% pass rate on the ABIM certification examination. This relationship was significant for overall noncompliance, both within the resident (P < .001) and faculty (P < .05) surveys, for many areas within the two surveys (correlations ranged between -.07 and -.25, and P values ranged between .20 and < .001), and for the highest levels of noncompliance across areas of the resident (P < .001) and faculty (P < .04) surveys. CONCLUSIONS: ACGME resident and faculty surveys were significantly associated with ABIM program pass rates, supporting the importance of these surveys within the ACGME's Next Accreditation System.

Assessment in the context of licensure and certification.

Over the past 25 years, three major forces have had a significant influence on licensure and certification: the shift in focus from educational process to educational outcomes, the increasing recognition of the need for learning and assessment throughout a physician's career, and the changes in technology and psychometrics that have opened new vistas for assessment. These forces have led to significant changes in assessment for licensure and certification. To respond to these forces, licensure and certification programs have improved the ways in which their examinations are constructed, scored, and delivered. In particular, we note the introduction of adaptive testing; automated item creation, scoring, and test assembly; assessment engineering; and data forensics. Licensure and certification programs have also expanded their repertoire of assessments with the rapid development and adoption of simulation and workplace-based assessment. Finally, they have invested in research intended to validate their programs in four ways: (a) the acceptability of the program to stakeholders, (b) the extent to which stakeholders are encouraged to learn and improve, (c) the extent to which there is a relationship between performance in the programs and external measures, and (d) the extent to which there is a relationship between performance as measured by the assessment and performance in practice. Over the past 25 years, changes in licensure and certification have been driven by the educational outcomes movement, the need for lifelong learning, and advances in technology and psychometrics. Over the next 25 years, we expect these forces to continue to exert pressure for change which will lead to additional improvement and expansion in examination processes, methods of assessment, and validation research.

Self-assessment of practice performance: development of the ABIM Practice Improvement Module (PIM).

BACKGROUND: Quality measurement and improvement in practice are requirements for Maintenance of Certification by the American Board of Medical Specialties boards and a component of many pay for performance programs. OBJECTIVE: To describe the development of the American Board of Internal Medicine (ABIM) Practice Improvement Module (PIM) and the average performance of ABIM diplomates who have completed the Preventive Cardiology PIM. DESIGN: Observational study of self-administered practice quality improvement. SETTING: Office practices through the United States. PARTICIPANTS: A total of 179 cardiologists and general internists completing requirements for ABIM Maintenance of Certification from 2004 through 2005. MEASUREMENTS: Physicians self-audited at least 25 charts to obtain performance measures, patient demographics, and coronary heart disease risk factors. At least 25 patients completed surveys regarding their experience of care in the physician's practice. Physicians completed a self-assessment survey detailing the presence of various practice systems. RESULTS: The mean rate for systolic blood pressure control was 48%, for diastolic blood pressure 84%, and for low-density lipoprotein (LDL) cholesterol at goal 65%. Of patients 61% rated the quality of care as excellent and 58% rated the practices excellent at encouraging questions and answering them clearly. More than 85% of patients reported "no problem" obtaining a prescription refill, scheduling an appointment, reaching someone in the practice with a question, or obtaining lab results. Targets for improvement were increasing the rates for LDL cholesterol or systolic blood pressure at goal, improving patients' physical activity, patient education, and accuracy of risk assessment. Improvement strategies included implementing chart forms, patient education, or care management processes. LIMITATIONS: Patients and charts were selected by physicians reporting their performance for the purpose of MOC. CONCLUSIONS: The Preventive Cardiology PIM successfully provides a self-assessment of practice performance and provides guidance in helping physicians initiate a cycle of quality improvement in their practices.

The subspecialization rate of third year internal medicine residents from 1992 through 1998.

BACKGROUND: The appropriateness of U.S. physician workforce size and the proportion of generalists versus specialists have long been debated. Difficulty collecting reliable data and varying methodologies complicate clear analysis of workforce questions. PURPOSE: This work examines the rate at which internists subspecialized during the 1990s. It also compares two approaches for estimating subspecialization rates: (a) following resident classes longitudinally ("cohort" approach), and (b) comparing 1st year fellowship (F-1) class size to the previous year's 3rd-year resident (R-3) class size (F-1/R-3). METHODS: Data were collected through the American Board of Internal Medicine's tracking program. Physicians completing their R-3 year in 1992 through 1998 were the participants. The proportion of each R-3 group that eventually entered subspecialty training was examined. Demographic data for those entering subspecialty training and those who did not were compared. Subspecialization rate estimates for the cohort and F-1/R-3 approaches were also compared. RESULTS: The number of internists increased, whereas the number entering subspecialty training declined. Men were more likely to enter a subspecialty than women. International medical school graduates were more likely to enter a subspecialty than U.S. medical school graduates. University-based residency program trainees were more likely to enter a subspecialty than community hospital program trainees. Those entering subspecialty training tended to be younger and score higher on the internal medicine certification examination than those who did not. Almost identical estimates where produced by the cohort and F-1/R-3 approaches. CONCLUSIONS: There was a downward trend in the rate at which internists entered subspecialty training during the 1990s. The two methodologies examined produced similar results.