ABSTRACT
Importance: Despite its importance to medical education and competency assessment for internal medicine trainees, evidence about the relationship between physicians' milestone residency ratings or the American Board of Internal Medicine's initial certification examination and their hospitalized patients' outcomes is sparse. Objective: To examine the association between physicians' milestone ratings and certification examination scores and hospital outcomes for their patients. Design, Setting, and Participants: Retrospective cohort analyses of 6898 hospitalists completing training in 2016 to 2018 and caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019 at US hospitals. Main Outcomes and Measures: Primary outcome measures included 7-day mortality and readmission rates. Thirty-day mortality and readmission rates, length of stay, and subspecialist consultation frequency were also assessed. Analyses accounted for hospital fixed effects and adjusted for patient characteristics, physician years of experience, and year. Exposures: Certification examination score quartile and milestone ratings, including an overall core competency rating measure equaling the mean of the end of residency milestone subcompetency ratings categorized as low, medium, or high, and a knowledge core competency measure categorized similarly. Results: Among 455â¯120 hospitalizations, median patient age was 79 years (IQR, 73-86 years), 56.5% of patients were female, 1.9% were Asian, 9.8% were Black, 4.6% were Hispanic, and 81.9% were White. The 7-day mortality and readmission rates were 3.5% (95% CI, 3.4%-3.6%) and 5.6% (95% CI, 5.5%-5.6%), respectively, and were 8.8% (95% CI, 8.7%-8.9%) and 16.6% (95% CI, 16.5%-16.7%) for mortality and readmission at 30 days. Mean length of stay and number of specialty consultations were 3.6 days (95% CI, 3.6-3.6 days) and 1.01 (95% CI, 1.00-1.03), respectively. A high vs low overall or knowledge milestone core competency rating was associated with none of the outcome measures assessed. For example, a high vs low overall core competency rating was associated with a nonsignificant 2.7% increase in 7-day mortality rates (95% CI, -5.2% to 10.6%; P = .51). In contrast, top vs bottom examination score quartile was associated with a significant 8.0% reduction in 7-day mortality rates (95% CI, -13.0% to -3.1%; P = .002) and a 9.3% reduction in 7-day readmission rates (95% CI, -13.0% to -5.7%; P < .001). For 30-day mortality, this association was -3.5% (95% CI, -6.7% to -0.4%; P = .03). Top vs bottom examination score quartile was associated with 2.4% more consultations (95% CI, 0.8%-3.9%; P < .003) but was not associated with length of stay or 30-day readmission rates. Conclusions and Relevance: Among newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries.
Subject(s)
Certification , Clinical Competence , Hospital Mortality , Internal Medicine , Internship and Residency , Patient Readmission , Humans , Internal Medicine/education , Internal Medicine/standards , Patient Readmission/statistics & numerical data , Retrospective Studies , Internship and Residency/standards , Certification/standards , United States , Female , Male , Length of Stay/statistics & numerical data , Medicare , Aged , Hospitalists/standards , Educational Measurement/standardsABSTRACT
BACKGROUND: The 2014 adoption of the Milestone ratings system may have affected evaluation bias against minoritized groups. OBJECTIVE: To assess bias in internal medicine (IM) residency knowledge ratings against Black or Latino residents-who are underrepresented in medicine (URiM)-and Asian residents before versus after Milestone adoption in 2014. DESIGN: Cross-sectional and interrupted time-series comparisons. SETTING: U.S. IM residencies. PARTICIPANTS: 59 835 IM residents completing residencies during 2008 to 2013 and 2015 to 2020. INTERVENTION: Adoption of the Milestone ratings system. MEASUREMENTS: Pre-Milestone (2008 to 2013) and post-Milestone (2015 to 2020) bias was estimated as differences in standardized knowledge ratings between U.S.-born and non-U.S.-born minoritized groups versus non-Latino U.S.-born White (NLW) residents, with adjustment for performance on the American Board of Internal Medicine IM certification examination and other physician characteristics. Interrupted time-series analysis measured deviations from pre-Milestone linear bias trends. RESULTS: During the pre-Milestone period, ratings biases against minoritized groups were large (-0.40 SDs [95% CI, -0.48 to -0.31 SDs; P < 0.001] for URiM residents, -0.24 SDs [CI, -0.30 to -0.18 SDs; P < 0.001] for U.S.-born Asian residents, and -0.36 SDs [CI, -0.45 to -0.27 SDs; P < 0.001] for non-U.S.-born Asian residents). These estimates decreased to less than -0.15 SDs after adoption of Milestone ratings for all groups except U.S.-born Black residents, among whom substantial (though lower) bias persisted (-0.26 SDs [CI, -0.36 to -0.17 SDs; P < 0.001]). Substantial deviations from pre-Milestone linear bias trends coincident with adoption of Milestone ratings were also observed. LIMITATIONS: Unobserved variables correlated with ratings bias and Milestone ratings adoption, changes in identification of race/ethnicity, and generalizability to Milestones 2.0. CONCLUSION: Knowledge ratings bias against URiM and Asian residents was ameliorated with the adoption of the Milestone ratings system. However, substantial ratings bias against U.S.-born Black residents persisted. PRIMARY FUNDING SOURCE: None.
Subject(s)
Bias , Clinical Competence , Internship and Residency , Humans , Certification , Cross-Sectional Studies , Hispanic or Latino , United States , Black or African American , AsianABSTRACT
PURPOSE: Medical oncologists have a variety of options for demonstrating proficiency in providing high-quality patient care. Perhaps, the best-known opportunity for demonstrating individual expertise and lifelong learning is the American Board of Internal Medicine (ABIM) maintenance of certification (MOC) program. At the practice level, ASCO has offered the Quality Oncology Practice Initiative (QOPI) as a means of optimizing cancer care delivery. In this study, we assess the association between active involvement in MOC on an individual basis and whether that individual's practice is involved with the QOPI program. METHODS: We evaluated 13,600 US medical oncologists initially certified by the ABIM and divided them into those initially certified before 1990 (the year in which ABIM started to require periodic recertification), those from 1990 to 2007, and those from 2008 to 2016. It was then determined which of these had let their certificates expire by 2020. These data were then compared with practices that participated in QOPI from 2017 to 2019, resulting in the matching of 97% of physicians. RESULTS: Of individuals initially certified before 1990 (and technically with lifelong certification), 22% were in QOPI practices. Among those who did not have lifelong certification, there was an association between QOPI participation and maintenance of ABIM certification. For those initially certified between 1990 and 2007, 35% of oncologists with up-to-date ABIM certification were in QOPI practices, whereas only 11% with expired ABIM certification were QOPI participants (P < .0001). For those in the 2008-2016 category, the numbers were 36% v 16%, respectively (P < .0001). CONCLUSION: Our analysis identifies a relationship between participation in these ABIM and ASCO proficiency programs. The reasons for this are likely complex and based on a variety of institutional, professional, monetary, and personal factors.
Subject(s)
Certification , Physicians , Humans , Medical Oncology , Quality of Health Care , United StatesABSTRACT
OBJECTIVE: To determine whether internists' initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF). DESIGN: Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017. SETTING: All hospitals in Pennsylvania. PARTICIPANTS: All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist. PRIMARY OUTCOME MEASURE: In-hospital mortality. RESULTS: Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians' demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; p<0.001). Patients cared for by physicians with initial certification had a 15.87% decrease in mortality compared with those cared for by non-certified physicians (mortality rate difference of 5.09 per 1000 patients; 95% CI 2.12 to 8.05; p<0.001). The adjusted OR for MOC was 0.804 (95% CI 0.697 to 0.926; p=0.003). Patients cared for by physicians who completed MOC had an 18.91% decrease in mortality compared with those cared for by MOC lapsed physicians (mortality rate difference of 6.22 per 1000 patients; 95% CI 2.0 to 10.4; p=0.004). CONCLUSIONS: Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.
Subject(s)
Heart Failure , Myocardial Infarction , Physicians , Certification , Heart Failure/therapy , Hospital Mortality , Hospitalization , Humans , Internal Medicine , Myocardial Infarction/therapy , Pennsylvania/epidemiology , Retrospective Studies , United StatesABSTRACT
BACKGROUND: The pass rate on the American Board of Internal Medicine (ABIM) nephrology certifying exam has declined and is among the lowest of all internal medicine (IM) subspecialties. In recent years, there have also been fewer applicants for the nephrology fellowship match. METHODS: This retrospective observational study assessed how changes between 2010 and 2019 in characteristics of 4094 graduates of US ACGME-accredited nephrology fellowship programs taking the ABIM nephrology certifying exam for the first time, and how characteristics of their fellowship programs were associated with exam performance. The primary outcome measure was performance on the nephrology certifying exam. Fellowship program pass rates over the decade were also studied. RESULTS: Lower IM certifying exam score, older age, female sex, international medical graduate (IMG) status, and having trained at a smaller nephrology fellowship program were associated with poorer nephrology certifying exam performance. The mean IM certifying exam percentile score among those who subsequently took the nephrology certifying exam decreased from 56.7 (SD, 27.9) to 46.1 (SD, 28.7) from 2010 to 2019. When examining individuals with comparable IM certifying exam performance, IMGs performed less well than United States medical graduates (USMGs) on the nephrology certifying exam. In 2019, only 57% of nephrology fellowship programs had aggregate 3-year certifying exam pass rates ≥80% among their graduates. CONCLUSIONS: Changes in IM certifying exam performance, certain trainee demographics, and poorer performance among those from smaller fellowship programs explain much of the decline in nephrology certifying exam performance. IM certifying exam performance was the dominant determinant.
Subject(s)
Certification/trends , Educational Measurement/statistics & numerical data , Fellowships and Scholarships/trends , Internal Medicine/education , Nephrology/education , Adult , Age Factors , Certification/statistics & numerical data , Education, Medical, Graduate/statistics & numerical data , Education, Medical, Graduate/trends , Fellowships and Scholarships/statistics & numerical data , Female , Foreign Medical Graduates/statistics & numerical data , Humans , Internal Medicine/statistics & numerical data , Internal Medicine/trends , Male , Nephrology/statistics & numerical data , Nephrology/trends , Osteopathic Physicians/statistics & numerical data , Sex Factors , United StatesABSTRACT
Importance: Opioid musculoskeletal pain overprescribing was widespread in the mid-2000s. The degree to which prescribing changed as awareness of the danger grew among physicians with different levels of clinical knowledge remains unstudied. Objective: To compare the association of clinical knowledge with opioid prescribing from 2009 to 2011 when prescribing peaked nationally with 2015 to 2017 when guidelines shifted away from opioid prescribing. Design, Setting, and Participants: This cross-sectional study included 10 246 midcareer general internal medicine physicians in the United States who saw patients who were Medicare beneficiaries with Part D enrollment from 2009 to 2017. Main Outcomes and Measures: Any opioid prescription and high dosage or long duration (HDLD) (>7 days or >50 daily morphine milligram equivalents) opioid prescriptions filled within 7 days of applicable visits for new low back pain concerns. Associations between opioid prescribing for new low back pain concerns during outpatient visits and clinical knowledge measured by prior year American Board of Internal Medicine (ABIM) Maintenance of Certification examination performance were estimated using serial cross-sectional logit regressions. Regression covariates included yearly examination quartile (ie, knowledge quartile) interacted with 3-year group dummies (ie, early: 2009-2011; middle: 2012-2014; late: 2015-2017), state and year dummies, physician, practice, patient characteristics, and state opioid regulations. Results: Of the 55â¯387 low back pain visits included in this study, 37â¯185 (67.1%) were visits with female patients, 41â¯978 (75.8%) were with White patients, and the mean (SE) age of patients was 76.2 (<0.01) years. The rate of opioid prescribing was 21.6% (11â¯978) for any opioid prescription and 17.6% (9759) for HDLD prescriptions. From 2009 to 2011, visits with physicians in the highest and lowest knowledge quartiles had similar adjusted opioid prescribing rates with a 0.5 (95% CI, -1.9 to 3.0) percentage point difference. By 2015 to 2017, visits with physicians in the highest knowledge quartile prescribed opioids less frequently that physicians in the lowest knowledge quartile (4.6 percentage point difference; 95% CI, -7.5 to -1.8 percentage points). Visits in which HDLD opioids were prescribed showed no difference in the early period but showed a difference in the late period when comparing physicians in the highest and lowest knowledge quartiles (early period: difference -0.1; 95% CI, -2.4 to 2.2 percentage points; late period difference: 4.8; 95% CI, -7.4 to -2.1 percentage points). Conclusions and Relevance: In this cross-sectional study, when the standard of care shifted away from routine opioid prescribing, physicians who performed well on an ABIM examination were less likely to prescribe opioids for back pain than physicians who performed less well on the examination.
Subject(s)
Analgesics, Opioid/administration & dosage , Back Pain/drug therapy , Clinical Competence/standards , Practice Patterns, Physicians'/standards , Adult , Clinical Competence/statistics & numerical data , Cross-Sectional Studies , Female , Humans , Male , Middle Aged , Practice Patterns, Physicians'/statistics & numerical dataABSTRACT
OBJECTIVE: Diagnostic error is a key healthcare concern and can result in substantial morbidity and mortality. Yet no study has investigated the relationship between adverse outcomes resulting from diagnostic errors and one potentially large contributor to these errors: deficiencies in diagnostic knowledge. Our objective was to measure that associations between diagnostic knowledge and adverse outcomes after visits to primary care physicians that were at risk for diagnostic errors. SETTING/PARTICIPANTS: 1410 US general internists who recently took their American Board of Internal Medicine Maintenance of Certification (ABIM-IM-MOC) exam treating 42 407 Medicare beneficiaries who experienced 48 632 'index' outpatient visits for new problems at risk for diagnostic error because the presenting problem (eg, dizziness) was related to prespecified diagnostic error sensitive conditions (eg, stroke). OUTCOME MEASURES: 90-day risk of all-cause death, and, for outcome conditions related to the index visits diagnosis, emergency department (ED) visits and hospitalisations. DESIGN: Using retrospective cohort study design, we related physician performance on ABIM-IM-MOC diagnostic exam questions to patient outcomes during the 90-day period following an index visit at risk for diagnostic error after controlling for practice characteristics, patient sociodemographic and baseline clinical characteristics. RESULTS: Rates of 90-day adverse outcomes per 1000 index visits were 7 for death, 11 for hospitalisations and 14 for ED visits. Being seen by a physician in the top versus bottom third of diagnostic knowledge during an index visit for a new problem at risk for diagnostic error was associated with 2.9 fewer all-cause deaths (95% CI -5.0 to -0.7, p=0.008), 4.1 fewer hospitalisations (95% CI -6.9 to -1.2, p=0.006) and 4.9 fewer ED visits (95% CI -8.1% to -1.6%, p=0.003) per 1000 visits. CONCLUSION: Higher diagnostic knowledge was associated with lower risk of adverse outcomes after visits for problems at heightened risk for diagnostic error.
Subject(s)
Physicians, Primary Care , Diagnostic Errors , Emergency Service, Hospital , Hospitalization , Humans , Medicare , Outpatients , Retrospective Studies , United StatesABSTRACT
PURPOSE: To examine whether there are group differences in milestone ratings submitted by program directors working with clinical competency committees (CCCs) based on gender for internal medicine (IM) residents and whether women and men rated similarly on milestones perform comparably on subsequent in-training and certification examinations. METHOD: This national retrospective study examined end-of-year medical knowledge (MK) and patient care (PC) milestone ratings and IM In-Training Examination (IM-ITE) and IM Certification Examination (IM-CE) scores for 2 cohorts (2014-2017, 2015-2018) of U.S. IM residents at ACGME-accredited programs. It included 20,098/21,440 (94%) residents, with 9,424 women (47%) and 10,674 men (53%). Descriptive statistics and differential prediction techniques using hierarchical linear models were performed. RESULTS: For MK milestone ratings in PGY-1, men and women showed no statistical difference at a significance level of .01 (P = .02). In PGY-2 and PGY-3, men received statistically higher average MK ratings than women (P = .002 and P < .001, respectively). In contrast, men and women received equivalent average PC ratings in each PGY (P = .47, P = .72, and P = .80, for PGY-1, PGY-2, and PGY-3, respectively). Men slightly outperformed women with similar MK or PC ratings in PGY-1 and PGY-2 on the IM-ITE by about 1.7 and 1.5 percentage points, respectively, after adjusting for covariates. For PGY-3 ratings, women and men with similar milestone ratings performed equivalently on the IM-CE. CONCLUSIONS: Milestone ratings were largely similar for women and men. Generally, women and men with similar MK or PC milestone ratings performed similarly on future examinations. Although there were small differences favoring men on earlier examinations, these differences disappeared by the final training year. It is questionable whether these small differences are educationally or clinically meaningful. The findings suggest fair, unbiased milestone ratings generated by program directors and CCCs assessing residents.
Subject(s)
Clinical Competence , Educational Measurement , Internal Medicine/education , Sexism , Adult , Certification , Female , Humans , Internship and Residency , Male , Retrospective Studies , Sex Factors , United StatesABSTRACT
OBJECTIVES: Eleven months into the coronavirus disease 2019 pandemic, the country faces accelerating rates of infections, hospitalizations, and deaths. Little is known about the experiences of critical care physicians caring for the sickest coronavirus disease 2019 patients. Our goal is to understand how high stress levels and shortages faced by these physicians during Spring 2020 have evolved. DESIGN: We surveyed (October 23, 2020 to November 16, 2020) U.S. critical care physicians treating coronavirus disease 2019 patients who participated in a National survey earlier in the pandemic (April 23, 2020 to May 3, 2020) regarding their stress and shortages they faced. SETTING: ICU. PATIENTS: Coronavirus disease 2019 patients. INTERVENTION: Irrelevant. MEASUREMENT: Physician emotional distress/physical exhaustion: low (not at all/not much), moderate, or high (a lot/extreme). Shortage indicators: insufficient ICU-trained staff and shortages in medication, equipment, or personal protective equipment requiring protocol changes. MAIN RESULTS: Of 2,375 U.S. critical care attending physicians who responded to the initial survey, we received responses from 1,356 (57.1% response rate), 97% of whom (1,278) recently treated coronavirus disease 2019 patients. Two thirds of physicians (67.6% [864]) reported moderate or high levels of emotional distress in the Spring versus 50.7% (763) in the Fall. Reports of staffing shortages persisted with 46.5% of Fall respondents (594) reporting a staff shortage versus 48.3% (617) in the Spring. Meaningful shortages of medication and equipment reported in the Spring were largely alleviated. Although personal protective equipment shortages declined by half, they remained substantial. CONCLUSIONS: Stress, staffing, and, to a lesser degree, personal protective equipment shortages faced by U.S. critical care physicians remain high. Stress levels were higher among women. Considering the persistence of these findings, rising levels of infection nationally raise concerns about the capacity of the U.S. critical care system to meet ongoing and future demands.
Subject(s)
COVID-19/psychology , Critical Care/psychology , Occupational Stress , Physicians/psychology , Psychological Distress , Adult , Disease Hotspot , Equipment and Supplies, Hospital/supply & distribution , Female , Humans , Male , Middle Aged , Personal Protective Equipment/supply & distribution , SARS-CoV-2 , Surveys and Questionnaires , United States/epidemiology , Workforce , WorkplaceABSTRACT
Importance: Use of health care services and physician practice patterns have been shown to vary widely across the United States. Although practice patterns-in particular, physicians' ability to provide high-quality, high-value care-develop during training, the association of a physician's regional practice environment with that ability is less well understood. Objective: To examine the association between health care intensity in the region where physicians practice and their ability to practice high-value care, specifically for physicians whose practice environment changed due to relocation after residency. Design, Setting, and Participants: This cohort study included a national sample of 3896 internal medicine physicians who took the 2002 American Board of Internal Medicine initial certification examination followed approximately 1 decade (April 21, 2011, to May 7, 2015) later by the Maintenance of Certification (MOC) examination. At the time of the MOC examination, 2714 of these internists were practicing in a new region. Data were analyzed from March 6, 2016, to May 21, 2018. Exposures: Intensity of care in the Dartmouth Atlas hospital referral region (HRR), measured by per-enrollee end-of-life physician visits (primary) and current practice type (secondary). Main Outcomes and Measures: The outcome, a physician's ability to practice high-value care, was assessed using the Appropriately Conservative Management (ACM) score on the MOC examination, measuring performance across all questions for which the correct answer was the most conservative option. The exposure, regional health care intensity, was measured as per-enrollee end-of-life physician visits in the Dartmouth Atlas HRR of the physician's practice. Results: Among the 3860 participating internists included in the analysis (2030 men [52.6%]; mean [SD] age, 45.6 [4.5] years), those who moved to regions in the quintile of highest health care intensity had an ACM score 0.22 SD lower (95% CI, -0.32 to -0.12) than internists who moved to regions in the quintile of lowest intensity, controlling for postresidency ACM scores. This difference reflected scoring in the 44th compared with the 53rd percentile of all examinees. This association was mildly attenuated (0.18 SD less; 95% CI, -0.28 to -0.09) after adjustment for physician and practice characteristics. Conclusions and Relevance: This study found that practice patterns of internists who relocate after residency training appear to migrate toward norms of the new region. The demands of practicing in high-intensity regions may erode the ability to practice high-value conservative care.
Subject(s)
Internal Medicine , Physicians , Adult , Clinical Competence , Female , Humans , Internal Medicine/organization & administration , Internal Medicine/standards , Internal Medicine/statistics & numerical data , Internship and Residency , Male , Middle Aged , Physicians/organization & administration , Physicians/standards , Retrospective Studies , WorkplaceABSTRACT
PURPOSE: Critics argue that the American Board of Internal Medicine's medical oncology Maintenance of Certification examination requires medical oncologists with a narrow scope of practice to spend time studying material that is no longer relevant to their practice. However, no data are available describing the scope of practice for medical oncologists. METHODS: Using Medicare claims, we examined the scope of practice for 9,985 medical oncologists who saw 8.6 million oncology conditions in 2016, each of which was assigned to 1 of 23 different condition groups. Scope of practice was then measured as the percentage of oncology conditions within each of the 23 groups. We grouped physicians with similar scopes of practice by applying K-means clustering to the percentage of conditions seen. The scope of practice for each physician cluster was determined from the cancers that encompassed the majority of average oncology conditions seen among physicians composing the cluster. RESULTS: We found 20 distinct scope-of-practice clusters. The largest (n = 6,479 [65.5%]) had a general oncology scope of practice. The remaining physicians focused on a narrow scope of cancers, including 22.6% focused on ≥ 1 solid tumors and 11.9% focused on hematologic malignancies. The largest focused cluster accounted for 7.7% of physicians focused on breast cancer. CONCLUSION: A single American Board of Internal Medicine Maintenance of Certification assessment in medical oncology is most appropriate for approximately 65% of certified medical oncologists' practices. However, the addition of assessments focused on breast cancer and hematologic malignancies could increase this figure to upwards of 85% of certified medical oncologists.
Subject(s)
Oncologists , Scope of Practice , Aged , Certification , Humans , Medical Oncology , Medicare , United StatesABSTRACT
BACKGROUND: In certain medical specialties, board certification is associated with a lower risk of state medical board disciplinary actions. OBJECTIVE: The association between maintaining American Board of Emergency Medicine (ABEM) certification and state medical disciplinary actions had not been studied. This study was undertaken to determine if maintaining ABEM certification was associated with a lower risk of disciplinary action. METHODS: This investigation was a historical cohort study using Cox regression. Physicians who did not have a lapse in ABEM certification were compared with physicians who had a lapse to determine the risk of disciplinary action. Lapsing was determined at the expiration of the initial certificate. This study included all physicians who obtained initial ABEM certification from 1980-2005. Additional covariates of interest included the number of attempts on the ABEM Qualifying Examination (1 vs. >1), the geographic region of the physician's residence, and the country of medical school. RESULTS: There were 23,002 physicians in the study cohort. Of these, 3370 (14.7%) let their certification lapse after initial certification. There were 701 (3.0%) physicians with disciplinary events. Lapsed physicians had higher rates of disciplinary actions than physicians who did not lapse (6.4% vs. 2.5%). ABEM-certified physicians who did not lapse were significantly less likely to be disciplined as physicians who let their certificate lapse (hazard ratio 0.50 [95% confidence interval 0.42-0.59]). CONCLUSIONS: The absolute incidence of physicians with a disciplinary action in this study cohort was low (3.0%). Maintaining ABEM certification was associated with a lower risk of state medical board disciplinary actions.
Subject(s)
Certification/statistics & numerical data , Employee Discipline/statistics & numerical data , State Government , Certification/standards , Clinical Competence/standards , Clinical Competence/statistics & numerical data , Emergency Medicine/methods , Emergency Medicine/standards , Emergency Medicine/statistics & numerical data , Humans , Proportional Hazards Models , United StatesABSTRACT
PURPOSE: As part of the American Board of Internal Medicine's (ABIM's) continuing effort to update its Maintenance of Certification (MOC) program, a content validity tool was used to conduct structured reviews of MOC exam blueprints (i.e., test specification tables) by the physician community. Results from the Cardiovascular Disease MOC blueprint review are presented to illustrate the process ABIM conducted for several internal medicine disciplines. METHOD: Ratings of topic frequency and importance were collected from cardiologists in 2016 using a three-point scale (low, medium, high). The web-based survey instrument presented 188 blueprint topic descriptions, each combined with five patient-related tasks (e.g., diagnosis, treatment). Descriptive statistics and chi-square analysis were employed. RESULTS: Responses from 441 review participants were analyzed. Frequency and importance ratings were aggregated as a composite statistic representing clinical relevance, and exam assembly criteria were modified to select questions, or items, addressing clinically relevant content only. Specifically, ≥ 88% of exam items now address high-importance topics, including ≤ 15% on topics that are also low frequency; and ≤ 12% of exam items now address medium-importance topics, including ≤ 3% on topics that are also low frequency. The updated blueprint has been published for test takers and provides enhanced information on content that would and would not be tested in subsequent examinations. It is linked to more detailed feedback that examinees receive on items answered incorrectly. CONCLUSIONS: The blueprint review garnered valuable feedback from the physician community and provided new evidence for the content validity of the Cardiovascular Disease MOC exam.
Subject(s)
Attitude of Health Personnel , Cardiologists/psychology , Certification/standards , Clinical Competence/standards , Educational Measurement/standards , Internal Medicine/education , Internal Medicine/standards , Adult , Female , Humans , Male , Middle Aged , Reproducibility of Results , United StatesABSTRACT
PURPOSE: To evaluate validity evidence for internal medicine milestone ratings across programs for three resident cohorts by quantifying "not assessable" ratings; reporting mean longitudinal milestone ratings for individual residents; and correlating medical knowledge ratings across training years with certification examination scores to determine predictive validity of milestone ratings for certification outcomes. METHOD: This retrospective study examined milestone ratings for postgraduate year (PGY) 1-3 residents in U.S. internal medicine residency programs. Data sources included milestone ratings, program characteristics, and certification examination scores. RESULTS: Among 35,217 participants, there was a decreased percentage with "not assessable" ratings across years: 1,566 (22.5%) PGY1s in 2013-2014 versus 1,219 (16.6%) in 2015-2016 (P = .01), and 342 (5.1%) PGY3s in 2013-2014 versus 177 (2.6%) in 2015-2016 (P = .04). For individual residents with three years of ratings, mean milestone ratings increased from around 3 (behaviors of an early learner or advancing resident) in PGY1 (ranging from a mean of 2.73 to 3.19 across subcompetencies) to around 4 (ready for unsupervised practice) in PGY3 (mean of 4.00 to 4.22 across subcompetencies, P < .001 for all subcompetencies). For each increase of 0.5 units in two medical knowledge (MK1, MK2) subcompetency ratings, the difference in examination scores for PGY3s was 19.5 points for MK1 (P < .001) and 19.0 for MK2 (P < .001). CONCLUSIONS: These findings provide evidence of validity of the milestones by showing how training programs have applied them over time and how milestones predict other training outcomes.
Subject(s)
Educational Measurement/standards , Educational Status , Internal Medicine/education , Cohort Studies , Education, Medical, Graduate/methods , Education, Medical, Graduate/statistics & numerical data , Educational Measurement/methods , Educational Measurement/statistics & numerical data , Follow-Up Studies , Humans , Internal Medicine/standards , Internal Medicine/statistics & numerical data , Longitudinal Studies , Retrospective StudiesABSTRACT
BACKGROUND AND OBJECTIVES: Medical specialty and subspecialty fellowship programs administer subject-specific in-training examinations to provide feedback about level of medical knowledge to fellows preparing for subsequent board certification. This study evaluated the association between the American Society of Nephrology In-Training Examination and the American Board of Internal Medicine Nephrology Certification Examination in terms of scores and passing status. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: The study included 1684 nephrology fellows who completed the American Society of Nephrology In-Training Examination in their second year of fellowship training between 2009 and 2014. Regression analysis examined the association between In-Training Examination and first-time Nephrology Certification Examination scores as well as passing status relative to other standardized assessments. RESULTS: This cohort included primarily men (62%) and international medical school graduates (62%), and fellows had an average age of 32 years old at the time of first completing the Nephrology Certification Examination. An overwhelming majority (89%) passed the Nephrology Certification on their first attempt. In-Training Examination scores showed the strongest association with first-time Nephrology Certification Examination scores, accounting for approximately 50% of the total explained variance in the model. Each SD increase in In-Training Examination scores was associated with a difference of 30 U (95% confidence interval, 27 to 33) in certification performance. In-Training Examination scores also were significantly associated with passing status on the Nephrology Certification Examination on the first attempt (odds ratio, 3.46 per SD difference in the In-Training Examination; 95% confidence interval, 2.68 to 4.54). An In-Training Examination threshold of 375, approximately 1 SD below the mean, yielded a positive predictive value of 0.92 and a negative predictive value of 0.50. CONCLUSIONS: American Society of Nephrology In-Training Examination performance is significantly associated with American Board of Internal Medicine Nephrology Certification Examination score and passing status.
Subject(s)
Certification , Educational Measurement , Nephrology/education , Adult , Female , Humans , Internal Medicine , MaleABSTRACT
BACKGROUND: Some have questioned whether successful performance in the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program is meaningful. The association of the ABIM Internal Medicine (IM) MOC examination with state medical board disciplinary actions is unknown. OBJECTIVE: To assess risk of disciplinary actions among general internists who did and did not pass the MOC examination within 10 years of initial certification. DESIGN: Historical population cohort study. PARTICIPANTS: The population of internists certified in internal medicine, but not a subspecialty, from 1990 through 2003 (n = 47,971). INTERVENTION: ABIM IM MOC examination. SETTING: General internal medicine in the USA. MAIN MEASURES: The primary outcome measure was time to disciplinary action assessed in association with whether the physician passed the ABIM IM MOC examination within 10 years of initial certification, adjusted for training, certification, demographic, and regulatory variables including state medical board Continuing Medical Education (CME) requirements. KEY RESULTS: The risk for discipline among physicians who did not pass the IM MOC examination within the 10 year requirement window was more than double than that of those who did pass the examination (adjusted HR 2.09; 95% CI, 1.83 to 2.39). Disciplinary actions did not vary by state CME requirements (adjusted HR 1.02; 95% CI, 0.94 to 1.16), but declined with increasing MOC examination scores (Kendall's tau-b coefficient = - 0.98 for trend, p < 0.001). Among disciplined physicians, actions were less severe among those passing the IM MOC examination within the 10-year requirement window than among those who did not pass the examination. CONCLUSIONS: Passing a periodic assessment of medical knowledge is associated with decreased state medical board disciplinary actions, an important quality outcome of relevance to patients and the profession.
Subject(s)
Certification/standards , Employee Discipline/statistics & numerical data , Internal Medicine/education , Adult , Cohort Studies , Female , Humans , Male , Professional Competence , Time Factors , United StatesABSTRACT
BACKGROUND: Breast cancer is a leading cause of death in the United States. Continuing medical education programs such as the American Board of Internal Medicine's Maintenance of Certification (MOC) program can increase early detection of cancers by educating physicians about the benefits of screening. Did the imposition of American Board of Internal Medicine's MOC requirement affect guideline-compliant mammography screening? METHOD: To address this question, we took advantage of a natural experiment that occurred when one group of general internists was required to complete MOC by 2001 because they initially certified in 1991 (MOC required) and another group was grandfathered out of this requirement because they initially certified in 1989 (MOC grandfathers). To measure associations with the MOC requirement, we compared mammography screening in the 2 years before and the 3 years after the 2001 MOC requirement among beneficiaries treated by the MOC-required physicians and compared this difference with the same difference in mammography screening among a control group of beneficiaries treated by the MOC-grandfathered physicians. RESULTS: We found that the MOC requirement was associated with a regression adjusted 2.8% increase (p < .001) in annual screening and 1.7% increase (p < .001) in biennial screening. When we limited the sample to beneficiaries with no screening at baseline (1999 and 2000), these figures increased to 8.5% (p = .02) and 6.4% (p = .01), respectively. CONCLUSIONS: The MOC requirement was associated with an improvement in guideline-compliant mammography screening with the most pronounced improvements among women who were the least adherent at baseline and therefore might have benefited the most from screening.
Subject(s)
Breast Neoplasms/diagnosis , Certification , Education, Medical, Continuing , Internal Medicine/standards , Mammography , Physicians/standards , Age Factors , Clinical Competence , Early Detection of Cancer , Female , Guideline Adherence , Humans , Mass Screening , Medicare , United StatesABSTRACT
PURPOSE: To investigate how changing to or from solo practice settings relates to maintenance of certification (MOC) exam performance. METHOD: The authors conducted a retrospective analysis of exam pass/fail outcomes for 7,112 physicians who successfully completed their initial MOC cycle from 2000 to 2004. Initial physician MOC practice characteristics records, demographic information, and exam performance were linked with exam pass/fail outcomes from their second MOC cycle from 2006 to 2014 (5,215 physicians after attrition). Exam pass/fail outcomes for physicians' second MOC cycle were compared among four groups: those who remained in group practice across both MOC cycles, those who changed from group to solo practice, those who changed from solo to group practice, and those who remained in solo practice across both MOC cycles. RESULTS: Physicians who changed from solo to group practice performed significantly better than those who remained in solo practice (odds ratio [OR] = 1.67; 95% confidence interval [CI] = 1.11, 2.51; P = .027). Conversely, physicians changing from group to solo practice performed significantly worse than physicians staying in group practice (OR = 0.60; 95% CI = 0.45, 0.80; P = .002). Meanwhile, physicians who changed from solo to group practice performed similarly to physicians remaining in group practice (OR = 0.95; 95% CI = 0.67, 1.35; P = 0.76). CONCLUSIONS: Changes in solo/group practice status were associated with second-cycle MOC exam performance. This study provides evidence that the context in which a physician practices may have an impact on their MOC exam performance.
Subject(s)
Certification/statistics & numerical data , Education, Medical, Continuing/statistics & numerical data , Group Practice/statistics & numerical data , Internal Medicine/statistics & numerical data , Private Practice/statistics & numerical data , Adult , Clinical Competence , Female , Humans , Internal Medicine/methods , Male , Middle Aged , Odds Ratio , Retrospective StudiesABSTRACT
BACKGROUND: Electronic resources are increasingly used in medical practice. Their use during high-stakes certification examinations has been advocated by many experts, but whether doing so would affect the capacity to differentiate between high and low abilities is unknown. OBJECTIVE: To determine the effect of electronic resources on examination performance characteristics. DESIGN: Randomized controlled trial. SETTING: Medical certification program. PARTICIPANTS: 825 physicians initially certified by the American Board of Internal Medicine (ABIM) who passed the Internal Medicine Certification examination or sat for the Internal Medicine Maintenance of Certification (IM-MOC) examination in 2012 to 2015. INTERVENTION: Participants were randomly assigned to 1 of 4 conditions: closed book using typical or additional time, or open book (that is, UpToDate [Wolters Kluwer]) using typical or additional time. All participants took the same modified version of the IM-MOC examination. MEASUREMENTS: Primary outcomes included item difficulty (how easy or difficult the question was), item discrimination (how well the question differentiated between high and low abilities), and average question response time. Secondary outcomes included examination dimensionality (that is, the number of factors measured) and test-taking strategy. Item response theory was used to calculate question characteristics. Analysis of variance compared differences among conditions. RESULTS: Closed-book conditions took significantly less time than open-book conditions (mean, 79.2 seconds [95% CI, 78.5 to 79.9 seconds] vs. 110.3 seconds [CI, 109.2 to 111.4 seconds] per question). Mean discrimination was statistically significantly higher for open-book conditions (0.34 [CI, 0.32 to 0.35] vs. 0.39 [CI, 0.37 to 0.41] per question). A strong single dimension showed that the examination measured the same factor with or without the resource. LIMITATION: Only 1 electronic resource was evaluated. CONCLUSION: Inclusion of an electronic resource with time constraints did not adversely affect test performance and did not change the specific skill or factor targeted by the examination. Further study on the effect of resource inclusion on other examinations is warranted. PRIMARY FUNDING SOURCE: ABIM Foundation.
Subject(s)
Certification/methods , Decision Support Systems, Clinical , Educational Measurement/methods , Internal Medicine/education , Adult , Clinical Competence , Female , Formative Feedback , Humans , Male , Middle Aged , Time Factors , United StatesABSTRACT
Importance: Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine's (ABIM's) Maintenance of Certification program (MOC). Therefore, it is important to understand the degree to which this examination reflects conditions seen in practice, one dimension of content validity, which focuses on the match between content in the discipline and the topics on the examination questions. Objective: To assess whether the frequency of questions on IM-MOC examinations were concordant with the frequency of conditions seen in practice. Design, Setting, and Participants: The 2010-2013 IM-MOC examinations were used to calculate the percentage of questions for 186 medical condition categories from the examination blueprint, which balances examination content by considering importance and frequency of conditions seen in practice. Nationally representative estimates of conditions seen in practice by general internists were estimated from the primary diagnosis for 13â¯832 office visits (2010-2013 National Ambulatory Medical Care Surveys) and 108â¯472 hospital stays (2010 National Hospital Discharge Survey). Exposures: Prevalence of conditions included on the IM-MOC examination questions. Main Outcomes and Measures: The outcome measure was the concordance between the percentages of IM-MOC examination questions and the percentages of conditions seen in practice during either office visits or hospital stays for each of 186 condition categories (eg, diabetes mellitus, ischemic heart disease, liver disease). The concordance thresholds were 0.5 SD of the weighted mean percentages of the applicable 186 conditions seen in practice (0.74% for office visits; 0.51% for hospital stays). If the absolute differences between the percentages of examination questions and the percentages of office visit conditions or hospital stay conditions seen were less than the applicable concordance threshold, then the condition category was judged to be concordant. Results: During the 2010-2013 IM-MOC examination periods, 3600 questions (180 questions per examination form) were administered and 3461 questions (96.1%) were mapped into the 186 study conditions (mean, 18.6 questions per condition). Comparison of the percentages of 186 categories of medical conditions seen in 13â¯832 office visits and 108â¯472 hospital stays with the percentages of 3461 questions on IM-MOC examinations revealed that 2389 examination questions (69.0%; 95% CI, 67.5%-70.6% involving 158 conditions) were categorized as concordant. For concordance between questions and office visits only, 2010 questions (58.08%; 95% CI, 56.43%-59.72% of all examination questions) involving 145 conditions were categorized as concordant. For concordance between questions and hospital stays only, 1456 questions (42.07%; 95% CI, 40.42%-43.71% of all examination questions) involving 122 conditions were categorized as concordant. Conclusions and Relevance: Among questions on IM-MOC examinations from 2010-2013, 69% were concordant with conditions seen in general internal medicine practices, although some areas of discordance were identified.