Suicidal ideation, perception of personal safety, and career regret among emergency medicine residents during the COVID-19 pandemic.

Objectives: The COVID-19 pandemic was disruptive for trainees and may have affected career decisions for some learners. This study examined the impact of the pandemic on emergency medicine (EM) resident perceptions of their mental health, perceptions of personal safety, and career choice regret. Methods: This was a cross-sectional survey study administered following the 2021 American Board of Emergency Medicine In-Training Examination (ITE). Survey measures included suicidal ideation (SI), COVID concerns in terms of infection prevention and control (IPC) training, COVID risk to self and/or COVID risk to family, and COVID-related career regret. COVID concerns were compared by gender and race/ethnicity using Pearson's chi-square tests. Multivariable logistic regression models were used to test the association between SI and COVID concerns, resident characteristics, and program characteristics. Results: A total of 6980 out of 8491 EM residents (82.2%) from 244 programs completed the survey. Only 1.1% of participants reported insufficient training in COVID IPC practices. Participants were concerned about COVID risk to themselves (40.3%) and to their families (63.3%) due to their job roles. These concerns were more common among women or nonbinary (vs. men); all other races/ethnicities (vs. non-Hispanic Whites); senior residents (vs. PGY-1, PGY-2 residents); and residents who were married or in relationships (vs. single or divorced). A total of 6.1% of participants reported that COVID made them reconsider choosing EM as their career. Career regret in this cohort was higher than that in the proportion (3.2%) expressing career regret in the 2020 ITE (p < 0.001). Career regret was more common among women or nonbinary (vs. men); all other races/ethnicities (vs. non-Hispanic Whites); and senior residents (vs. PGY-1, PGY-2 residents). The overall SI rate was 2.6%, which did not differ from that of the 2020 sample of EM residents (2.5%, p = 0.88). Conclusions: Many EM residents reported concerns about COVID risks to themselves and their families. Although the rate of SI remained unchanged, more EM residents reported career regret during the COVID pandemic.

Prospective validity evidence for the abbreviated emergency medicine Copenhagen Burnout Inventory.

BACKGROUND: Prior research has provided retrospective validity evidence for an abbreviated Copenhagen Burnout Inventory (CBI) to measure burnout among emergency medicine (EM) residents. We sought to provide additional validity and reliability evidence for the two-factor, six-item abbreviated CBI. METHODS: This cross-sectional study used data from the abbreviated CBI that was administered following the 2022 American Board of Emergency Medicine In-training Examination. Confirmatory factor analysis (CFA) was performed and the prevalence of burnout among EM residents was determined. RESULTS: Of the 8918 eligible residents, 7465 (83.7%) completed the abbreviated CBI. CFA confirmed the previously developed model of two factors using six items answered with a 1- to 5-point Likert scale. The internal factor was derived from personal and work-related burnout and the external factor was related to caring for patients. The reliability was determined using Cronbach's alpha (0.87). The overall prevalence of burnout was 49.4%; the lowest prevalence was at the EM1 level (43.1%) and the highest was at the EM2 level (53.8%). CONCLUSIONS: CFA of the abbreviated CBI demonstrated good reliability and model fit. The two-factor, six-item survey instrument identified an increase in the prevalence of burnout among EM residents that coincided with working in the COVID-19 environment. The abbreviated CBI has sufficient reliability and validity evidence to encourage its broader use.

Association between the American Board of Emergency Medicine Oral Certifying Examination and Future State Medical Board Disciplinary Actions.

Objectives: The American Board of Emergency Medicine (ABEM) requires a written examination (the Qualifying Examination) followed by the Oral Certifying Examination (OCE) to obtain ABEM certification. Maintaining ABEM certification is associated with fewer state medical board (SMB) disciplinary actions. We sought to determine the association between poor initial performance on the OCE and subsequent severe SMB disciplinary action. Methods: We included physicians who completed US categorical emergency medicine residencies in 2016 and earlier. We classified OCE performance as good (passed on first attempt) and poor (never passed or required > 1 attempt to pass). We obtained data on physician SMB disciplinary actions from the National Practitioner Data Bank that were limited to actions that denied licensure or altered the status of a medical license (eg, suspension). We determined the association between poor OCE performance and subsequent severe SMB disciplinary action. Results: Of 34,871, 93.5% passed the OCE on the first attempt, 6.1% required multiple attempts, and 0.3% never passed. Of the physicians (93.5%) with good OCE performance, 1.0% received a severe SMB action. Among physicians with poor OCE performance, 2.3% received a severe action; and of those who never passed, 1.7% received a severe action (Table 1). Poor OCE performance was associated with an increased odds of severe SMB disciplinary action (OR 2.21, 95% CI: 1.57-3.12). Conclusion: Physicians with poor OCE performance exhibited higher odds of experiencing a subsequent severe SMB disciplinary action. The OCE may have utility as a predictor of future professionalism or clinical performance.

Comparison of performance data between emergency medicine 1-3 and 1-4 program formats.

Objective: This study compares performance data from physicians completing 3-year versus 4-year emergency medicine residency training programs. Currently, there are 2 training formats and little is known about objective performance differences. Methods: This was a retrospective cross-sectional analysis of emergency residents and physicians. Multiple analyses were conducted comparing physicians' performances, including Accreditation Council of Graduate Medical Education Milestones and American Board of Emergency Medicine In-training Examination (ITE), Qualifying Examination (QE), Oral Certification Examination (OCE), and program extensions from 3-year and 4-year residency programs. Some confounding variables were not or could not be considered, such as rationale for medical students to choose one format over another, as well as application and final match rates. Results: Milestone scores are higher for emergency medicine 3 residents in 1-3 programs (3.51) versus emergency medicine 3 residents in 1-4 programs (3.07; P < 0.001, d = 1.47) and highest for emergency medicine 4 residents (3.67). There was no significant difference in program extension rates (emergency medicine 1-3, 8.1%; emergency medicine 1-4, 9.6%; P = 0.05, ω = 0.02). ITE scores were higher for emergency medicine 1, 2, and 3 residents from 1-3 programs and emergency medicine 4 residents from 1-4 programs scored highest. Mean QE score was slightly higher for emergency 1-3 physicians (83.55 vs 83.00; P < 0.01, d = 0.10). QE pass rate was higher for emergency 1-3 physicians (93.1% vs 90.8%; P < 0.001, ω = 0.08). Mean OCE score was slightly higher for emergency 1-4 physicians (5.67 vs 5.65; P = 0.03, d = -0.07) but did not reach a priori statistical significance (α < 0.01). OCE pass rate was also slightly higher for emergency 1-4 physicians (96.9% vs 95.5%; P = 0.06, ω = -0.07) but also non-significant. Conclusions: These results suggest that although performance measures demonstrate small differences between physicians from emergency medicine 1-3 and 1-4 programs, these differences are limited in their ability to make causal claims about performance on the basis of program format alone.

Factors associated with emergency physician income.

Objective: Income fairness is important, but there are limited data that describe income equity among emergency physicians. Understanding the magnitude of and factors associated with income differences may be helpful in eliminating disparities. This study analyzed the associations of demographic factors, training, practice setting, and board certification with emergency physician income. Methods: We distributed a survey to professional members of the American College of Emergency Physicians. The survey included questions on annual income, educational background, practice characteristics, gender, age, race, ethnicity, international medical graduate status, type of medical degree (MD vs DO), completion of a subspecialty fellowship, job characteristics, and board certification. Respondents also reported annual income. We used linear regression to determine the respondent characteristics associated with reported annual income. Results: From 45,961 members we received 3407 responses (7.4%); 2350 contained complete data for regression analysis. The mean reported annual income was $315,306 (95% confidence interval [CI], $310,649 to $319,964). The mean age of the respondents was 47.4 years, 37.4% were women, 3.2% were races underrepresented in medicine (Black, American Indian, or Alaskan Native), and 4.8% were Hispanic or Latino. On linear regression, female gender was associated with lower reported annual income; difference -$43,565, 95% CI, -$52,217 to -$34,913. Physician age, degree (MD vs DO), underrepresented racial minority status, and underrepresented ethnic minority status were not associated with annual income. Fellowship training was associated with lower income; Accreditation Council for Graduate Medical Education (ACGME) program difference -$30,048; 95% CI, -$48,183 to -$11,912, non-ACGME-program difference -$27,640, 95% CI, -$40,970 to -$14,257. Working at a for-profit institution was associated with higher income; difference $12,290, 95% CI, $3693 to $20,888. Board certification was associated with higher income; difference, $43,267, 95% CI, $30,767 to $55,767. Conclusions: This study identified income disparities associated with gender, practice setting, fellowship completion, and American Board of Emergency Medicine or American Osteopathic Board of Emergency Medicine certification.

Early validity and reliability evidence for the American Board of Emergency Medicine Virtual Oral Examination.

Background: The American Board of Emergency Medicine (ABEM) in-person Oral Certification Examination (OCE) was halted abruptly in 2020 due to the COVID-19 pandemic. The OCE was reconfigured to be administered in a virtual environment starting in December 2020. Objectives: The purpose of this investigation was to determine whether there was sufficient validity and reliability evidence to support the continued use of the ABEM virtual Oral Examination (VOE) for certification decisions. Methods: This retrospective, descriptive study used multiple data sources to provide validity evidence and reliability data. Validity evidence focused on test content, response processes, internal structure (e.g., internal consistency and item response theory), and the consequences of testing. A multifaceted Rasch reliability coefficient was used to measure reliability. Study data were from two 2019 in-person OCEs and the first four VOE administrations. Results: There were 2279 physicians who took the 2019 in-person OCE examination and 2153 physicians who took the VOE during the study period. Among the OCE group, 92.0% agreed or strongly agreed that the cases on the examination were cases that an emergency physician should be expected to see; 91.1% of the VOE group agreed or strongly agreed. A similar pattern of responses given to a question about whether the cases on the examination were cases that they had seen. Additional evidence of validity was obtained by the use of the EM Model, the process for case development, the use of think-aloud protocols, and similar test performance patterns (e.g., pass rates). For reliability, the Rasch reliability coefficients for the OCE and the VOE during the study period were all >0.90. Conclusions: There was substantial validity evidence and reliability to support ongoing use of the ABEM VOE to make confident and defensible certification decisions.

Workplace Mistreatment, Career Choice Regret, and Burnout in Emergency Medicine Residency Training in the United States.

STUDY OBJECTIVE: The influence of workplace mistreatment on the well-being and career satisfaction of emergency medicine residents is unknown. This study examined the relationships between burnout, career choice regret, and workplace mistreatment in a national sample of emergency medicine residents. METHODS: This was a secondary analysis of a survey study on the prevalence of workplace mistreatment among emergency residents. Residents who reported emotional exhaustion or depersonalization at least once per week were considered to have burnout. Residents who reported dissatisfaction with their decision to become an emergency physician were considered to have career choice regret. Respondents also reported the type (discrimination, abuse, sexual harassment) and frequency of mistreatment over the academic year. Multivariable logistic regression, adjusting for program characteristics, was used to examine resident characteristics associated with burnout and career choice regret, with the frequency of mistreatment as a covariate. RESULTS: Of the 8,162 eligible residents, 7,680 (94.1 %) participated. About a third of respondents reported burnout (2,188 of 6,902, 31.7%), whereas a minority (224 of 6,923, 3.2%) reported career choice regret. Of the 7,087 responses on mistreatment frequency, 2,117 (29.9%) reported "a few times per year," and 1,296 (18.3%) reported "a few times per month or more." Compared with residents who never experienced mistreatment, residents who reported increasing frequencies of mistreatment were associated with having burnout-from mistreatment a few times per year (OR [odds ratio],1.6; 99% CI [confidence interval], 1.3 to 1.9) to a few times per month or more (OR, 3.3; 99% CI, 2.7 to 4.1). Compared with residents without burnout, residents who reported burnout were associated with having career choice regret (OR, 11.3; 99% CI, 7.0 to 18.1). After adjusting for burnout, there were no significant relationships between the frequency of mistreatment and career choice regret. CONCLUSIONS: Workplace mistreatment is associated with burnout, but not career choice regret, among emergency medicine residents. Efforts to address workplace mistreatment may improve emergency medicine residents' professional well-being.

Reliability and validity support for an abbreviated Copenhagen burnout inventory using exploratory and confirmatory factor analysis.

Objective: The Copenhagen Burnout Inventory (CBI) is an open-access, valid, and reliable instrument measuring burnout that includes 19 items distributed across the following 3 domains (factors): personal burnout, work burnout, and patient burnout. The primary objective of this study was to determine the validity and reliability of an abbreviated CBI to assess burnout in emergency medicine residents. Methods: This cross-sectional study used data from the CBI that followed the 2021 American Board of Emergency Medicine In-training Examination. Exploratory factor analysis (EFA) was followed by confirmatory factor analysis (CFA). Results: Of the 8491 eligible residents, 7225 (85.1%) completed the survey; the EFA cohort included 3613 residents and the CFA cohort included 3612 residents. EFA showed 2 eigenvalues ≥1, an internal factor and an external factor. There were 6 CBI items that contributed to the 2 factors. The first factor was related to personal burnout and work-related burnout and the second factor was related to working with patients. There were 4 CBI items that contributed to the internal factor and 2 CBI items that contributed to the external factor. Using the abbreviated CBI, the incidence of a resident having 1 or both types of burnout was 34.1%. Conclusions: This study provides validity evidence and reliability support for the use of a 6-item, 2-factor abbreviated CBI. A shorter, reliable, valid, and publicly accessible burnout inventory provides numerous advantages for burnout research in emergency medicine.

Emergency Medicine Residency Does Not Reduce a Racial Minority-Based Test Performance Gap.

BACKGROUND: Health care inequity is corrected more readily when safe, high-quality care is provided by physicians who reflect the gender, race, and ethnicity of patient communities. It is important to train and evaluate racially diverse physicians involved in residency training. OBJECTIVE: This study sought to determine any test-taking differences for black Emergency Medicine (EM) residents and whether any such differences would narrow as residency progressed. METHODS: This was an observational, cross-sectional study that reviewed performance (scaled scores) on the American Board of Emergency Medicine (ABEM) In-Training Examination (ITE) for 2018, 2019, and 2020. The study included EM residents in 3-year programs who took the ITE. A linear regression model was used for the variables of race, which included black physicians and white physicians (reference group), and level of training (EM resident year 1 [EM1] as the reference group). RESULTS: There were 9591 residents included; 539 were black and 9052 were white. Mean scaled scores were higher as a function of training level. Regression showed a scaled score intercept of 73.51. The ITE score increased for all groups as a function of training level (EM2 ß = +5.45, p < 0.0001; EM3 ß = +8.09, p < 0.0001). The regression coefficient for black residents was -5.87 (p < 0.0001). There was relative improvement by training level compared with improvement in the reference group, but this difference was not materially or statistically significant. CONCLUSION: In this study of the ABEM ITE, a test-taking performance gap identified early in residency for black physicians persisted into late residency.

The Emergency Medicine Physician Workforce: Projections for 2030.

STUDY OBJECTIVE: The goals of this study were to determine the current and projected supply in 2030 of contributors to emergency care, including emergency residency-trained and board-certified physicians, other physicians, nurse practitioners, and physician assistants. In addition, this study was designed to determine the current and projected demand for residency-trained, board-certified emergency physicians. METHODS: To forecast future workforce supply and demand, sources of existing data were used, assumptions based on past and potential future trends were determined, and a sensitivity analysis was conducted to determine how the final forecast would be subject to variance in the baseline inputs and assumptions. Methods included: (1) estimates of the baseline workforce supply of physicians, nurse practitioners, and physician assistants; (2) estimates of future changes in the raw numbers of persons entering and leaving that workforce; (3) estimates of the productivity of the workforce; and (4) estimates of the demand for emergency care services. The methodology assumes supply equals demand in the base year and estimates the change between the base year and 2030; it then compares supply and demand in 2030 under different scenarios. RESULTS: The task force consensus was that the most likely future scenario is described by: 2% annual graduate medical education growth, 3% annual emergency physician attrition, 20% encounters seen by a nurse practitioner or physician assistant, and 11% increase in emergency department visits relative to 2018. This scenario would result in a surplus of 7,845 emergency physicians in 2030. CONCLUSION: The specialty of emergency medicine is facing the likely oversupply of emergency physicians in 2030. The factors leading to this include the increasing supply of and changing demand for emergency physicians. An organized, collective approach to a balanced workforce by the specialty of emergency medicine is imperative.

Emergency Medicine Resident Burnout and Examination Performance.

OBJECTIVES: Burnout afflicts emergency physicians (EPs) to a significant degree. The impact of burnout spans from decreased clinical efficiency to increased medical errors to heightened risk of physician suicide. This large-scale study captures responses from emergency medicine (EM) residents regarding two burnout items and examines the correlation between in-training examination (ITE) scores and burnout risk as well as that between residency year and burnout risk. METHODS: This was a prospective, mixed-methods, cross-sectional cohort study. All residents in U.S. categorical EM residents who took the 2019 ITE were included. At the end of the ITE, residents were invited to complete a voluntary survey that included two items from the Maslach Burnout Inventory (MBI) that have been found to be strongly indicative of burnout: one about self-perception of being burned out and one about feelings of callousness. Responses were on a 7-level Likert scale (1-7), ranging from very low frequency (1) to very high frequency (7). Measurements included the number of residents in each year-level of training (EM1-EM4), the MBI item ratings, and the ABEM ITE score. Performance, as measured by the scaled, equated score, was compared to the MBI item responses. A corrected Spearman's correlation coefficient (ρ) was used to compare continuous data (score) against a discrete ordinal variable (MBI Likert response). RESULTS: There were 2,501 EM1 residents, 2,389 EM2 residents, 2,206 EM3 residents, and 616 EM4 residents in the study group. There were 7,206 (93.4%) physicians who completed the first MBI question about burnout; 7,172 (93%) completed the second MBI question about callousness. There was no statistically significant association between the burnout item response and ITE performance (ρ = -0.03; p = 0.015). There was a positive, statistically significant association between the callousness item response and higher ITE performance (ρ = 0.07; p < 0.001). There was a statistically significant association between the response to the burnout item and training level (ρ = 0.07; p <0.001). There was also a statistically significant association between the response to the callousness item and training level (ρ = 0.15; p < 0.001). The overall prevalence of burnout risk in various training levels were EM1, 28.2%; EM2, 39%; EM3, 41.1%; and EM4, 43.3%. CONCLUSIONS: Our study found no significant correlation between ITE score and burnout risk. There was a weakly positive correlation between ITE scores and callousness. Based on our study results, ITE scores may not be useful in prognosticating burnout risk for EM residents and, interestingly, higher ITE scores correlated to stronger feelings of callousness. Our study indicates that EM residents at higher levels of training reported stronger self-perceptions of burnout and callousness. Further investigation into why residents at higher levels of training may experience greater burnout risk is warranted.

Costs and Benefits of Initial Certification for Emergency Medicine Residency Graduates.

BACKGROUND: Graduates of emergency medicine residency programs can seek certification from the American Board of Emergency Medicine (ABEM), yet the costs and perceived value by residents is not clear. OBJECTIVE: This report sought to better define the value of board certification by asking physicians taking the ABEM Oral Certification Examination (OCE) to describe its costs (eg, time, money) and perceived benefits. METHODS: A descriptive, cross-sectional, voluntary, anonymous survey was administered to physicians taking the 2018 spring and fall ABEM OCEs. Response frequencies were used to report response rates. RESULTS: There were 2016 physicians who participated in the 2018 OCEs, of whom 1565 (78%) completed a survey. With respect to preparation, 38% (599 of 1565 responses) spent more than 30 hours preparing for the examination. Regarding the expense of preparing for the examination, 21% (328) spent nothing, 50% (776) spent less than $1,000, and 2% (38) spent more than $3,000. Most physicians (80%, 1254) reported a learning benefit to preparing for and taking the OCE. There were 49% (765) of respondents who reported that preparing for the examination reinforced their knowledge of emergency medicine; 20% (311) reported no learning benefit. Most physicians (92%, 1442) reported that ABEM certification provided a career benefit, the most common of which was more career opportunities (69%, 1076). CONCLUSIONS: Initial certification requires a considerable investment of time and money. Physicians seeking initial ABEM certification found both learning and professional benefits, with the most frequently reported being reinforcement of medical knowledge and more career opportunities.

The Association Between Maintaining American Board of Emergency Medicine Certification and State Medical Board Disciplinary Actions.

BACKGROUND: In certain medical specialties, board certification is associated with a lower risk of state medical board disciplinary actions. OBJECTIVE: The association between maintaining American Board of Emergency Medicine (ABEM) certification and state medical disciplinary actions had not been studied. This study was undertaken to determine if maintaining ABEM certification was associated with a lower risk of disciplinary action. METHODS: This investigation was a historical cohort study using Cox regression. Physicians who did not have a lapse in ABEM certification were compared with physicians who had a lapse to determine the risk of disciplinary action. Lapsing was determined at the expiration of the initial certificate. This study included all physicians who obtained initial ABEM certification from 1980-2005. Additional covariates of interest included the number of attempts on the ABEM Qualifying Examination (1 vs. >1), the geographic region of the physician's residence, and the country of medical school. RESULTS: There were 23,002 physicians in the study cohort. Of these, 3370 (14.7%) let their certification lapse after initial certification. There were 701 (3.0%) physicians with disciplinary events. Lapsed physicians had higher rates of disciplinary actions than physicians who did not lapse (6.4% vs. 2.5%). ABEM-certified physicians who did not lapse were significantly less likely to be disciplined as physicians who let their certificate lapse (hazard ratio 0.50 [95% confidence interval 0.42-0.59]). CONCLUSIONS: The absolute incidence of physicians with a disciplinary action in this study cohort was low (3.0%). Maintaining ABEM certification was associated with a lower risk of state medical board disciplinary actions.

Emergency medicine practice environment and impact on concert examination performance.

OBJECTIVE: The ABEM ConCert Examination is a summative examination that ABEM-certified physicians are required to pass once in every 10-year cycle to maintain certification. This study was undertaken to identify practice settings of emergency physicians, and to determine if there was a difference in performance on the 2017 ConCert between physicians of differing practice types and settings. METHODS: This was a mixed methods cross sectional-study, using a post-examination survey and test performance data. All physicians taking the 2017 ConCert Examination who completed three survey questions pertaining to practice type, practice locations, and teaching were included. These three questions address different aspects of academia: self-identification, an academic setting, and whether the physician teaches. RESULTS: Among 2796 test administrations of the 2017 ConCert Examination, 2693 (96.3%) completed the three survey questions about practice environment. The majority (N = 2054; 76.3%) self-identified as primarily being a community physician, 528 (19.6%) as academic, and 111 (4.1%) as other. The average ConCert Examination score for community physicians was 83.5 (95% CI, 83.3-83.8); the academic group was 84.8 (95% CI, 84.3-85.3); and the other group was 82.3 (95% CI, 81.1-83.6). After controlling for initial ability as measured by the Qualifying Examination score, there was no significant difference in performance between academic and community physicians (p = .10). CONCLUSIONS: Academic emergency physicians and community emergency physicians scored similarly on the ConCert. Working at a community teaching hospital was associated with higher examination performance. Teaching medical learners, especially non-emergency medicine residents, was also associated with better examination performance.

Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones.

BACKGROUND: In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies. OBJECTIVE: This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies. METHODS: This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables. RESULTS: There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS. CONCLUSIONS: Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.

Initial Validity Analysis of the American Board of Emergency Medicine Enhanced Oral Examination.

OBJECTIVES: The American Board of Emergency Medicine (ABEM) has introduced a new testing format for the oral certification examination (OCE): the enhanced oral or "eOral" format. The purpose of this study was to perform initial validity analyses of the eOral format. The two hypotheses were: 1) the case content in the eOral format was sufficiently similar to clinical practice and 2) the eOral case materials were sufficiently similar to clinical practice. The eOral and traditional formats were compared for these characteristics. METHODS: This was a prospective survey study. The survey was administered as a voluntary postexamination activity at the end of the 2015 spring (April 25-27) and fall (October 10-13) ABEM OCEs. The survey is a routine part of the ABEM oral examination experience. For 2015, two additional questions were added to gauge the similarity of the eOral format to clinical practice. Validity was defined by content and substantive elements within Messick's model of construct validity as well as portions of Kane's validity model. RESULTS: Of the 1,746 physicians who took the oral examination, 1,380 physicians (79.0%) completed all or part of the study survey questions. The majority of respondents agreed the patient presentations in the cases were similar (strongly agreed or agreed) to cases seen in clinical practice, in both the traditional cases (95.1%) and the eOral cases (90.1%). Likewise, the majority of respondents answered that the case materials (e.g., laboratory, radiographs) were similar (strongly agreed or agreed) to what they encounter in clinical practice, both in the traditional format (85.8%) and in the eOral cases (93.7%). CONCLUSIONS: Most emergency physicians reported that the types of cases tested in the traditional and eOral formats were similar to cases encountered in clinical practice. In addition, most physicians found the case materials to be similar to what is seen in clinical practice. This study provides early validity evidence for the eOral format.

The Relationship Between ACGME Duty Hour Requirements and Performance on the American Board of Emergency Medicine Qualifying Examination.

BACKGROUND: In 2003, the Accreditation Council for Graduate Medical Education (ACGME) instituted requirements that limited the number of hours residents could spend on duty, and in 2011, it revised these requirements. OBJECTIVE: This study explored whether the implementation of the 2003 and 2011 duty hour limits was associated with a change in emergency medicine residents' performance on the American Board of Emergency Medicine (ABEM) Qualifying Examination (QE). METHODS: Beginning with the 1999 QE and ending with the 2014 QE, candidates for whom all training occurred without duty hour requirements (Group A), candidates under the first set of duty hour requirements (Group C), and candidates under the second set of duty hour requirements (Group E) were compared. Comparisons included mean scores and pass rates. RESULTS: In Group A, 5690 candidates completed the examination, with a mean score of 82.8 and a 90.2% pass rate. In Group C, 8333 candidates had a mean score of 82.4 and a 90.5% pass rate. In Group E, there were 1269 candidates, with a mean score of 82.5 and an 89.4% pass rate. There was a small but statistically significant decrease in the mean scores (0.04, P < .001) after implementation of the first duty hour requirements, but this difference did not occur after implementation of the 2011 standards. There was no difference among pass rates for any of the study groups (χ2 = 1.68, P = .43). CONCLUSIONS: We did not identify an association between the 2003 and 2011 ACGME duty hour requirements and performance of test takers on the ABEM QE.

The ABMS MOC Part III Examination: Value, Concerns, and Alternative Formats.

This article describes the presentations and discussions at a conference co-convened by the Council on Medical Education of the American Medical Association (AMA) and by the American Board of Medical Specialties (ABMS). The conference focused on the ABMS Maintenance of Certification (MOC) Part III Examination. This article, reflecting the conference agenda, covers the value of and evidence supporting the examination, as well as concerns about the cost of the examination, and-given the current format-its relevance. In addition, the article outlines alternative formats for the examination that four ABMS member boards are currently developing or implementing. Lastly, the article presents contrasting views on the approach to professional self-regulation. One view operationalizes MOC as a high-stakes, pass-fail process while the other perspective holds MOC as an organized approach to support continuing professional development and improvement. The authors hope to begin a conversation among the AMA, the ABMS, and other professional stakeholders about how knowledge assessment in MOC might align with the MOC program's educational and quality improvement elements and best meet the future needs of both the public and the physician community.