A validity study of COMLEX-USA Level 3 with the new test design.

CONTEXT: The National Board of Osteopathic Medical Examiners (NBOME) administers the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA), a three-level examination designed for licensure for the practice of osteopathic medicine. The examination design for COMLEX-USA Level 3 (L3) was changed in September 2018 to a two-day computer-based examination with two components: a multiple-choice question (MCQ) component with single best answer and a clinical decision-making (CDM) case component with extended multiple-choice (EMC) and short answer (SA) questions. Continued validation of the L3 examination, especially with the new design, is essential for the appropriate interpretation and use of the test scores. OBJECTIVES: The purpose of this study is to gather evidence to support the validity of the L3 examination scores under the new design utilizing sources of evidence based on Kane's validity framework. METHODS: Kane's validity framework contains four components of evidence to support the validity argument: Scoring, Generalization, Extrapolation, and Implication/Decision. In this study, we gathered data from various sources and conducted analyses to provide evidence that the L3 examination is validly measuring what it is supposed to measure. These include reviewing content coverage of the L3 examination, documenting scoring and reporting processes, estimating the reliability and decision accuracy/consistency of the scores, quantifying associations between the scores from the MCQ and CDM components and between scores from different competency domains of the L3 examination, exploring the relationships between L3 scores and scores from a performance-based assessment that measures related constructs, performing subgroup comparisons, and describing and justifying the criterion-referenced standard setting process. The analysis data contains first-attempt test scores for 8,366 candidates who took the L3 examination between September 2018 and December 2019. The performance-based assessment utilized as a criterion measure in this study is COMLEX-USA Level 2 Performance Evaluation (L2-PE). RESULTS: All assessment forms were built through the automated test assembly (ATA) procedure to maximize parallelism in terms of content coverage and statistical properties across the forms. Scoring and reporting follows industry-standard quality-control procedures. The inter-rater reliability of SA rating, decision accuracy, and decision consistency for pass/fail classifications are all very high. There is a statistically significant positive association between the MCQ and the CDM components of the L3 examination. The patterns of associations, both within the L3 subscores and with L2-PE domain scores, fit with what is being measured. The subgroup comparisons by gender, race, and first language showed expected small differences in mean scores between the subgroups within each category and yielded findings that are consistent with those described in the literature. The L3 pass/fail standard was established through implementation of a defensible criterion-referenced procedure. CONCLUSIONS: This study provides some additional validity evidence for the L3 examination based on Kane's validity framework. The validity of any measurement must be established through ongoing evaluation of the related evidence. The NBOME will continue to collect evidence to support validity arguments for the COMLEX-USA examination series.

Clinical competence, communication ability and adherence to choosing wisely recommendations for lipid reducing drug use in older adults.

BACKGROUND: Although lipid-lowering drugs are not recommended for primary prevention in patients 75+, prevalence of use is high and there is unexplained variation in prescribing between physicians. The objective of this study was to determine if physician communication ability and clinical competence are associated with prescribing lipid-lowering drugs for primary and secondary prevention. METHODS: We used a cohort of 4,501 international medical graduates, 161,214 U.S. Medicare patients with hyperlipidemia (primary prevention) and 49,780 patients with a history of cardiovascular disease (secondary prevention) not treated with lipid-lowering therapy who were seen by study physicians in ambulatory care. Clinical competence and communication ability were measured by the ECFMG clinical assessment examination. Physician citizenship, age, gender, specialty and patient characteristics were also measured. The outcome was an incident prescription of lipid-lowering drug, evaluated using multivariable GEE logistic regression models for primary and secondary prevention for patients 75+ and 65-74. RESULTS: Patients 75+ were less likely than those 65-74 to receive lipid-lowering drugs for primary (OR 0.62, 95% CI 0.59-0.66) and secondary (OR 0.70, 95% CI 0.63-0.78) prevention. For every 20% increase in clinical competence score, the odds of prescribing therapy for primary prevention to patients 75+ increased by 24% (95% CI 1.02-1.5). Communication ability had the opposite effect, reducing the odds of prescribing for primary prevention by 11% per 20% score increase (95% CI 0.8-0.99) for both age groups. Physicians who were citizens of countries with higher proportions of Hispanic (South/Central America) or Asian (Asia/Oceania) people were more likely to prescribe treatment for primary prevention, and internal medicine specialists were more likely to treat for secondary prevention than primary care physicians. CONCLUSION: Clinical competence, communication ability and physician citizenship are associated with lipid-lowering drug prescribing for primary prevention in patients aged 75+.

Do clinical and communication skills scores on credentialing exams predict potentially inappropriate antibiotic prescribing?

BACKGROUND: There is considerable variation among physicians in inappropriate antibiotic prescribing, which is hypothesized to be attributable to diagnostic uncertainty and ineffective communication. The objective of this study was to evaluate whether clinical and communication skills are associated with antibiotic prescribing for upper respiratory infections and sinusitis. METHODS: A cohort study of 2,526 international medical graduates and 48,394 U.S. Medicare patients diagnosed by study physicians with an upper respiratory infection or sinusitis between July 2014 and November 2015 was conducted. Clinical and communication skills were measured by scores achieved on the Clinical Skills Assessment examination administered by the Educational Commission for Foreign Medical Graduates (ECFMG) as a requirement for entry into U.S residency programs. Medicare Part D data were used to determine whether patients were dispensed an antibiotic following an outpatient evaluation and management visit with the study physician. Physician age, sex, specialty and practice region were retrieved from the ECFMG databased and American Medical Association (AMA) Masterfile. Multivariate GEE logistic regression was used to evaluate the association between clinical and communication skills and antibiotic prescribing, adjusting for other physician and patient characteristics. RESULTS: Physicians prescribed an antibiotic in 71.1% of encounters in which a patient was diagnosed with sinusitis, and 50.5% of encounters for upper respiratory infections. Better interpersonal skills scores were associated with a significant reduction in the odds of antibiotic prescribing (OR per score decile 0.93, 95% CI 0.87-0.99), while greater proficiency in clinical skills and English proficiency were not. Female physicians, those practicing internal medicine compared to family medicine, those with citizenship from the US compared to all other countries, and those practicing in southern of the US were also more likely to prescribe potentially unnecessary antibiotics. CONCLUSIONS: Based on this study, physicians with better interpersonal skills are less likely to prescribe antibiotics for acute sinusitis and upper respiratory infections. Future research should examine whether tailored interpersonal skills training to help physicians manage patient expectations for antibiotics could reduce unnecessary antibiotic prescribing.

Comparison of Bedside and Video-Based Capillary Refill Time Assessment in Children.

OBJECTIVES: Capillary refill time (CRT) to assess peripheral perfusion in children with suspected shock may be subject to poor reproducibility. Our objectives were to compare video-based and bedside CRT assessment using a standardized protocol and evaluate interrater and intrarater consistency of video-based CRT (VB-CRT) assessment. We hypothesized that measurement errors associated with raters would be low for both standardized bedside CRT and VB-CRT as well as VB-CRT across raters. METHODS: Ninety-nine children (aged 1-12 y) had 5 consecutive bedside CRT assessments by an experienced critical care clinician following a standardized protocol. Each CRT assessment was video recorded on a black background. Thirty video clips (10 with bedside CRT < 1 s, 10 with CRT 1-2 s, and 10 with CRT > 2 s) were randomly selected and presented to 10 clinicians twice in randomized order. They were instructed to push a button when they visualized release of compression and completion of a capillary refill. The correlation and absolute difference between bedside and VB-CRT were assessed. Consistency across raters and within each rater was analyzed using the intraclass correlation coefficient (ICC). A Generalizability study was performed to evaluate sources of variation. RESULTS: We found moderate agreement between bedside and VB-CRT observations (r = 0.65; P < 0.001). The VB-CRT values were shorter by 0.17 s (95% confidence interval, 0.09-0.25; P < 0.001) on average compared with bedside CRT. There was moderate agreement in VB-CRT across raters (ICC = 0.61). Consistency of repeated VB-CRT within each rater was moderate (ICC = 0.71). Generalizability study revealed the source of largest variance was from individual patient video clips (57%), followed by interaction of the VB-CRT reviewer and patient video clip (10.7%). CONCLUSIONS: Bedside and VB-CRT observations showed moderate consistency. Using video-based assessment, moderate consistency was also observed across raters and within each rater. Further investigation to standardize and automate CRT measurement is warranted.

Medical Council of Canada Qualifying Examinations and performance in future practice.

The purpose of medical licensing examinations is to protect the public from practitioners who do not have adequate knowledge, skills, and abilities to provide acceptable patient care, and therefore evaluating the validity of these examinations is a matter of accountability. Our objective was to discuss the Medical Council of Canada's Qualifying Examinations (MCCQEs) Part I (QE1) and Part II (QE2) in terms of how well they reflect future performance in practice. We examined the supposition that satisfactory performance on the MCCQEs are important determinants of practice performance and, ultimately, patient outcomes. We examined the literature before the implementation of the QE2 (pre-1992), post QE2 but prior to the implementation of the new Blueprint (1992-2018), and post Blueprint (2018-present). The literature suggests that MCCQE performance is predictive of future physician behaviours, that the relationship between examination performance and outcomes did not attenuate with practice experience, and that associations between examination performance and outcomes made sense clinically. While the evidence suggests the MCC qualifying examinations measure the intended constructs and are predictive of future performance, the validity argument is never complete. As new competency requirements emerge, we will need to develop valid and reliable mechanisms for determining practice readiness in these areas.

L'objectif des examens donnant lieu au titre de Licencié du Conseil médical du Canada est de protéger le public en garantissant que les praticiens possèdent les connaissances, les habiletés et les aptitudes nécessaires pour offrir des soins satisfaisants aux patients; par conséquent, l'évaluation de la validité de ces examens est une question de responsabilité. Notre objectif était de déterminer dans quelle mesure l'Examen d'aptitude du Conseil médical du Canada (EACMC), partie I, et l'EACMC, partie II reflètent le rendement futur des médecins dans leur pratique.Nous avons examiné l'hypothèse selon laquelle des résultats satisfaisants aux EACMC sont des déterminants importants du rendement dans la pratique future et, ultimement, des résultats rapportés pour les patients. Nous avons examiné les écrits publiés avant l'introduction de l'EACMC,-partie II (avant 1992), post EACMC-partie II ci mais avant l'adoption du Plan directeur (1992-2018), ainsi que ceux publiés post adoption du Plan directeur (2018-présent).La littérature suggère que la performance à l'EACMC permet de prédire les comportements futurs des médecins, que le rapport entre la performance à l'examen et les résultats dans la pratique perdure, et que les associations entre la performance à l'examen et les résultats sont liés sur le plan clinique.Bien que les données probantes indiquent que les examens d'aptitude du CMC (EACMC) mesurent les concepts visés et permettent de prédire le rendement des médecins dans leur pratique future, la démarche de validité n'est pas complète. Au fur et à mesure que de nouvelles exigences en matière de compétences émergent, nous devrons élaborer des mécanismes valides et fiables pour déterminer la capacité à exercer dans ces domains.

Associations between initial American Board of Internal Medicine certification and maintenance of certification status of attending physicians and in-hospital mortality of patients with acute myocardial infarction or congestive heart failure: a retrospective cohort study of hospitalisations in Pennsylvania, USA.

OBJECTIVE: To determine whether internists' initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF). DESIGN: Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017. SETTING: All hospitals in Pennsylvania. PARTICIPANTS: All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist. PRIMARY OUTCOME MEASURE: In-hospital mortality. RESULTS: Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians' demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; p<0.001). Patients cared for by physicians with initial certification had a 15.87% decrease in mortality compared with those cared for by non-certified physicians (mortality rate difference of 5.09 per 1000 patients; 95% CI 2.12 to 8.05; p<0.001). The adjusted OR for MOC was 0.804 (95% CI 0.697 to 0.926; p=0.003). Patients cared for by physicians who completed MOC had an 18.91% decrease in mortality compared with those cared for by MOC lapsed physicians (mortality rate difference of 6.22 per 1000 patients; 95% CI 2.0 to 10.4; p=0.004). CONCLUSIONS: Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.

International Medical Graduates in the United States Psychiatry Workforce.

OBJECTIVE: This study describes the supply, distribution, and characteristics of international medical graduate (IMG) psychiatrists who provide services in the USA. METHODS: Cross-sectional study design, using descriptive statistics based on combined data from the American Medical Association (2020 Physician Masterfile) and the Educational Commission for Foreign Medical Graduates. RESULTS: International medical graduates continue to make significant contributions to the US physician workforce. As a group, they represent 29% of active psychiatrists in the USA, compared to 23% in all other medical specialties. Many IMG psychiatrists were US citizens who obtained their medical degrees outside the USA or Canada, often in the Caribbean. In some states (i.e., Florida, New Jersey), over 40% of active psychiatrists are IMGs. Over 30% of IMG psychiatrists graduated from medical schools in India and Pakistan. CONCLUSIONS: This study provides an overview of the psychiatric workforce in the USA, quantifying the specific contribution of IMGs. Several factors, including immigration policies, continued expansion of US medical schools, and the number of available residency positions, could impact the flow of IMGs to the US. Longitudinal studies are needed to better understand the implications for workforce composition and distribution, and their potential impact on the care of psychiatric patients.

A Concordance Study of COMLEX-USA and USMLE Scores.

BACKGROUND: Under the single GME accreditation system, residency programs receive applicants from MD- and DO-granting medical schools, each of which have their own set of licensing examinations, making concordance studies increasingly relevant. Previous studies comparing Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) and United States Medical Licensing Examination (USMLE) scores have been limited in sample size and examinee composition and have yielded comparisons that may not be generalizable across all applicants. Some osteopathic medical students take USMLE in addition to COMLEX-USA, often at considerable cost and effort, with the aim of making themselves more desirable to potential residency programs. Having more reliable comparisons of COMLEX-USA and USMLE scores would allow program directors to better estimate a score on the alternate examination. OBJECTIVE: To derive an accurate concordance between COMLEX-USA and USMLE scores, based on a large sample of osteopathic students who took both examinations. METHODS: Five colleges of osteopathic medicine, representing various regions of the United States, participated in this study. The data included demographics and COMLEX-USA and USMLE scores from September 2015 through August 2020 for students who took both examinations. We derived the concordance between COMLEX-USA and USMLE scores using equipercentile matching. RESULTS: Comparisons of demographic characteristics showed only minor differences between the sample and the overall population for COMLEX-USA takers, although scores for the study sample were, on average, greater. CONCLUSIONS: A strong association exists between the scores on the COMLEX-USA and USMLE examinations, allowing prediction of performance on USMLE from COMLEX-USA.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education.

PURPOSE: The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once. METHOD: In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores. RESULTS: G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13). CONCLUSIONS: While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

Making the grade: licensing examination performance by medical school accreditation status.

BACKGROUND: Accreditation systems strive to ensure the quality of undergraduate (basic) medical education and encourage ongoing improvements. Despite increasing global emphasis on quality assurance activities, there is limited research linking accreditation of medical education to improved student and graduate outcomes. The purpose of this study is to compare the United States Medical Licensing Examination® (USMLE®) performance of students and graduates who attended international medical schools accredited by an agency recognized by the World Federation of Medical Education (WFME) to individuals who attended schools that did not meet this criterion. METHODS: During the 2018-2020 study period, 39,650 individuals seeking Educational Commission for Foreign Medical Graduates® (ECFMG®) certification took one or more USMLE examinations. We cross-tabulated USMLE performance (first-attempt pass/fail result) and medical school accreditation status. RESULTS: Individuals seeking ECFMG certification who attended international medical schools accredited by an agency recognized by WFME had higher or comparable USMLE first-attempt pass rates compared to individuals who attended medical schools that did not meet this criterion. CONCLUSIONS: Implementing and maintaining meaningful accreditation systems requires substantial resources. These results provide important positive evidence that external evaluation of educational programs is associated, on average, with better educational outcomes, including in the domains of basic science, clinical knowledge, and clinical skills performance.

Association of clinical competence, specialty and physician country of origin with opioid prescribing for chronic pain: a cohort study.

BACKGROUND: Although little is known about why opioid prescribing practices differ between physicians, clinical competence, specialty training and country of origin may play a role. We hypothesised that physicians with stronger clinical competence and communication skills are less likely to prescribe opioids and prescribe lower doses, as do medical specialists and physicians from Asia. METHODS: Opioid prescribing practices were examined among international medical graduates (IMGs) licensed to practise in the USA who evaluated Medicare patients for chronic pain problems in 2014-2015. Clinical competence was assessed by the Educational Commission for Foreign Medical Graduates (ECFMG) Clinical Skills Assessment. Physicians in the ECFMG database were linked to the American Medical Association Masterfile. Patients evaluated for chronic pain were obtained by linkage to Medicare outpatient and prescription files. Opioid prescribing was measured within 90 days of evaluation visits. Prescribed dose was measured using morphine milligram equivalents (MMEs). Generalised estimating equation logistic and linear regression estimated the association of clinical competence, specialty, and country of origin with opioid prescribing and dose. RESULTS: 7373 IMGs evaluated 65 012 patients for chronic pain; 15.2% received an opioid prescription. Increased clinical competence was associated with reduced opioid prescribing, but only among female physicians. For every 10% increase in the clinical competence score, the odds of prescribing an opioid decreased by 16% for female physicians (OR 0.84, 95% CI 0.75 to 0.94) but not male physicians (OR 0.99, 95% CI 0.92 to 1.07). Country of origin was associated with prescribed opioid dose; US and Canadian citizens prescribed higher doses (adjusted MME difference +3.56). Primary care physicians were more likely to prescribe opioids, but surgical and hospital-based specialists prescribed higher doses. CONCLUSIONS: Clinical competence at entry into US graduate training, physician gender, specialty and country of origin play a role in opioid prescribing practices.

International Medical Graduate Resident Wellness: Examining Qualitative Data From J-1 Visa Physician Recipients.

PURPOSE: International medical graduates (IMGs), approximately 25% of the U.S. physician workforce, have unique needs as they enter residency programs. This study identified wellness barriers and challenges that IMGs encounter as they transition to the United States. METHOD: The authors analyzed results from 3 open-ended questions in a 21-item survey. This survey was administered in December 2019 to 11,504 IMG resident physicians sponsored by the Educational Commission for Foreign Medical Graduates' J-1 visa program. These questions asked respondents to describe challenges to their wellness, how they maintain wellness, and resources that would have aided their transition. Data were analyzed using a mixed-methods approach, including both qualitative descriptions and category frequencies. RESULTS: Of the surveys administered, 7,817 responses (68% response rate) were received. Respondents identified challenges navigating cultural differences (1,314, 17%), health care system (1,108, 14%), distance from family and friends (890, 11%), bureaucratic barriers (724, 9%), and language/communication and finances (575, 7%; 565, 7%, respectively). They also specified that friendships/relationships (2,800, 36%) followed by exercise (2,318, 30%), family (1,822, 23%), socialization (1,001, 13%), and healthy eating (775, 10%) were factors important to their wellness. Respondents requested more information about socialization (741, 9%), bureaucratic support (456, 6%), IMG support networks (427, 5%), financial support (404, 5%), and greater online resources (240, 3%). CONCLUSIONS: IMGs have needs and concerns specific to their demographic group. Participants' responses suggested that they wanted additional support in the workplace and their personal lives. Answers also indicated that IMGs experienced a unique set of stressors such as fluctuating immigration laws that U.S. medical graduates do not face. Finally, this study supports a body of research that connects social and physical wellness. By identifying and describing these challenges, the authors seek to inform the development of specific programs and resources to improve IMG resident wellness.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific.

INTRODUCTION: Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established. METHODS: Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters. RESULTS: The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability. CONCLUSIONS: The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

Medical School Accreditation Factors Associated With Certification by the Educational Commission for Foreign Medical Graduates (ECFMG): A 10-Year International Study.

PURPOSE: In 2024, international medical graduates seeking Educational Commission for Foreign Medical Graduates (ECFMG) certification will be required to graduate from an accredited medical school. This study's goal was to examine relationships between medical school accreditation variables and ECFMG certification for a global sample. METHOD: Using ECFMG databases, the authors created a 10-year cohort (January 1, 2007-December 31, 2016) of certification applicants, defined as individuals who had attempted at least 2 examinations required for certification. The authors aggregated applicant data at the school level, excluding schools with < 80 applicants. School accreditation statuses were based on agency websites. School region, age, and time of first accreditation were included. Analyses included descriptive and bivariate statistics and multiple linear regressions adjusting for school start year and year of first accreditation. RESULTS: The cohort included 128,046 applicants from 1,973 medical schools across 162 countries. After excluding low-volume schools, 318 schools across 81 countries remained. These provided 99,598 applicants and 77,919 certificate holders, three-quarters of whom came from the Caribbean, South-Central Asia, and West Asia regions. Two hundred and fifty (78.6%) schools were accredited; 68 (21.4%) were not. Most ECFMG applicants (n = 84,776, 85.1%) and certificate holders (n = 68,444, 87.8%) attended accredited medical schools. Accredited schools had higher rates of ECFMG certification among graduates than nonaccredited schools in comparisons that included all schools (75.0% [standard deviation (SD) = 10.6%] vs 68.3% (SD = 15.9%), P < .001), and for countries that had both accredited and nonaccredited schools (73.9% [SD = 11.4%] vs 67.3% [SD = 16.8%], P = .023). After adjusting for age of school, longer duration of accreditation was associated with higher certification rates (P < .001). CONCLUSIONS: Accreditation had a positive association with ECFMG certification rates. Future studies should investigate how accreditation processes might account for higher certification rates.

Competence in Decision Making: Setting Performance Standards for Critical Care.

BACKGROUND: Health care professionals must be able to make frequent and timely decisions that can alter the illness trajectory of intensive care patients. A competence standard for this ability is difficult to establish yet assuring practitioners can make appropriate judgments is an important step in advancing patient safety. We hypothesized that simulation can be used effectively to assess decision-making competence. To test our hypothesis, we used a "standard-setting" method to derive cut scores (standards) for 16 simulated ICU scenarios targeted at decision-making skills and applied them to a cohort of critical care trainees. METHODS: Panelists (critical care experts) reviewed digital audio-video performances of critical care trainees managing simulated critical care scenarios. Based on their collectively agreed-upon definition of "readiness" to make decisions in an ICU setting, each panelist made an independent judgment (ready, not ready) for a large number of recorded performances. The association between the panelists' judgments and the assessment scores was used to derive scenario-specific performance standards. RESULTS: For all 16 scenarios, the aggregate panelists' ratings (ready/not ready for independent decision making) were positively associated with the performance scores, permitting derivation of performance standards for each scenario. CONCLUSIONS: Minimum competence standards for high-stakes decision making can be established through standard-setting techniques. We effectively identified "front-line" providers who are, or are not, ready to make independent decisions in an ICU setting. Our approach may be used to assure stakeholders that clinicians are competent to make appropriate judgments. Further work is needed to determine whether our approach is effective in simulation-based assessments in other domains.

International Medical Graduates in the Pediatric Workforce in the United States.

BACKGROUND AND OBJECTIVES: To describe the supply, distribution, and characteristics of international medical graduates (IMGs) in pediatrics who provide patient care in the United States. METHODS: Cross-sectional study, combining data from the 2019 Physician Masterfile of the American Medical Association and the Educational Commission for Foreign Medical Graduates database. RESULTS: In total, 92 806 pediatric physicians were identified, comprising 9.4% of the entire US physician workforce. Over half are general pediatricians. IMGs account for 23.2% of all general pediatricians and pediatric subspecialists. Of all IMGs in pediatrics, 22.1% or 4775 are US citizens who obtained their medical degree outside the United States or Canada, and 15.4% (3246) attended medical school in the Caribbean. Fifteen non-US medical schools account for 29.9% of IMGs currently in active practice in pediatrics in the United States. IMGs are less likely to work in group practice or hospital-based practice and are more likely to be employed in solo practice (compared with US medical school graduates). CONCLUSIONS: With this study, we provide an overview of the pediatric workforce, quantifying the contribution of IMGs. Many IMGs are US citizens who attend medical school abroad and return to the United States for postgraduate training. Several factors, including the number of residency training positions, could affect future numbers of IMGs entering the United States. Longitudinal studies are needed to better understand the implications that workforce composition and distribution may have for the care of pediatric patients.

Canadians studying medicine abroad and their journey to secure postgraduate training in Canada or the United States.

BACKGROUND: From national and international workforce perspectives, Canadians studying medicine abroad (CSAs) are a growing provider group. Some were born in Canada whereas others immigrated as children. They study medicine in various countries, often attempting both American and Canadian medical licensure pathways. METHODS: Using data from the Educational Commission for Foreign Medical Graduates (ECFMG) and the Medical Council of Canada (MCC), we looked at CSAs who attempted to secure residency positions in both Canada and the United States. We detailed the CSAs' countries of birth and medical education. We tracked these individuals through their postgraduate education programs to enumerate their success rate and categorize the geographic locations of their training. RESULTS: The majority of CSAs study medicine in one of 10 countries. The remainder are disbursed across 88 other countries. Most CSAs were born in Canada (62%). Approximately 1/3 of CSA from the 2004-2016 cohort had no record of entering a residency program in Canada or the United States (U.S.). Recently graduated CSAs were most likely to secure residency training in Ontario and New York. CONCLUSION: Many CSAs attempt to secure residency training in both Canada and the U.S. Quantifying success rates may be helpful for Canadians thinking about studying medicine abroad. Understanding the educational pathways of CSAs will be useful for physician labour workforce planning.

CONTEXTE: Selon une perspective nationale et internationale des effectifs, les Canadiens qui étudient la médecine à l'étranger (CEE) représentent un groupe en croissance. Certains sont nés au Canada, alors que d'autres ont immigré durant leur enfance. Ils étudient la médecine dans divers pays, essayant souvent parallèlement d'obtenir un permis américain et canadien pour exercer la médecine. MÉTHODES: À l'aide de données de l'Educational Commission for Foreign Medical Graduates (ECFMG) et du Conseil médical du Canada (CMC), nous avons examiné les CEE qui avaient tenté d'obtenir des postes de résidence à la fois au Canada et aux États-Unis. Nous avons extrait des données quant au pays de naissance et à la formation médicale de ces CEE. Nous avons suivi ces personnes dans leurs processus de demande d'admission à des programmes de formation postdoctorale pour rapporter leur taux de succès et catégoriser les emplacements géographiques de leur formation. RÉSULTATS: Nous avons identifié 10 pays d'où provenaient la plupart de ces CEE. Les autres CEE provenaient de 88 autres pays. La plupart de ces CEE sont nées au Canada (62 %). Environ 1/3 des CEE de la cohorte de 2004 à 2016 ne possède pas de dossier d'inscription à un programme de résidence au Canada ou aux États-Unis. Les CEE récemment diplômés étaient les plus susceptibles de suivre une formation en résidence en Ontario et dans l'État de New York. CONCLUSION: De nombreux CEE ont tenté d'obtenir un poste de résidence au Canada et aux États-Unis. Quantifier les taux de succès pourrait se révéler utile pour les Canadiens qui pensent à étudier la médecine à l'étranger. Comprendre les parcours éducatifs des CEE sera utile à la planification des effectifs médicaux.

Prevalence of International Medical Graduates From Muslim-Majority Nations in the US Physician Workforce From 2009 to 2019.

Importance: Historically, the US physician workforce has included a large number of international medical graduates (IMGs). Recent US immigration policies may affect the inflow of IMGs, particularly those who are citizens of Muslim-majority nations. Objectives: To provide an overview of the characteristics of IMGs from Muslim-majority nations, including their contributions to the US physician workforce, and to describe trends in the number of applications for certification to the Educational Commission for Foreign Medical Graduates between 2019 and 2018, both overall and for citizens of Muslim-majority nations. Design, Setting, and Participants: This cross-sectional study, which included 1 065 606 US physicians listed in the 2019 American Medical Association Physician Masterfile and 156 017 applicants to the Educational Commission for Foreign Medical Graduates certification process between 2009 and 2018, used a repeated cross-sectional study design to review the available data, including country of medical school attended, citizenship when entering medical school, and career information, such as present employment, specialty, and type of practice. Exposures: Country of citizenship when entering medical school. Main Outcomes and Measures: Physician counts and demographic information from the 2019 American Medical Association Physician Masterfile and applicant data from the Educational Commission for Foreign Medical Graduates from 2009 to 2018. Results: Of 1 065 606 physicians in the American Medical Association Physician Masterfile, 263 029 (24.7%) were IMGs, of whom 48 354 were citizens of Muslim-majority countries at time of entry to medical school, representing 18.4% of all IMGs. Overall, 1 in 22 physicians in the US was an IMG from a Muslim-majority nation, representing 4.5% of the total US physician workforce. More than half of IMGs from Muslim-majority nations (24 491 [50.6%]) come from 3 countries: Pakistan (14 352 [29.7%]), Iran (5288 [10.9%]), and Egypt (4851 [10.0%]). The most prevalent specialties include internal medicine (10 934 [23.6%]), family medicine (3430 [7.5%]), pediatrics (2767 [5.9%]), and psychiatry (2251 [4.8%]), with 18 229 (38.1%) practicing in primary care specialties. The number of applicants for Educational Commission for Foreign Medical Graduates certification from Muslim-majority countries increased from 2009 (3227 applicants) to 2015 (4244 applicants), then decreased by 2.1% in 2016 to 4254 applicants, 4.3% in 2017 to 4073 applicants, and 11.5% in 2018 to 3604 applicants. Much of this decrease could be attributed to fewer citizens from Pakistan (1042 applicants in 2015 to 919 applicants in 2018), Egypt (493 applicants in 2015 to 309 applicants in 2018), Iran (281 applicants in 2015 to 182 applicants in 2018), and Saudi Arabia (337 applicants in 2015 to 163 applicants in 2018) applying for certification. Conclusions and Relevance: Based on the findings of this study, the number of ECFMG applicants from Muslim-majority countries decreased from 2015 to 2018. The US physician workforce will continue to rely on IMGs for some time to come. To the extent that citizens from some countries no longer seek residency positions in the US, gaps in the physician workforce could widen.