Predictors for success and failure in international medical graduates: a systematic review of observational studies.

BACKGROUND: International Medical Graduates (IMG) are an essential part of the international physician workforce, and exploring the predictors of success and failure for IMGs could help inform international and national physician labour workforce selection and planning. The objective of this study was to explore predictors for success for selection of IMGs into high stakes postgraduate training positions and practice and not necessarily for informing IMGs. METHODS: We searched 11 databases, including Medline, Embase and LILACS, from inception to February 2022 for studies that explored the predictors of success and failure in IMGs. We reported baseline probability, effect size in relative risk (RR), odds ratio (OR) or hazard ratio (HR) and absolute probability change for success and failure across six groups of outcomes, including success in qualifying exams, or certificate exams, successful matching into residency, retention in practice, disciplinary actions, and outcomes of IMG clinical practice. RESULTS: Twenty-five studies (375,549 participants) reported the association of 93 predictors of success and failure for IMGs. Female sex, English fluency, graduation recency, higher scores in USMLE step 2 and participation in a skill assessment program were associated with success in qualifying exams. Female sex, English fluency, previous internship and results of qualifying exams were associated with success in certification exams. Retention to work in Canada was associated with several factors, including male sex, graduating within the past five years, and completing residency over fellowships. In the UK, IMGs and candidates who attempted PLAB part 1, ≥ 4 times vs. first attempters, and candidates who attempted PLAB part 2, ≥ 3 times vs. first attempters were more likely to be censured in future practice. Patients treated by IMGs had significantly lower mortalities than those treated by US graduates, and patients of IMGs had lower mortalities [OR: 0.82 (95% CI: 0.62, 0.99)] than patients of US citizens who trained abroad. CONCLUSIONS: This study informed factors associated with the success and failure of IMGs and is the first systematic review on this topic, which can inform IMG selection and future studies. SYSTEMATIC REVIEW REGISTRATION: PROSPERO: CRD42021252678.

Does spaced education improve clinical knowledge among Family Medicine residents? A cluster randomized controlled trial.

Spaced education is a learning strategy to improve knowledge acquisition and retention. To date, no robust evidence exists to support the utility of spaced education in the Family Medicine residency. We aimed to test whether alerts to encourage spaced education can improve clinical knowledge as measured by scores on the Canadian Family Medicine certification examination. METHOD: We conducted a cluster randomized controlled trial to empirically and pragmatically test spaced education using two versions of the Family Medicine Study Guide mobile app. 12 residency training programs in Canada agreed to participate. At six intervention sites, we consented 335 of the 654 (51%) eligible residents. Residents in the intervention group were sent alerts through the app to encourage the answering of questions linked to clinical cases. At six control sites, 299 of 586 (51%) residents consented. Residents in the control group received the same app but with no alerts. Incidence rates of case completion between trial arms were compared using repeated measures analysis. We linked residents in both trial arms to their knowledge scores on the certification examination of the College of Family Physicians of Canada. RESULTS: Over 67 weeks, there was no statistically significant difference in the completion of clinical cases by participants. The difference in mean exam scores and the associated confidence interval did not exceed the pre-defined limit of 4 percentage points. CONCLUSION: Further research is recommended before deploying spaced educational interventions in the Family Medicine residency to improve knowledge.

Leadership training in a family medicine residency program: Cross-sectional quantitative survey to inform curriculum development.

OBJECTIVE: To assess the current status of leadership training as perceived by family medicine residents to inform the development of a formal leadership curriculum. DESIGN: Cross-sectional quantitative survey. SETTING: Department of Family Medicine at McMaster University in Hamilton, Ont, in December 2013. PARTICIPANTS: A total of 152 first- and second-year family medicine residents. MAIN OUTCOME MEASURES: Family medicine residents' attitudes toward leadership, perceived level of training in various leadership domains, and identified opportunities for leadership training. RESULTS: Overall, 80% (152 of 190) of residents completed the survey. On a Likert scale (1 = strongly disagree, 4 = neutral, 7 = strongly agree), residents rated the importance of physician leadership in the clinical setting as high (6.23 of 7), whereas agreement with the statement "I am a leader" received the lowest rating (5.28 of 7). At least 50% of residents desired more training in the leadership domains of personal mastery, mentorship and coaching, conflict resolution, teaching, effective teamwork, administration, ideals of a healthy workplace, coalitions, and system transformation. At least 50% of residents identified behavioural sciences seminars, a lecture and workshop series, and a retreat as opportunities to expand leadership training. CONCLUSION: The concept of family physicians as leaders resonated highly with residents. Residents desired more personal and system-level leadership training. They also identified ways that leadership training could be expanded in the current curriculum and developed in other areas. The information gained from this survey might facilitate leadership development among residents through application of its results in a formal leadership curriculum.

Test ordering for preventive health care among family medicine residents.

OBJECTIVE: To determine which screening tests family medicine residents order as part of preventive health care. DESIGN: A cross-sectional survey. SETTING: Alberta and Ontario. PARTICIPANTS: First- and second-year family medicine residents at the University of Alberta in Edmonton, the University of Calgary in Alberta, and McMaster University in Hamilton, Ont, during the 2011 to 2012 academic year. MAIN OUTCOME MEASURES: Demographic information, Likert scale ratings assessing ordering attitudes, and selections from a list of 38 possible tests that could be ordered for preventive health care for sample 38-year-old and 55-year-old female and male patients. Descriptive and comparative statistics were calculated. RESULTS: A total of 318 of 482 residents (66%) completed the survey. Recommended or appropriate tests were ordered by 82% (for cervical cytology) to 95% (for fasting glucose measurement) of residents. Across the different sample patients, residents ordered an average of 3.3 to 5.7 inappropriate tests per patient, with 58% to 92% ordering at least 1 inappropriate test per patient. The estimated average excess costs varied from $38.39 for the 38-year-old man to $106.46 for the 55-year-old woman. More regular use of a periodic health examination screening template did not improve ordering (P = .88). CONCLUSION: In general, residents ordered appropriate preventive health tests reasonably well but also ordered an average of 3.3 to 5.7 inappropriate tests for each patient. Training programs need to provide better education for trainees around inappropriate screening and work hard to establish good ordering behaviour in preparation for entering practice.

Predicting international medical graduate success on college certification examinations: responding to the Thomson and Cohl judicial report on IMG selection.

OBJECTIVE: To determine predictors of international medical graduate (IMG) success in accordance with the priorities highlighted by the Thomson and Cohl judicial report on IMG selection. DESIGN: Retrospective assessment using regression analyses to compare the information available at the time of resident selection with those trainees' national certification examination outcomes. SETTING: McMaster University in Hamilton, Ont. PARTICIPANTS: McMaster University IMG residents who completed the program between 2005 and 2011. MAIN OUTCOME MEASURES: Associations between IMG professional experience or demographic characteristics and examination outcomes. RESULTS: The analyses revealed that country of study and performance on the Medical Council of Canada Evaluating Examination are among the predictors of performance on the College of Family Physicians of Canada and the Royal College of Physicians and Surgeons of Canada certification examinations. Of interest, the analyses also suggest discipline-specific relationships between previous professional experience and examination success. CONCLUSION: This work presents a useful technique for further improving our understanding of the performance of IMGs on certification examinations in North America, encourages similar interinstitutional analyses, and provides a foundation for the development of tools to assist with IMG education.

Selection of international medical graduates into postgraduate training positions in Canada. Who applies? Who is selected?

Background: International medical graduates (IMGs) are an essential part of the Canadian physician workforce. Considering current pressures on the health care system, an update regarding application numbers and match rates for IMGs to postgraduate positions in Canada is needed. Methods: We conducted a quantitative cross-sectional study to explore the characteristics of IMGs who are currently applying to the Canadian Residency Matching Service (CaRMS) positions to gain a broad understanding of the composition of this group and the factors associated with successful matching. Results: Out of 1,725 applicants in 2019, 14.1% matched on the first attempt and 6.4% after two to three attempts. Only 22.7% matched with a position (57.6% women). Applicants submitted an average 19.6 site/program applications. The percentage of IMGs matched did not statistically differ by gender. The relationship between the year of graduation or geographic area of medical school qualified and matching was significant for the first and second iterations, with current-year graduates and Oceania/Pacific Islands applicants more likely to match. Conclusions: This study provided us with accurate numbers and information about the Canadians studying abroad and IMG groups applying, and factors associated with being matched to the IMG positions through CaRMS, which will be instrumental in informing future selection implications for Canada.

Contexte: Les diplômés hors du Canada et des États-Unis (DHCEU) constituent un élément essentiel de la main-d'œuvre médicale au Canada. Compte tenu des pressions qui s'exercent actuellement sur le système de santé, il est nécessaire de faire le point sur le nombre de candidatures et les taux de jumelage des DHCEU à des postes de résidence au Canada. Méthodes: Nous avons mené une étude quantitative transversale pour explorer les caractéristiques des DHCEU qui postulent actuellement aux postes du Service canadien de jumelage des résidents (CaRMS) afin de mieux comprendre la composition de ce groupe et les facteurs associés à un jumelage réussi. Résultats: Sur 1 725 candidats en 2019, 14,1 % ont été jumelés dès leur première tentative et 6,4 % après 2 ou 3 tentatives. Seulement 22,7 % des candidats ont obtenu un poste (57,6 % de femmes). En moyenne, les candidats ont soumis des demandes à 19,6 endroits/programmes. Le pourcentage de DHCEU jumelés n'était pas statistiquement différent selon le sexe. La relation entre l'année d'obtention du diplôme ou la zone géographique de la faculté de médecine où il avait été obtenu et le jumelage était significative pour le premier et le deuxième tours, les diplômés de l'année en cours et les candidats de l'Océanie/îles du Pacifique étant plus susceptibles d'être jumelés. Conclusions: Cette étude nous a fourni des chiffres et des renseignements précis sur les Canadiens qui étudient à l'étranger et les groupes de DHCEU qui posent leur candidature aux postes destinés aux DHCEU dans le cadre du CaRMS, ainsi que sur les facteurs associés à un jumelage réussi, ce qui contribuera à guider la sélection des futurs candidats au Canada.

Interventions to improve medication reconciliation in primary care.

OBJECTIVE: To systematically review all primary care intervention studies designed to implement medication reconciliation for effects on medication discrepancies, clinical outcomes, and patient knowledge of their medications. DATA SOURCES: We searched MEDLINE (1988-March 2008); Healthstar (1966-March 2008); CINAHL (1982-March 2008); EMBASE (1980-March 2008); Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects, Cochrane Methodology Register, and Health Technology Assessments; and unpublished material. No language restrictions were applied. Search terms included medication reconciliation, medication errors, prescribing error, medication systems, adverse drug events, drug utilization review, medication list, medication record, and medications management. STUDY SELECTION AND DATA ABSTRACTION: Randomized controlled trials or before-and-after studies that examined the effect of various interventions on medication discrepancies either in ambulatory settings or at hospital discharge among community-dwelling adults were included. Two reviewers independently assessed studies to determine inclusion. Level of agreement between the reviewers was good, with unweighted Cohen's kappa of 0.71. Two of 3 independent reviewers abstracted data and evaluated validity from included studies. Disagreements between reviewers were resolved by consensus. DATA SYNTHESIS: Four trials met the inclusion criteria. Two before-and-after studies (n = 275) in ambulatory care examining systematic medication reconciliation at each visit produced conflicting results. One study showed a reduction in the proportion of medication discrepancies from 88.5% to 49.1% (OR 0.13; 95% CI 0.07 to 0.21); the other showed no benefit. One randomized controlled trial and one before-and-after study (n = 202) evaluated pharmacist medication review at hospital discharge. Neither showed a benefit. Heterogeneity precluded pooling of studies. All included studies had significant design flaws. CONCLUSIONS: There is no good quality evidence demonstrating the effectiveness of medication reconciliation in the primary care setting. Further research is needed.

Design Process and Utilization of a Novel Clinical Decision Support System for Neuropathic Pain in Primary Care: Mixed Methods Observational Study.

BACKGROUND: Computerized clinical decision support systems (CDSSs) have emerged as an approach to improve compliance of clinicians with clinical practice guidelines (CPGs). Research utilizing CDSS has primarily been conducted in clinical contexts with clear diagnostic criteria such as diabetes and cardiovascular diseases. In contrast, research on CDSS for pain management and more specifically neuropathic pain has been limited. A CDSS for neuropathic pain has the potential to enhance patient care as the challenge of diagnosing and treating neuropathic pain often leads to tension in clinician-patient relationships. OBJECTIVE: The aim of this study was to design and evaluate a CDSS aimed at improving the adherence of interprofessional primary care clinicians to CPG for managing neuropathic pain. METHODS: Recommendations from the Canadian CPGs informed the decision pathways. The development of the CDSS format and function involved participation of multiple stakeholders and end users in needs assessment and usability testing. Clinicians, including family medicine physicians, residents, and nurse practitioners, in three academic teaching clinics were trained in the use of the CDSS. Evaluation over one year included the measurement of utilization of the CDSS; change in reported awareness, agreement, and adoption of CPG recommendations; and change in the observed adherence to CPG recommendations. RESULTS: The usability testing of the CDSS was highly successful in the prototype environment. Deployment in the clinical setting was partially complete by the time of the study, with some limitations in the planned functionality. The study population had a high level of awareness, agreement, and adoption of guideline recommendations before implementation of CDSS. Nevertheless, there was a small and statistically significant improvement in the mean awareness and adoption scores over the year of observation (P=.01 for mean awareness scores at 6 and 12 months compared with baseline, for mean adoption scores at 6 months compared with baseline, and for mean adoption scores at 12 months). Documenting significant findings related to diagnosis of neuropathic pain increased significantly. Clinicians accessed CPG information more frequently than they utilized data entry functions. Nurse practitioners and first year family medicine trainees had higher utilization than physicians. CONCLUSIONS: We observed a small increase in the adherence to CPG recommendations for managing neuropathic pain. Clinicians utilized the CDSS more as a source of knowledge and as a training tool than as an ongoing dynamic decision support.

Validation of the GALS musculoskeletal screening exam for use in primary care: a pilot study.

BACKGROUND: As the proportion of the Canadian population > or =65 grows, so too does the prevalence of musculoskeletal (MSK) conditions. Approximately 20% of visits to family physicians occur as a result of MSK complaints. The GALS (Gait, Arms, Legs, and Spine) screening examination was developed to assist in the detection of MSK abnormalities. Although MSK exams are primarily performed by rheumatologists or other MSK specialists, expanding their use in primary health care may improve the detection of MSK conditions allowing for earlier treatment. The primary goal of this study was to evaluate the use of the GALS locomotor screen in primary care by comparing the results of assessments of family physicians with those of rheumatologists. The secondary goal was to examine the incidence of MSK disorders and assess the frequency with which new diagnoses not previously documented in patients' charts were identified. METHODS: Patients > or =65 years old recruited from an academic family health centre were examined by a rheumatologist and a family physician who recorded the appearance of each participant's gait and the appearance and movement of the arms, legs and spine by deeming them normal or abnormal. GALS scores were compared between physicians with the proportion of observed (Pobs), positive (Ppos) and negative (Pneg) agreement being the primary outcomes. Kappa statistics were also calculated. Descriptive statistics were used to describe the number of "new" diagnoses by comparing rheumatologists' findings with each patient's family practice chart. RESULTS: A total of 99 patients consented to participate (92 with previously diagnosed MSK conditions). Results showed reasonable agreement between family physicians and rheumatologists; Pobs = 0.698, Ppos = 0.614 and Pneg = 0.752. The coefficient of agreement (estimated Kappa) was 0.3675 for the composite GALS score. For individual components of the GALS exam, the highest agreement between family physicians and rheumatologists was in the assessment of gait and arm movement. CONCLUSION: Previously reported increases in undiagnosed signs and symptoms of musculoskeletal conditions have highlighted the need for a simple yet sensitive screening exam for the identification of musculoskeletal abnormalities. Results of this study suggest that family physicians can efficiently use the GALS examination in the assessment of populations with a high proportion of musculoskeletal issues.

Admission factors associated with international medical graduate certification success: a collaborative retrospective review of postgraduate medical education programs in Ontario.

BACKGROUND: The failure rate on certification examinations of The College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC) is significantly higher for international medical graduates than for Canadian medical school graduates. The purpose of the current study was to generate evidence that supports or refutes the validity of hypotheses proposed to explain the lower success rates. METHODS: We conducted retrospective analyses of admissions and certification data to determine the factors associated with success of international medical graduate residents on the certification examinations. International medical graduates who entered an Ontario residency program between 2005 and 2012 and had written a certification examination by the time of the analysis (2015) were included in the study. Data available at the time of admission for each resident, including demographic characteristics, previous experiences and previous professional experiences, were collected from each of the 6 Ontario medical schools and matched with certification examination results provided by The CFPC and the RCPSC. We developed logistic regression models to determine the association of each factor with success on the examinations. RESULTS: Data for 900 residents were analyzed. The models revealed resident age to be strongly associated with performance across all examinations. Fluency in English, female sex and the Human Development Index value associated with the country of medical school training had differential associations across the examinations. INTERPRETATION: The findings should contribute to an improved understanding of certification success by international medical graduates, help residency programs identify at-risk residents and underpin the development of specific educational and remedial interventions. In considering the results, it should be kept in mind that some variables are not amenable to changes in selection criteria.

Do Weekly Alerts From a Mobile Application Influence Reading During Residency?

BACKGROUND: The benefits of "spaced education" have been documented for residents in highly focused specialties. We found no published studies of spaced education in family medicine. In this study, we report on the feasibility of delivering weekly alerts from a mobile application (app) developed for exam preparation, to increase the reading of clinical information in the family medicine residency. DESIGN: This is a 2-phase mixed methods study. Phase one is a quasi-experimental study of resident reading of information related to priority topics in family medicine. Reading was documented by page views in a noncommercial mobile app. PARTICIPANTS: All incoming first-year residents at two university training programs in Canada. The intervention group received one alert per week to priority topics on the app, beginning in their second month of residency. The control group was given access to the same app, but received no alerts. RESULTS: In this paper, we report the phase one preliminary findings. In the intervention group, 81 of 96 first year residents consented. At the control site, 79 of 85 residents consented. After 100 days, intervention group residents had viewed more pages of clinical information across all 99 priority topics (1,546 versus 900) and per topic (15.7 versus 9.1 pages, P < 0.0003). On average, each increase of one visit to the app following a weekly alert was associated with an increase of 3.2 visits to pages of clinical information in the app. CONCLUSION: A weekly alert delivered via mobile app shows promise with respect to reading in the family medicine residency.

A Clinical Decision Support System for Chronic Pain Management in Primary Care: Usability testing and its relevance.

BACKGROUND: Clinical decision support systems (CDSSs) that are integrated into electronic medical records may be useful for encouraging practice change compliant with clinical practice guidelines. OBJECTIVE: To engage end users to inform early phase CDSS development through a process of usability testing. METHODS: A sequential exploratory mixed method approach was used. Interprofessional clinician participants (seven in iteration 1 and six in iteration 2) were asked to 'think aloud' while performing various tasks on the CDSS and then complete the System Usability Scale (SUS). Changes were made to the CDSS after each iteration.Results Barriers and facilitators were identified: systemic; user interface (most numerous barriers); content (most numerous facilitators) and technical. The mean SUS score was 81.1 (SD = 12.02) in iteration 1 and 70.40 (SD = 6.78) in iteration 2 (p > 0.05). CONCLUSIONS: Qualitative data from usability testing were valuable in the CDSS development process. SUS scores were of limited value at this development stage.