Association between regional quota program in medical schools and practical clinical competency based on General Medicine In-Training Examination score: a nationwide cross-sectional study of resident physicians in Japan.

PURPOSE: A regional quota program (RQP) was introduced in Japan to ameliorate the urban-rural imbalance of physicians. Despite concerns about the low learning abilities of RQP graduates, the relationship between the RQP and practical clinical competency after initiating clinical residency has not been evaluated. METHODS: We conducted a nationwide cross-sectional study to assess the association between the RQP and practical clinical competency based on General Medicine In-Training Examination (GM-ITE) scores. We compared the overall and category GM-ITE results between RQP graduates and other resident physicians. The relationship between the RQP and scores was examined using multilevel linear regression analysis. RESULTS: There were 4978 other resident physicians and 1119 RQP graduates out of 6097 participants from 593 training hospitals. Being younger; preferring internal, general, or emergency medicine; managing fewer inpatients; and having fewer ER shifts were all characteristics of RQP graduates. In multilevel multivariable linear regression analysis, there was no significant association between RQP graduates and total GM-ITE scores (coefficient: 0.26; 95% confidence interval: -0.09, 0.61; P = .15). The associations of RQP graduates with GM-ITE scores in each category and specialty were not clinically relevant. However, in the same multivariable model, the analysis did reveal that total GM-ITE scores demonstrated strong positive associations with younger age and GM preference, both of which were significantly common in RQP graduates. CONCLUSION: Practical clinical competency evaluated based on the GM-ITE score showed no clinically relevant differences between RQP graduates and other resident physicians. Key messages What is already known on this topic Many countries offer unique admission processes to medical schools and special undergraduate programs to increase the supply of physicians in rural areas. Concerns have been raised about the motivation, learning capabilities, and academic performance of the program graduates. What this study adds This nationwide cross-sectional study in Japan revealed clinical competency based on the scores from the General Medicine In-Training Examination showed no clinically relevant differences between graduates of regional quota programs and other resident physicians. How this study might affect research, practice, or policy The study provides evidence to support the Japanese regional quota program from the perspective of clinical competency after initiating clinical practice.

Association between prolonged weekly duty hours and self-study time among residents: a cross-sectional study.

PURPOSE: In 2024, the Japanese government will enforce a maximum 80-hour weekly duty hours (DHs) regulation for medical residents. Although this reduction in weekly DHs could increase the self-study time (SST) of these residents, the relationship between these two variables remains unclear. The aim of the study was to investigate the relationship between the SST and DHs of residents in Japan. METHODS: In this nationwide cross-sectional study, the subjects were candidates of the General Medicine In-Training Examination in the 2020 academic year. We administered questionnaires and categorically asked questions regarding daily SST and weekly DHs during the training period. To account for hospital variability, proportional odds regression models with generalized estimating equations were used to analyse the association between SST and DHs. RESULTS: Of the surveyed 6117 residents, 32.0% were female, 49.1% were postgraduate year-1 residents, 83.8% were affiliated with community hospitals, and 19.9% worked for ≥80 hours/week. Multivariable analysis revealed that residents working ≥80 hours/week spent more time on self-study than those working 60-70 hours/week. Conversely, residents who worked <50 hours/week spent less time on self-study than those who worked 60-70 hours/week. The factors associated with longer SST were sex, postgraduate year, career aspiration for internal medicine, affiliation with community hospitals, academic involvement, and well-being. CONCLUSION: Residents with long DHs had longer SSTs than residents with short DHs. Future DH restrictions may not increase but rather decrease resident SST. Effective measures to encourage self-study are required, as DH restrictions may shorten SST.

The vascular surgery in-training examination predicts performance on qualifying examination.

OBJECTIVE: Vascular surgery trainees participate in the vascular surgery in-training examination (VSITE) during each year of their training. Although the VSITE was developed as a low-stakes, formative examination, performance on that examination might correlate with the pass rates for the Vascular Surgery Board written qualifying examination (VQE) and oral certifying examination (VCE) and might, therefore, guide both trainees and program directors. The present study was designed to examine the ability of the VSITE to predict trainees' performance on the VQE and VCE. METHODS: All first-time candidates of the Vascular Surgery Board VQE and VCE were analyzed from 2016 to 2020, including those from both the integrated and independent training pathways. VSITE scores from the final year of training were associated with the VQE scores and the probability of passing the VQE and VCE both. Linear and logistic regression models were used to determine the ability of VSITE results to predict the VQE scores and the probability of passing each board examination. RESULTS: VSITE scores available for the 559 candidates (69.3% male; 30.7% female) who had completed the VQE and 369 candidates (66.7% male; 33.3% female) who had completed both the VQE and the VCE. The linear regression model results for the final year of training showed that the VSITE scores explained 34% of the variance in the VQE scores (29% for the integrated and 37% for the independent trainees). Logistic regression demonstrated that the final year VSITE scores were a significant predictor of passing the VQE for both integrated and independent trainees (P < .001). A VSITE score of 500 during the final year of training predicted a VQE passing probability of >90% for each group of candidates. The probability of passing the VQE decreased to 73% for candidates from integrated programs, 61% for candidates from independent programs, and 64% for the whole cohort when the score was 400. The VSITE scores were a significant predictor of passing the VCE only for the candidates from independent programs (odds ratio, 1.01; 95% confidence interval, 1.00-1.02; P < .01), for whom a VSITE score of 400 correlated with an 82% probability of passing the VCE. CONCLUSIONS: VSITE performance is predictive of passing the VQE for trainees from both integrated and independent training paradigms. Vascular surgery trainees and training programs should optimize their preparation and educational efforts to maximize performance on the VSITE during their final year of training to improve the likelihood of passing the VQE. Further analysis of the predictive value of VSITE scores during the earlier years of training might allow the board certification examinations to be administered earlier in the final year of training.

Impact of the resident duty hours on in-training examination score: A nationwide study in Japan.

PURPOSE: The relationship between duty hours (DH) and the performance of postgraduate residents is needed to establish appropriate DH limits. This study explores their relationship using the General Medicine In-training Examination (GM-ITE). MATERIALS AND METHODS: In this cross-sectional study, GM-ITE examinees of 2019 had participated. We analyzed data from the examination and questionnaire, including DH per week (eight categories). We examined the association between DH and GM-ITE score, using random-intercept linear models with and without adjustments. RESULTS: Five thousand five hundred and ninety-three participants (50.7% PGY-1, 31.6% female, 10.0% university hospitals) were included. Mean GM-ITE scores were lower among residents in Category 2 (45-50 h; mean score difference, -1.05; p < 0.001) and Category 4 (55-60 h; -0.63; p = 0.008) compared with residents in Category 5 (60-65 h; Reference). PGY-2 residents in Categories 2-4 had lower GM-ITE scores compared to those in Category 5. University residents in Category 1 and Category 5 showed a large mean difference (-3.43; p = 0.01). CONCLUSIONS: DH <60-65 h per week was independently associated with lower resident performance, but more DH did not improve performance. DH of 60-65 h per week may be the optimal balance for a resident's education and well-being.

An updated analysis of shoulder and elbow questions on the Orthopedic In-Training Examination.

BACKGROUND: The Orthopedic In-Training Examination (OITE) is an annual examination for orthopedic surgery residents used to assess orthopedic knowledge across a national standard. Having an updated understanding of currently tested topics and resources is useful to help residents guide their education. PURPOSE: The purpose of this study is to analyze the shoulder and elbow domain of the OITE in an effort to provide current trends and commonly tested topics. METHODS: All OITE questions related to shoulder and elbow topics over the years 2009-2013 and 2017-2020 were analyzed. Subcategories, the number and types of references used, publication lag time, imaging modalities, taxonomic classification, and resident performance were recorded. RESULTS: Shoulder and elbow topics comprised 8.61% of all OITE questions from 2009-2013 and 2017-2020. The most commonly tested shoulder topics were rotator cuff arthropathy and reverse total shoulder arthroplasty (13.6%), followed by hemiarthroplasty and total shoulder arthroplasty (12.9%), rotator cuff-related pathology (12.9%), anterior shoulder instability and/or dislocation (10.2%), and general anatomy (10.2%). The most commonly tested elbow topics were trauma (21%), ulnar collateral ligament injuries (12.12%), general anatomy (10%), and arthroplasty (10%). Decisions regarding management or appropriate next steps (taxonomy T3) comprised 39% of all question types. The incorporation of imaging modalities continues to be an important component of the OITE questions. The Journal of Shoulder and Elbow Surgery (JSES), the Journal of the American Academy of Orthopedic Surgeons (JAAOS), the Journal of Bone and Joint Surgery (JBJS), and the American Journal of Sports Medicine (AJSM) comprised 65% of articles referenced in all questions over our analysis period. CONCLUSION: This study provides an updated analysis of trends within the shoulder and elbow domain of the OITE. Application of these data can aid residents in their preparation for the examination.

Difference in the general medicine in-training examination score between community-based hospitals and university hospitals: a cross-sectional study based on 15,188 Japanese resident physicians.

BACKGROUND: The general medicine in-training examination (GM-ITE) is designed to objectively evaluate the postgraduate clinical competencies (PGY) 1 and 2 residents in Japan. Although the total GM-ITE scores tended to be lower in PGY-1 and PGY-2 residents in university hospitals than those in community-based hospitals, the most divergent areas of essential clinical competencies have not yet been revealed. METHODS: We conducted a nationwide, multicenter, cross-sectional study in Japan, using the GM-ITE to compare university and community-based hospitals in the four areas of basic clinical knowledge". Specifically, "medical interview and professionalism," "symptomatology and clinical reasoning," "physical examination and clinical procedures," and "disease knowledge" were assessed. RESULTS: We found no significant difference in "medical interview and professionalism" scores between the community-based and university hospital residents. However, significant differences were found in the remaining three areas. A 1.28-point difference (95% confidence interval: 0.96-1.59) in "physical examination and clinical procedures" in PGY-1 residents was found; this area alone accounts for approximately half of the difference in total score. CONCLUSIONS: The standardization of junior residency programs and the general clinical education programs in Japan should be promoted and will improve the overall training that our residents receive. This is especially needed in categories where university hospitals have low scores, such as "physical examination and clinical procedures."

Analysis of Hip and Knee Reconstruction Questions on the Orthopedic In-Training Examination.

BACKGROUND: It is vital for orthopedic residents and residency programs to have a current understanding of the materials and resources utilized on the Orthopedic In-Training Examination (OITE) to tailor resident educational curricula accordingly. This study presents an updated analysis of the hip and knee section of the OITE. METHODS: All OITE questions related to hip and knee reconstruction over six examinations between 2014 and 2019 were analyzed for topic, subtopic, taxonomy, imaging modalities, resident performance, and references. RESULTS: There were 166 hip and knee reconstruction questions of 1600 OITE questions (10.4%) over a six-year period. The most commonly tested topics include mechanical properties of total knee and hip implants (10.8%), instability after THA (10.8%), periprosthetic fracture (10.2%), and prosthetic joint infection (10.2%). A total of 362 references were cited from 68 different sources. The most common sources were JOA, JBJS, JAAOS, and CORR, which were collectively responsible for 68% of all citations. There was an average publication lag of 7.1 years, with 75% of all citations falling within 10 years of the question date. Compared with a prior analysis from 2005 and 2009, there were significantly more complex multistep questions regarding treatment and fewer one-step knowledge recall questions (P = .003). Similarly, recent tests had significantly more questions involving interpretation of radiographs (55%, P < .001) and advanced imaging (9.6%, P < .001), compared with a decade ago. CONCLUSIONS: The OITE continues to evolve over time, incorporating recent literature and topics. The current analysis identifies high-yield topics and resources that can guide resident preparation for the OITE hip and knee section.

Impact of general medicine rotation training on the in-training examination scores of 11, 244 Japanese resident physicians: a Nationwide multi-center cross-sectional study.

BACKGROUND: Although general medicine (GM) faculty in Japanese medical schools have an important role in educating medical students, the importance of residents' rotation training in GM in postgraduate education has not been sufficiently recognized in Japan. To evaluate the relationship between the rotation of resident physicians in the GM department and their In-Training Examination score. METHODS: This study is a nationwide multi-center cross-sectional study in Japan. Participants of this study are Japanese junior resident physicians [postgraduate year (PGY)-1 and PGY-2] who took the General Medicine In-Training Examination (GM-ITE) in fiscal years 2016 to 2018 at least once (n = 11,244). The numbers of participating hospitals in the GM-ITE were 381, 459, and 503 in 2016, 2017, and 2018.The GM-ITE score consisted of four categories (medical interview/professionalism, symptomatology/clinical reasoning, physical examination/procedure, and disease knowledge). We evaluated relationship between educational environment (including hospital information) and the GM-ITE score. RESULTS: A total of 4464 (39.7%) residents experienced GM department rotation training. Residents who rotated had higher total scores than residents who did not rotate (38.1 ± 12.1, 36.8 ± 11.7, and 36.5 ± 11.5 for residents who experienced GM rotation training, those who did not experience this training in hospitals with a GM department, and those who did not experience GM rotation training in hospitals without a GM department, p = 0.0038). The association between GM rotation and competency remained after multivariable adjustment in the multilevel model: the score difference between GM rotation training residents and non-GM rotation residents in hospitals without a GM department was estimated as 1.18 (standard error, 0.30, p = 0.0001), which was approximately half of the standard deviation of random effects due to hospital variation (estimated as 2.00). CONCLUSIONS: GM rotation training improved the GM-ITE score of residents and should be considered mandatory for junior residents in Japan.

Evaluation of Progress of an ACGME-International Accredited Residency Program in Qatar.

Background: The American College of Physicians' (ACP) Internal Medicine In-Training Examination (IM-ITE) is designed to evaluate the cognitive knowledge of residents to aid them and program directors in evaluating the training experience. Objective: To determine the impact of the curriculum reform accompanied by the Accreditation Council for Graduate Medical Education (ACGME)-I alignment and accreditation on the internal medicine residency program (IMRP) using residents' performance in the ACP's ITE from 2008 to 2016, and where the IMRP stands in comparison to all ACGME and ACGME-I accredited programs. Methods: This is a descriptive study conducted at a hospital-based IMRP in Doha, Qatar from 2008 to 2016. The study population is 1052 residents at all levels of training in IMRP. The ACP-generated ITE results of all the United States and ACGME-I accredited programs were compared with IM-ITE results in Qatar. These results were expressed in the total program average and the ranking percentile. Results: There is a progressive improvement in resident performance in Qatar as shown by the rise in total average program score from 52% in 2008 to 72% in 2016 and the sharp rise in percentile rank from 3rd percentile in 2008 to 93rd percentile in 2016 with a dramatic increase during the period 2013 to 2014 (from 32nd percentile to 73rd percentile), which represents the period of ACGME-I accreditation. None of the factors (ethnicity, USMLE or year of residency) were statistically significant with a p value >0.05 and standard coefficient ( - 0.017-0.495). There was negligible correlation between the USMLE test scores with the residents' ITE scores with a p value = 0.023 and a Pearson correlation r = 0.097. Conclusion: The initial ACGME-I alignment followed by the accreditation, together with whole curriculum redesign to a structured, competency-based program starting from 2008, has led to an improvement in the ITE scores in the IMRP. This was further evidenced by the lack of change in the residency entry selection criteria.

Predictors for Failing the American Board of Radiology Core Examination.

OBJECTIVE. Since the American Board of Radiology (ABR) instituted its new board certification pathway, our residency program has had more residents fail the core examination than was typical with the prior pathway. We performed a single-center retrospective study to evaluate predictors of ABR core examination failure. MATERIALS AND METHODS. Data regarding U.S. Medical Licensing Examination (USMLE) steps 1 and 2, ACR diagnostic radiology in-training examinations, the number of image interpretations, academic degree (doctor of medicine or doctor of osteopathy), status as an American or foreign medical graduate, and Alpha Omega Alpha national medical honor society status were gathered and evaluated through logistic regression and generalized additive logistic regression. Data were gathered for all residents who took the ABR core examination from 2013 to 2017. RESULTS. Six of 30 residents (20%) failed the ABR core examination on the first attempt. The ACR in-training examination scores for 1st- and 3rd-year residents were significantly related to ABR core examination failure (p = 0.027 and p = 0.035, respectively), with significant nonlinearity (p = 0.037 and p = 0.033, respectively). The suggested baseline percentile score was the 30th percentile for 1st-year residents and the 20th percentile for 3rd-year residents. USMLE step 1 and 2 scores were significantly related to ABR core examination failure (p = 0.041 and p = 0.043, respectively), without significant nonlinearity (p = 0.35 and p = 0.09, respectively). However, residents with scores of less than 220 on USMLE steps 1 and 2 seemed to be at risk. CONCLUSION. Low scores on USMLE steps 1 and 2 and 1st- and 3rd-year ACR in-training examinations were associated with ABR core examination failure. If validated more broadly, these cutoffs may serve as predictors of ABR core examination failure and may facilitate identification and remediation of at-risk residents.

Measuring change in knowledge acquisition of Rwandan residents: using the American Board of Pediatrics International In-Training Examination (I-ITE) as an independent tool to monitor individual and departmental improvements during the Human Resources for Health program: an observational study.

BACKGROUND: Rwanda is the only African country to use the pediatric International In-Training Examination (I-ITE). The objectives of this study were to use the scores from the I-ITE to outline the baseline level of knowledge of Rwandan residents entering the pediatric residency and the trends in knowledge acquisition from 2012 to 2018, during the Human Resources for Health (HRH) Program, an education partnership between the Rwanda Ministry of Health and a consortium of US universities. METHODS: A retrospective descriptive analysis of the I-ITE exam scores, taken by all Rwandan pediatric residents for five of the six academic years of the study period. Individual resident scores were weighted using the non-Rwandan I-ITE sites to minimise confounding from annual variations in exam difficulty. Statistical analysis included descriptives with ANOVA to compare variation in annual mean scores. RESULTS: Eighty-four residents took 213 I-ITE exam sittings over the five exam cycles. The mean weighted I-ITE score of all residents increased from 34% in 2013 to 49% (p < 0.001) in 2018. The 32-point gap between the mean US-ITE and Rwandan I-ITE score in 2012-2013 was reduced to a 16-point gap in 2017-2018. First year resident (PG1) scores, which likely reflect the knowledge level of undergraduate medical students entering the residency program, increased from 34.8 to 44.3% (p = 0.002) between 2013 and 2018. CONCLUSIONS: The I-ITE is an independent, robust tool, measuring both learners and the institutional factors supporting residents. This is the first study to demonstrate that the I-ITE can be used to monitor resident knowledge acquisition in resource-limited settings, where assessment of resident knowledge can be a major challenge facing the academic medicine community. The significant increase in I-ITE scores between 2012 and 18 reflects the substantial curricular reorganisation accomplished through collaboration between Rwandan and US embedded faculty and supports the theory that programs such as HRH are highly effective at improving the quality of residency programs and undergraduate medical education.

Awareness of Diagnostic Error among Japanese Residents: a Nationwide Study.

BACKGROUND: Residents' understanding of diagnostic error may differ between countries. We sought to explore the relationship between diagnostic error knowledge and self-study, clinical knowledge, and experience. METHODS: Our nationwide study involved postgraduate year 1 and 2 (PGY-1 and -2) Japanese residents. The Diagnostic Error Knowledge Assessment Test (D-KAT) and General Medicine In-Training Examination (GM-ITE) were administered at the end of the 2014 academic year. D-KAT scores were compared with the benchmark scores of US residents. Associations between D-KAT score and gender, PGY, emergency department (ED) rotations per month, mean number of inpatients handled at any given time, and mean daily minutes of self-study were also analyzed, both with and without adjusting for GM-ITE scores. Student's t test was used for comparisons with linear mixed models and structural equation models (SEM) to explore associations with D-KAT or GM-ITE scores. RESULTS: The mean D-KAT score among Japanese PGY-2 residents was significantly lower than that of their US PGY-2 counterparts (6.2 vs. 8.3, p < 0.001). GM-ITE scores correlated with ED rotations (≥6 rotations: 2.14; 0.16-4.13; p = 0.03), inpatient caseloads (5-9 patients: 1.79; 0.82-2.76; p < 0.001), and average daily minutes of self-study (≥91 min: 2.05; 0.56-3.53; p = 0.01). SEM revealed that D-KAT scores were directly associated with GM-ITE scores (ß = 0.37, 95% CI: 0.34-0.41) and indirectly associated with ED rotations (ß = 0.06, 95% CI: 0.02-0.10), inpatient caseload (ß = 0.04, 95% CI: 0.003-0.08), and average daily minutes of study (ß = 0.13, 95% CI: 0.09-0.17). CONCLUSIONS: Knowledge regarding diagnostic error among Japanese residents was poor compared with that among US residents. D-KAT scores correlated strongly with GM-ITE scores, and the latter scores were positively associated with a greater number of ED rotations, larger caseload (though only up to 15 patients), and more time spent studying.

Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance.

The Accreditation Council for Graduate Medical Education's Next Accreditation System requires training programs to demonstrate that fellows are achieving competence in medical knowledge (MK), as part of a global assessment of clinical competency. Passing American Board of Internal Medicine (ABIM) certification examinations is recognized as a metric of MK competency. This study examines several in-training MK assessment approaches and their ability to predict performance on the ABIM Hematology or Medical Oncology Certification Examinations. Results of a Hematology In-Service Examination (ISE) and an Oncology In-Training Examination (ITE), program director (PD) ratings, demographic variables, United States Medical Licensing Examination (USMLE), and ABIM Internal Medicine (IM) Certification Examination were compared. Stepwise multiple regression and logistic regression analyses evaluated these assessment approaches as predictors of performance on the Hematology or Medical Oncology Certification Examinations. Hematology ISE scores were the strongest predictor of Hematology Certification Examination scores (ß = 0.41) (passing odds ratio [OR], 1.012; 95 % confidence interval [CI], 1.008-1.015), and the Oncology ITE scores were the strongest predictor of Medical Oncology Certification Examination scores (ß = 0.45) (passing OR, 1.013; 95 % CI, 1.011-1.016). PD rating of MK was the weakest predictor of Medical Oncology Certification Examination scores (ß = 0.07) and was not significantly predictive of Hematology Certification Examination scores. Hematology and Oncology ITEs are better predictors of certification examination performance than PD ratings of MK, reinforcing the effectiveness of ITEs for competency-based assessment of MK.

The relationship between performance on the Infectious Diseases In-Training and Certification Examinations.

BACKGROUND: The Infectious Diseases Society of America In-Training Examination (IDSA ITE) is a feedback tool used to help fellows track their knowledge acquisition during fellowship training. We determined whether the scores on the IDSA ITE and from other major medical knowledge assessments predict performance on the American Board of Internal Medicine (ABIM) Infectious Disease Certification Examination. METHODS: The sample was 1021 second-year fellows who took the IDSA ITE and ABIM Infectious Disease Certification Examination from 2008 to 2012. Multiple regression analysis was used to determine if ABIM Infectious Disease Certification Examination scores were predicted by IDSA ITE scores, prior United States Medical Licensing Examination (USMLE) scores, ABIM Internal Medicine Certification Examination scores, fellowship director ratings of medical knowledge, and demographic variables. Logistic regression was used to evaluate if these same assessments predicted a passing outcome on the certification examination. RESULTS: IDSA ITE scores were the strongest predictor of ABIM Infectious Disease Certification Examination scores (ß = .319), followed by prior ABIM Internal Medicine Certification Examination scores (ß = .258), USMLE Step 1 scores (ß = .202), USMLE Step 3 scores (ß = .130), and fellowship directors' medical knowledge ratings (ß = .063). IDSA ITE scores were also a significant predictor of passing the Infectious Disease Certification Examination (odds ratio, 1.017 [95% confidence interval, 1.013-1.021]). CONCLUSIONS: The significant relationship between the IDSA ITE score and performance on the ABIM Infectious Disease Certification Examination supports the use of the ITE as a valid feedback tool in fellowship training.

Faculty Prediction of In-Training Examination Scores of Emergency Medicine Residents: A Multicenter Study.

BACKGROUND: The Emergency Medicine In-Training Examination (EMITE) is one of the few validated instruments for medical knowledge assessment of emergency medicine (EM) residents. The EMITE is administered only once annually, with results available just 2 months before the end of the academic year. An earlier predictor of EMITE scores would be helpful for educators to institute timely remediation plans. A previous single-site study found that only 69% of faculty predictions of EMITE scores were accurate. OBJECTIVE: The goal of this article was to measure the accuracy with which EM faculty at five residency programs could predict EMITE scores for resident physicians. METHODS: We asked EM faculty at five different residency programs to predict the 2014 EMITE scores for all their respective resident physicians. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual scores. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed faculty background variables for correlation with the two outcomes. RESULTS: One hundred and eleven faculty participated in the study (response rate 68.9%). Mean prediction accuracy for all faculty was 60.0%. Mean prediction precision was 6.3%. Participants were slightly more accurate at predicting scores of noninterns compared to interns. No faculty background variable correlated with the primary or secondary outcomes. Eight participants predicted scores with high accuracy (>80%). CONCLUSIONS: In this multicenter study, EM faculty possessed only moderate accuracy at predicting resident EMITE scores. A very small subset of faculty members is highly accurate.

Faculty prediction of in-training examination scores of emergency medicine residents.

BACKGROUND: The Emergency Medicine In-Training Examination (EMITE) is one of the only valid tools for medical knowledge assessment in current use by emergency medicine (EM) residencies. However, EMITE results return late in the academic year, providing little time to institute potential remediation. OBJECTIVE: The goal of this study was to determine the ability of EM faculty to accurately predict resident EMITE scores prior to results return. METHODS: We asked EM faculty at the study site to predict the 2012 EMITE scores of the 50 EM residents 2 weeks prior to results being available. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual score. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed several faculty background variables, including years of experience, educational leadership status, and clinical hours worked, for correlation with the two outcomes. RESULTS: Thirty-two of the 38 faculty (84.2%, 95% confidence interval [CI] 69.6-92.6) participated in the study, rendering a total of 1600 predictions for 50 residents. Mean resident EMITE score was 81.1% (95% CI 79.5-82.8%). Mean prediction accuracy for all faculty participants was 69% (95% CI 65.9-72.1%). Mean prediction precision was 5.2% (95% CI 4.9-5.5%). Education leadership status was the only background variable correlated with the primary and secondary outcomes (Spearman's ρ = 0.51 and -0.53, respectively). CONCLUSION: Faculty possess only moderate accuracy at predicting resident EMITE scores. We recommend a multicenter study to evaluate the generalizability of the present results.

Musculoskeletal Knowledge on the in-Training Examination Improves in Family Medicine Residents Participating in a Longitudinal Sports Medicine Clinical Track.

Objectives: The study aims to assess the impacts of a sports medicine (SM) track on musculoskeletal (MSK) knowledge of family medicine (FM) residents. In-training examination (ITE) results were used to compare the MSK knowledge of FM residents with and without SM track participation. Methods: A single-center, retrospective study was completed on 85 FM residents from the 2018 to 2024 graduating classes who completed the ITE from 2017 to 2021. Residents were categorized by participation in the SM track, where half a day of FM continuity clinic per week is replaced with an SM clinic, supervised by a fellowship-trained SM physician. ITE scores throughout training were compared between the 2 groups using mixed-effects regression. Results: The ITE MSK scores increased among both SM track participants (+77 points/year, p = .001) and nonparticipants (+39 points/year, p = .001) throughout their training. By postgraduate year 3, SM track participants performed significantly better on the MSK portion of the ITE (+87 points compared to non-participants, p = .045). No significant difference in total ITE scores was seen between groups. Conclusions: Our data demonstrates that participation in an SM track is associated with an increase in MSK knowledge of ITE, suggesting that an SM track may provide FM residents with a better understanding of MSK conditions.

Comparison of Artificial Intelligence to Resident Performance on Upper-Extremity Orthopaedic In-Training Examination Questions.

Purpose: Currently, there is a paucity of prior investigations and studies examining applications for artificial intelligence (AI) in upper-extremity (UE) surgical education. The purpose of this investigation was to assess the performance of a novel AI tool (ChatGPT) on UE questions on the Orthopaedic In-Training Examination (OITE). We aimed to compare the performance of ChatGPT to the examination performance of hand surgery residents. Methods: We selected questions from the 2020-2022 OITEs that focused on both the hand and UE as well as the shoulder and elbow content domains. These questions were divided into two categories: those with text-only prompts (text-only questions) and those that included supplementary images or videos (media questions). Two authors (B.K.F. and G.S.M.) converted the accompanying media into text-based descriptions. Included questions were inputted into ChatGPT (version 3.5) to generate responses. Each OITE question was entered into ChatGPT three times: (1) open-ended response, which requested a free-text response; (2) multiple-choice responses without asking for justification; and (3) multiple-choice response with justification. We referred to the OITE scoring guide for each year in order to compare the percentage of correct AI responses to correct resident responses. Results: A total of 102 UE OITE questions were included; 59 were text-only questions, and 43 were media-based. ChatGPT correctly answered 46 (45%) of 102 questions using the Multiple Choice No Justification prompt requirement (42% for text-based and 44% for media questions). Compared to ChatGPT, postgraduate year 1 orthopaedic residents achieved an average score of 51% correct. Postgraduate year 5 residents answered 76% of the same questions correctly. Conclusions: ChatGPT answered fewer UE OITE questions correctly compared to hand surgery residents of all training levels. Clinical relevance: Further development of novel AI tools may be necessary if this technology is going to have a role in UE education.

Structured Textbook Review and Individualized Learning Plans Successfully Remediate Underperforming Residents and Improve General Surgery Program Performance on the ABSITE.

Background: We endeavored to create an evidence-based curriculum to improve general surgery resident fund of knowledge. Global and resident-specific interventions were employed to this end. These interventions were monitored via multiple choice question results on a weekly basis and American Board of Surgery In-Training Examination (ABSITE) performance. Methods: This study was performed in a prospective manner over a 2-year period. A structured textbook review with testing was implemented for all residents. A focused textbook question-writing assignment and a Surgical Council on Resident Education (SCORE)-based individualized learning plan (ILP) were implemented for residents scoring below the 35th percentile on the ABSITE. Results: Curriculum implementation resulted in a statistically significant reduction in the number of residents scoring below the 35th percentile, from 50% to 30.8% (P = .023). One hundred percent of residents initially scoring below the 35th percentile were successfully remediated over the study period. Average overall program ABSITE percentile scores increased from 38.5% to 51.4% over a 2-year period. Conclusion: Structured textbook review and testing combined with a question-writing assignment and a SCORE-focused ILP successfully remediated residents scoring below the 35th percentile and improved general surgery residency ABSITE performance.

A Comparative American Board of Surgery In-Training Examination (ABSITE) Performance Analysis Between International vs. Domestic Graduates and Doctor of Medicine (MD) vs. Doctor of Osteopathic Medicine (DO) Medical Degrees.

Introduction The American Board of Surgery (ABS) plays a pivotal role in certifying surgeons in the United States, with the American Board of Surgery In-Training Examination (ABSITE) serving as a critical assessment tool for general surgery residents aspiring for certification. The aim of this study is to compare the performance of international medical graduates (IMGs) to their domestic counterparts and assess the impact of different medical degrees on ABSITE scores. Notably, ABSITE scores often dictate the trajectory of a surgical career, including opportunities for fellowship placements in specialized fields such as plastic surgery. Methods This study focused on general surgery residents enrolled at Marshall University from 2014 to 2022. Data encompassing ABSITE scores, TrueLearn quiz percentages, and TrueLearn mock exam results were collected for analysis. Descriptive statistics summarized sample characteristics, and linear mixed models were employed to address correlations. Statistical analyses were conducted using the Statistical Analysis System (SAS) (version 9.4; SAS Institute Inc., Cary, NC, USA), with significance defined by a two-sided test with p < 0.05. Results Among the 48 participants, comprising 24 non-international medical graduates (nIMGs) and 24 IMGs, IMGs demonstrated superior performance across various metrics. They exhibited higher quiz percentages (67% vs. 61%; p = 0.0029), mock Exam 1 scores (64% vs. 58%; p = 0.0021), mock Exam 2 scores (66% vs. 58%; p = 0.0015), ABSITE scores (560 vs. 505; p = 0.010), and ABSITE percentages (74% vs. 68%; p = 0.0077) compared to nIMGs. Analysis between Doctor of Osteopathic Medicine (DO) and Doctor of Medicine (MD) participants revealed no statistically significant differences in performance metrics, highlighting the comparability of these medical degrees in the context of ABSITE scores and related assessments. Discussion/conclusion This study underscores the superior performance of IMGs over nIMGs in ABSITE examinations, shedding light on the critical role of ABSITE scores in shaping surgical careers. Higher scores correlate with enhanced opportunities for coveted fellowship placements, particularly in specialized fields like plastic surgery. Understanding these dynamics is crucial for resident training and navigating the competitive landscape of surgical sub-specialization. Future research endeavors can delve deeper into the factors influencing ABSITE performance, thereby facilitating the development of targeted interventions to support residents in achieving their career aspirations.