Performance of ChatGPT on American Board of Surgery In-Training Examination Preparation Questions.

INTRODUCTION: Chat Generative Pretrained Transformer (ChatGPT) is a large language model capable of generating human-like text. This study sought to evaluate ChatGPT's performance on Surgical Council on Resident Education (SCORE) self-assessment questions. METHODS: General surgery multiple choice questions were randomly selected from the SCORE question bank. ChatGPT (GPT-3.5, April-May 2023) evaluated questions and responses were recorded. RESULTS: ChatGPT correctly answered 123 of 200 questions (62%). ChatGPT scored lowest on biliary (2/8 questions correct, 25%), surgical critical care (3/10, 30%), general abdomen (1/3, 33%), and pancreas (1/3, 33%) topics. ChatGPT scored higher on biostatistics (4/4 correct, 100%), fluid/electrolytes/acid-base (4/4, 100%), and small intestine (8/9, 89%) questions. ChatGPT answered questions with thorough and structured support for its answers. It scored 56% on ethics questions and provided coherent explanations regarding end-of-life discussions, communication with coworkers and patients, and informed consent. For many questions answered incorrectly, ChatGPT provided cogent, yet factually incorrect descriptions, including anatomy and steps of operations. In two instances, it gave a correct explanation but chose the wrong answer. It did not answer two questions, stating it needed additional information to determine the next best step in treatment. CONCLUSIONS: ChatGPT answered 62% of SCORE questions correctly. It performed better at questions requiring standard recall but struggled with higher-level questions that required complex clinical decision making, despite providing detailed responses behind its rationale. Due to its mediocre performance on this question set and sometimes confidently-worded, yet factually inaccurate responses, caution should be used when interpreting ChatGPT's answers to general surgery questions.

International Medical Graduates are Comparable to American Medical Graduates as General Surgery Interns.

BACKGROUND: International medical graduates (IMGs) are often relegated to preliminary positions in general surgery (GS) owing to uncertainties about the candidate's performance in the American healthcare setting. We aimed to determine the comparative performance of IMGs and American medical graduates (AMGs) at baseline and assess these trends over the course of their GS internship. METHODS: Evaluations of all IMG preliminary and AMG categorical interns from 2013 to 2017 at our GS residency program were obtained from three faculty members to score overall performance, technical skills, interpersonal communication, and medical knowledge on a 10-point Likert scale. Scores on the American Board of Surgery In-Training Exam, an in-house preparation test, United States Medical Licensing Exam, and performance during the biannual multistation objective assessments were compared between the two resident groups. RESULTS: Seventy-two interns (28 [39%] AMG categorical and 44 [61%] IMG preliminary) met inclusion criteria. The AMG group had significantly higher median Step 1 and Step 2 scores compared with our IMG group (243 versus 231, P = 0.002, and 250 versus 246, P = 0.03, respectively).Although in-house preparation test scores were higher among IMGs (median [interquartile range] of 36 [33-40] among AMGs and 38 [34-45] among IMGs; P = 0.002), there were no statistically significant differences between the American Board of Surgery In-Training Exam scores of the two groups. The median scores for the four faculty evaluation components were similar between the AMG (7, 8, 7, 7) and IMG resident groups (7, 7, 7, 7; P = nonsignificant). IMGs scored significantly higher in both biannual multistation objective assessments than AMGs (median [interquartile range] July: 59 [47-91] versus 55 [37-62], P = 0.005; January: 103 [86-116] versus 91 [87-104], P = 0.03). CONCLUSIONS: It is reassuring to confirm that no matter where they are from, great candidates can perform well as surgical interns in a GS training program.

USMLE Scores and Clinical Rotation Role in Predicting ABSITE Performance Among Surgery Interns.

BACKGROUND: The American Board of Surgery In-training Examination (ABSITE) is an important marker of medical knowledge. It is unclear what factors predict or improve these scores. We evaluated demographics, United States Medical Licensing Examination (USMLE) step 1 and 2 scores, and surgical rotations during the intern year to determine if there were any correlations with the ABSITE performance. METHODS: This was a multicenter retrospective review during a 6-y period, investigating the correlation and association of demographics, USMLE scores, and types of rotations on the ABSITE percentile score of interns. Demographics included age, gender, race/ethnicity, U.S. versus international/foreign medical graduate for stratified analyses. Descriptive analysis was performed with ANOVA, correlation was evaluated with 95% confidence interval, and significance was defined as P < 0.05. RESULTS: Complete records obtained on 89 interns from six different general surgery programs over 6 y revealed that there was a significant correlation between USMLE 1 and 2 with the ABSITE. USMLE 2 correlation was the strongest (r = 0.44, 95% confidence interval = [0.25-0.60], P < 0.05). There was a significant difference in ABSITE performance (mean score difference of 17.3 percentile, P = 0.01) of interns who had an intensive care unit rotation before examination. Other surgical rotations were not associated with an ABSITE difference. Demographic factors such as age, gender, race/ethnicity, or medical graduate background status were not associated with ABSITE scores. CONCLUSIONS: USMLE step 2 scores had a higher correlation with intern ABSITE performance. An intensive care unit rotation before taking the ABSITE was associated with a significant difference in their percentile scores. Demographic factors were not correlated with ABSITE performance.

Addition of Advanced Practice Providers May Improve ABSITE Scores Through Enhanced Resident Education.

BACKGROUND: Resident work hour restrictions and required protected didactic time limit their ability to perform clinical duties and participate in structured education. Advanced practice providers (APPs) have previoulsy been shown to positively impact patients' outcomes and overall hospital costs. We describe a model in which nurse practitioners (NPs) improve resident education and American Board of Surgery In Training Examination (ABSITE) scores by providing support to our trauma and acute care surgery (ACS) service thereby protecting resident didactic time. MATERIALS AND METHODS: A new educational model aimed to improve ABSITE scores was created, increasing protected resident didactic time. The addition of three full-time NPs to the ACS service allowed implementation of this redesigned academic curriculum to be put into effect without neglecting patient or service-related responsibilities that were previously fulfilled by resident staff. Resident ABSITE results including standard score, percent correct, and percentile were compared before and after the educational changes were instituted. RESULTS: Eleven residents' scores were included. For each ABSITE score, we used a mixed model with time and postgraduate year (PGY) level as fixed effects and subject ID as a random effect. The interaction term between PGY level and time was not significant and removed from the model. A significant main effect of PGY level and of time was then observed. A statistically significant improvement in ABSITE scores after intervention was observed across all the PGY levels. Standard score increased 77.3 points (P-value = 0.001), percent correct increased 5.9% (P-value = 0.002), and percentile increased 23.8 (P-value = 0.02). Following the educational reform, no residents scored below the 35th percentile. CONCLUSIONS: Utilization of NPs on our ACS service provided adequate service coverage, allowing the implementation of an educational reform increasing protected resident education time and improved ABSITE scores.

Effects of Implementing a Breast Surgery Rotation on ABSITE Scores and Surgical Case Volume.

BACKGROUND: Little is known about general surgery trainees' education regarding management of breast problems. We sought to measure the impact of a dedicated breast surgery rotation on American Board of Surgery In-Service Examination (ABSITE) scores and operative volumes. METHODS: A breast surgery rotation was implemented at our program in July 2016. We obtained the January 2017 ABSITE scores for postgraduate year (PGY) 1-3 residents, and obtained the case volumes for PGY 1-3 residents during the years 2015-2016 and 2016-2017. RESULTS: We compared the performance on total questions and skin, soft tissue, and breast questions between the residents who had the breast rotation before the ABSITE to those that had it after. There was no difference in the average overall percentage (70.2% versus 71.7%, P = 0.55) or in the average skin, soft tissue, and breast percentage (70% versus 71.4%, P = 0.72). A postgraduate year-to-year comparison showed an increase in average total major cases among the PGY-1 residents (93.8 versus 166.8, P = 0.02), and an increase in average breast cases among the PGY-1 (17.8 versus 27 cases, P < 0.01) and PGY-2 (27.3 versus 47.7 cases, P = 0.02) years. There was an increase in the proportion of complex breast cases performed by PGY-3 residents (23.2% versus 33.1%, P = 0.01). CONCLUSIONS: A dedicated breast surgery rotation did not detract from the nonbreast general surgery educational experience of junior residents (as measured by ABSITE scores), and it increased the case volume of breast as well as total major cases among junior residents. A breast surgery rotation is valuable for strengthening surgical case volumes.

Relationships between study habits, burnout, and general surgery resident performance on the American Board of Surgery In-Training Examination.

BACKGROUND: The American Board of Surgery In-Training Examination (ABSITE) is used by programs to evaluate the knowledge and readiness of trainees to sit for the general surgery qualifying examination. It is often used as a tool for resident promotion and may be used by fellowship programs to evaluate candidates. Burnout has been associated with job performance and satisfaction; however, its presence and effects on surgical trainees' performance are not well studied. We sought to understand factors including burnout and study habits that may contribute to performance on the ABSITE examination. METHODS: Anonymous electronic surveys were distributed to all residents at 10 surgical residency programs (n = 326). Questions included demographics as well as study habits, career interests, residency characteristics, and burnout scores using the Oldenburg Burnout Inventory, which assesses burnout because of both exhaustion and disengagement. These surveys were then linked to the individual's 2016 ABSITE and United States Medical Licensing Examination (USMLE) step 1 and 2 scores provided by the programs to determine factors associated with successful ABSITE performance. RESULTS: In total, 48% (n = 157) of the residents completed the survey. Of those completing the survey, 48 (31%) scored in the highest ABSITE quartile (≥75th percentile) and 109 (69%) scored less than the 75th percentile. In univariate analyses, those in the highest ABSITE quartile had significantly higher USMLE step 1 and step 2 scores (P < 0.001), significantly lower burnout scores (disengagement, P < 0.01; exhaustion, P < 0.04), and held opinions that the ABSITE was important for improving their surgical knowledge (P < 0.01). They also read more frequently to prepare for the ABSITE (P < 0.001), had more disciplined study habits (P < 0.001), were more likely to study at the hospital or other public settings (e.g., library, coffee shop compared with at home; P < 0.04), and used active rather than passive study strategies (P < 0.04). Gender, marital status, having children, and debt burden had no correlation with examination success. Backward stepwise multiple regression analysis identified the following independent predictors of ABSITE scores: study location (P < 0.0001), frequency of reading (P = 0.0001), Oldenburg Burnout Inventory exhaustion (P = 0.02), and USMLE step 1 and 2 scores (P = 0.007 and 0.0001, respectively). CONCLUSIONS: Residents who perform higher on the ABSITE have a regular study schedule throughout the year, report less burnout because of exhaustion, study away from home, and have shown success in prior standardized tests. Further study is needed to determine the effects of burnout on clinical duties, career advancement, and satisfaction.

Evaluating the effectiveness of a mock oral educational program.

BACKGROUND: To obtain board certification, the American Board of Surgery requires graduates of general surgery training programs to pass both the written qualifying examination (QE) and the oral certifying examination (CE). In 2015, the pass rates for the QE and CE were 80% and 77%, respectively. In the 2011-2012 academic year, the University of Wisconsin instituted a mandatory, faculty-led, monthly CE preparation educational program (CE prep) as a supplement to their existing annual mock oral examination. We hypothesized that the implementation of these sessions would improve the first-time pass rate for residents taking the ABS CE at our institution. Secondary outcomes studied were QE pass rate, correlation with American Board of Surgery In-Training Examination (ABSITE) and mock oral examination scores, cost, and type of study materials used, perception of examination difficulty, and applicant preparedness. METHODS: A sixteen question survey was sent to 57 of 59 residents who attended the University of Wisconsin between the years of 2007 and 2015. Email addresses for two former residents could not be located. De-identified data for the ABSITE and first-time pass rates for the QE and CE examination were retrospectively collected and analyzed along with survey results. Statistical analysis was performed using SPSS version 22 (IBM Corp., Armonk, NY). P values < 0.05 were considered significant. RESULTS: Survey response rate was 77.2%. Of the residents who have attempted the CE, first-time pass rate was 76.0% (19 of 25) before the implementation of the formal CE Prep and 100% (22 of 22) after (P = 0.025). Absolute ABSITE score, and mock oral annual examination grades were significantly improved after the CE Prep was initiated (P values < 0.001 and 0.003, respectively), however, ABSITE percentile was not significantly different (P = 0.415). ABSITE raw score and percentile, as well as mock oral annual examination scores were significantly associated with passing the QE (0.032, 0.027, and 0.020, respectively), whereas mock oral annual examination scores alone were associated with passing the CE (P = 0.001). Survey results showed that residents perceived the CE to be easier than the annual mock oral after the institution of the CE prep course (P = 0.036), however, there was no difference in their perception of preparedness. Overall, applicants felt extremely prepared for the CE (4.70 ± 0.5, Likert scale 1-5). CONCLUSIONS: Formal educational programs instituted during residency can improve resident performance on the ABS certifying examination. The institution of a formal, faculty-led monthly CE preparation educational program at the University of Wisconsin has significantly improved the first-time pass rate for the ABS CE. Mock oral annual examination scores were also significantly improved. Furthermore, ABSITE scores correlate with QE pass rates, and mock oral annual examination scores correlate with pass rates for both QE and CE.

Scheduling surgical success: Early minimally invasive surgery rotations as a predictor of ABSITE performance.

INTRODUCTION: The American Board of Surgery In-Training Examination (ABSITE®) serves as a benchmark for assessing resident knowledge and comprehension in surgery training programs. While previous studies have examined factors such as USMLE Step 1 and Step 2 performance in ABSITE® preparation, the impact of rotational schedule on ABSITE® performance remains underexplored. METHODS: We retrospectively investigated the independent influence of rotational schedule on ABSITE® performance among interns at a single academic institution over an 11 year period. RESULTS: Early exposure to minimally invasive surgery rotations and specialty rotations such as surgical oncology and vascular surgery during the winter or just before ABSITE® was associated with higher ABSITE® percentiles when controlled for USMLE Step 2 performance. CONCLUSION: This study underscores the significance of rotational scheduling in optimizing ABSITE® performance and suggests potential strategies for enhancing resident preparation and success on this examination.

The Plan-Do-Study-Act (PDSA): An Iterative Approach to Optimize Residents Performance in the American Board of Surgery in-Training Exam (ABSITE).

INTRODUCTION: American Board of Surgery (ABS) In-Training Examination (ITE), or ABSITE, preparation requires an effective study approach. In 2014, the ABS announced the alignment of ABSITE to the SCORE® Curriculum. We hypothesized that implementing a Plan-Do-Study-Act (PDSA) approach would help surgery residents improve their performance on the ABSITE. METHOD: Over 20 years, in a single institution, residents' ABSITE performance was evaluated over 3 timeframes: Time A (2004-2013), no specific curriculum; Time B (2014-2019), an annual comprehensive ABSITE-simulated SCORE®-based multiple-choice exam (MCQ) was administered; and Time C (2020-2023), like Time B with the addition of the PDSA approach for those with less than 60% correct on the ABSITE-simulated SCORE®-based exam. At the beginning of the academic year, in July, all residents are encouraged to (1) initiate a study plan for the upcoming ABSITE using SCORE® guided by the published ABSITE outlines content topics (Plan), (2) take an ABSITE-simulated SCORE®-based exam in October (Do), (3) assess the results/scores (Study), and (4) identify appropriate next steps (Act). Correlational analysis was performed to evaluate the association between ABSITE scores and ABSITE-simulated SCORE®-based exam scores in Time B and Time C. The primary outcome was the change in the proportions of ABSITE scores <30th percentile. RESULTS: A total of 294 ABSITE scores of 94 residents (34 females and 60 males) were analyzed. We found stronger correlation between the correct percentage on ABSITE and ABSITE-simulated SCORE®-based exam scores in Time C (r = 0.73, p < 0.0001) compared to Time B (0.62, p < 0.0001). The percentage of residents with ABSITE scores lower than 30th percentile dropped significantly from 14.0% to 3.7% (p = 0.016). CONCLUSION: Implementing the Plan-Do-Study-Act (PDSA) approach using the SCORE® curriculum significantly enhances residents' performance on the ABSITE exam. Surgery residents are encouraged to use this approach and to utilize the SCORE-contents outlined by the ABS in their study plan.

National ABSITE Preparation Practices: A Survey of General Surgery Residency Program Directors.

OBJECTIVE: The American Board of Surgery In-Training Examination (ABSITE) was designed as a low-stakes, medical knowledge examination for US general surgery residency programs. However, in practice, this exam has been utilized for higher stakes purposes, such as resident promotion or remediation, and fellowship selection. Several studies have demonstrated the efficacy of ABSITE preparation resources, but best practices for ABSITE preparation and national preparatory habits are currently unknown. The aim of this work was to determine current residency programs' strategies for ABSITE preparation. DESIGN: We distributed an electronic survey to program directors or program coordinators of US general surgery programs asking them to anonymously report program ABSITE educational practices and ABSITE scores. We analyzed the proportion of responses using descriptive statistics and compared the effect of various strategies using the Mann-Whitney testing for nonparametric data. An average ABSITE percentile score was calculated for each residency based on program self-reported scores. SETTING: Association of Program Directors (APDS) Listserv PARTICIPANTS: General surgery residency programs participating at the time of distribution (278). RESULTS: Response rate was 24% (66/278); 41 programs (62.1%) identified as university-affiliated, and 25 (37.9%) were community-based. Median intern class size was 8 (range: 3-14), including preliminary interns. Average ABSITE percentile score was 52.8% (range 36.9%-67.6%). There were no significant differences in ABSITE scores based on affiliation or program size. Educational resources utilized for ABSITE preparation included SCORE (89.3%), Q-banks (50%), and surgical textbooks (25.8%). The majority (56.1%) of programs reported using a year-long curriculum for ABSITE preparation, and 66.6% used a time-limited curriculum completed in the months immediately prior to ABSITE. Most programs reported that ABSITE scores were a low priority (63.6%) or not a priority (13.6%). The existence of a year-long curriculum for ABSITE was positively correlated with score as compared to programs without a year-long curricula (53.9% vs 48.5%, p <0.01). Programs using a time-limited curriculum demonstrated lower scores as compared to programs without time-limited curricula (51.3% v 56.1%, p < 0.01). CONCLUSION: General surgery programs use a variety of strategies to prepare residents for the ABSITE. Despite reporting that they utilize ABSITE scores for a variety of high stakes purposes including evaluation for promotion and as a predictor of the preparedness for the ABS QE, many programs reported that they consider ABSITE scores as a low priority. A year-long focused curriculum was the only strategy correlated with increased scores, which may reflect the value of encouraging consistent studying and spaced repetition. Additional work is needed to guide programs in optimal utilization of ABSITE scores for remediation and resident evaluation, as well as understanding how ABSITE preparatory strategies correlate with clinical performance.

A Team-Based American Board of Surgery In-Training Examination (ABSITE) Competition Improves Exam Performance.

OBJECTIVE: The American Board of Surgery In-Training Examination (ABSITE) is an important factor in general surgery resident career development. The use of practice question banks for ABSITE preparation improves exam scores and completing more practice questions results in higher scores. The objective of this study is to describe the design of a novel, team-based ABSITE competition implemented into our residency program and analyze its effect on exam performance. DESIGN: For the 2023 to 2024 academic year (AY), residents were randomly sorted into teams. The number of TrueLearn (TL) practice questions completed by each team during the 6-month period preceding the 2024 ABSITE was monitored at regular intervals. Paired sample t-test was used to compare total questions completed and ABSITE percent scores from the prior AY. Simple linear regression was performed to determine if total completed questions predicted ABSITE percent scores, and if the change in total completed questions predicted a change in ABSITE percent scores for unique residents. SETTING: University-affiliated general surgery residency program. PARTICIPANTS: 34 clinical and research general surgery residents. RESULTS: Mean total TL questions completed per resident increased by 175.2, with a difference trending toward statistical significance (1037.9 ± 583.6 vs 1213.1 ± 596.0, p = 0.08). Mean ABSITE percent scores significantly increased by 4.6% (68.2 ± 8.7 vs 72.8 ± 5.7, p < 0.001). Total completed questions positively correlated with ABSITE percent scores (R2 = 0.21, F (1,66) = 17.04, p < 0.001). Among individual residents, completing more questions than the prior AY predicted improvement in ABSITE percent scores (R2 =.18, F (1,32) = 7.02, p = 0.01). Our model predicts that by completing 300 additional practice questions (e.g. 10 more questions/day for 1 month) a resident's ABSITE percent score will increase by 4.8 percentage points. CONCLUSION: Implementation of a team-based ABSITE competition is a straightforward and effective intervention that increases general surgery resident question bank utilization and significantly improves ABSITE percent scores.

Navigating the American Board of Surgery In-Training Examination (ABSITE) Success: Insights From Pre-assessment Practices in Preparing Surgical Residents for Competitive Sub-specialties.

Introduction The American Board of Surgery In-Training Examination (ABSITE) is a critical tool in assessing surgical residents' readiness for board certification and clinical practice. While various factors influencing ABSITE performance have been examined, the impact of innovative educational resources, such as TrueLearn, remains underexplored. TrueLearn's adaptive learning algorithms and comprehensive question banks offer a promising adjunct to traditional study methods. This study investigates the relationship between TrueLearn utilization and ABSITE performance among general surgery residents. Methods This retrospective study, ethically approved by the Marshall University Institutional Review Board (IRB No. 2097669-1), analyzed the performance of general surgery residents at Marshall University from 2014 to 2022. Data were collected on ABSITE scores. Additionally, quiz percentages (Quiz %) and scores from two mock exams (Exam 1 and Exam 2), all provided by the TrueLearn platform, were included in the analysis. Descriptive statistics summarized the sample characteristics. Linear mixed models were employed to examine the associations between TrueLearn engagement and ABSITE performance, accounting for the correlated nature of the data and addressing any missing data at random. Statistical analyses were conducted using the Statistical Analysis System (SAS, version 9.4; SAS Institute Inc., Cary, NC), with significance defined as a p-value < 0.05. Results The study cohort included 58 residents from 2016 to 2022. Linear mixed model analysis revealed significant positive correlations between TrueLearn Quiz %, Exam 1 scores, and Exam 2 scores with ABSITE performance. A 1% increase in Quiz % was associated with a 0.77-point rise in ABSITE scores (95% CI: 0.65, 0.89; p < 0.0001). For Exam 1, each point increase corresponded to a 6.36-point increase in ABSITE scores (95% CI: 5.01, 7.7; p < 0.0001), while Exam 2 scores showed a 3.8-point increase per point (95% CI: 2.74, 4.86; p < 0.0001). Discussion and conclusion Our findings underscore the significant impact of TrueLearn engagement on ABSITE performance, with higher quiz percentages and mock exam scores predictive of better ABSITE outcomes. This suggests that regular use of TrueLearn's educational resources enhances residents' knowledge and exam readiness. These results advocate for the integration of innovative educational tools such as TrueLearn into surgical training programs to optimize study strategies and improve exam performance. However, the study's retrospective design and single-institution focus limit the generalizability of the findings. Future research should explore these relationships in diverse settings and specialties and consider additional factors influencing ABSITE performance. This study highlights the positive association between TrueLearn utilization and ABSITE performance among general surgery residents, emphasizing the importance of innovative educational resources in surgical training. By enhancing engagement with platforms such as TrueLearn, surgical programs can improve residents' readiness for high-stakes examinations, ultimately contributing to the development of proficient surgical practitioners.

Near-Zero Temperature Coefficient of Resonance Frequency in Rare-Earth Titanates via the Phase Transition Strategy.

This study presents an approach to achieve a near-zero temperature coefficient of resonance frequency (τf) in rare-earth titanate microwave dielectric ceramics (MWDCs) by inducing a phase transition. By Zr4+ substitution at the B site, a series of Sm2Ti1-xZrxO5 (0.02 ≤ x ≤ 0.55) ceramics are synthesized using the solid-state method to intentionally alter the radius ratio of the A/B sites, realizing in a controlled phase transition from orthorhombic (Pnma) to biphasic coexistence and ultimately to cubic (Fd3̅m) structure. The phase composition is rigorously identified through X-ray diffraction (XRD) Rietveld refinement, high-resolution transmission electron microscopy (HRTEM), selected-area electron diffraction (SAED), and Raman spectroscopy. A comprehensive analysis is conducted to elucidate the relationships between factors such as ionic polarizability, packing fraction, bond valence, complex chemical bonding, and far-infrared reflectivity spectra with microwave dielectric properties. The results demonstrate that these ceramics exhibit a broad range of permittivity (14.30-23.18), high-quality factors (14,828-22,300 GHz), opposite temperature coefficient of resonance frequency (-16.0 to + 22.4 ppm/°C), and nice thermal conductivity (1.81-2.76 W·m-1·K-1), particularly at x = 0.30 with a near-zero τf value of +1.6 ppm/°C. The findings not only provide insights into designing MWDCs with a near-zero τf but also offer a promising route for developing advanced microwave materials with improved performance and reliability.

Preliminary Development of a Program ABSITE Dashboard (PAD) to Guide Curriculum Innovation.

PURPOSE: Medical Knowledge for general surgery residents' is assessed by the American Board of Surgery In- Training Examination (ABSITE). ASBITE score reports contain many metrics residency directors can utilize to assess resident progress and perform program evaluation. The purpose of this study was to develop a framework to evaluate program effectiveness in teaching specific subtest and subtopic areas of the ABSITE, using ABSITE score reports as an indicator. The aim is to demonstrate the identification of topic areas of weakness in program-wide performance on the ABSITE to guide proposed modification of the general surgery residency program curriculum, and to initiate development of a data visualizing dashboard to communicate these metrics. METHODS: A single institution retrospective study was performed utilizing ABSITE score reports from general surgery residents at a large academic training program from 2017 to 2020. ABSITE performance metrics from 320 unique records were entered into a database; statistical analysis for linear trends and variance were conducted for standard scores, subtest standard scores, and incorrect subtest topics. Deviation from national average scores were calculated by subtracting the national average score from each subtest score for each trainee. Data were displayed as medians or proportions and are displayed to optimize visualization as a proof-of-concept for the development of a program dashboard. RESULTS: Trends and variance in general surgery program and cohort performance on various elements of the ABSITE were visualized using figures and tables that represent a prototype for a program dashboard. Figure A1 demonstrates one example, in which a heatmap displays the median deviation from national average scores for each subtest by program year. Boxplots show the distribution of the deviation from national average, range for national average scores, and the recorded scores for each subtest by program year. Trends in median deviation from the national average scores are displayed for each program year paneled by subtest or for each exam year paneled by cohort. Median change in overall test scores from one program year to another in a cohort is visualized as a table. Bar graphs show the most often missed topics across all program years and heatmaps were generated showing the proportion of times each topic was missed for each subtest and exam year. CONCLUSIONS: We demonstrate use of ABSITE reports to identify specific thematic areas of opportunities for curriculum modification and innovation as an element of program evaluation. In this study we demonstrate, through data analysis and visualization, feasibility for the creation of a Program ABSITE Dashboard (PAD) that enhances the use of ABSITE reports for formative program evaluation and can guide modifications to surgery program curriculum and educational practices.

Structured Textbook Review and Individualized Learning Plans Successfully Remediate Underperforming Residents and Improve General Surgery Program Performance on the ABSITE.

Background: We endeavored to create an evidence-based curriculum to improve general surgery resident fund of knowledge. Global and resident-specific interventions were employed to this end. These interventions were monitored via multiple choice question results on a weekly basis and American Board of Surgery In-Training Examination (ABSITE) performance. Methods: This study was performed in a prospective manner over a 2-year period. A structured textbook review with testing was implemented for all residents. A focused textbook question-writing assignment and a Surgical Council on Resident Education (SCORE)-based individualized learning plan (ILP) were implemented for residents scoring below the 35th percentile on the ABSITE. Results: Curriculum implementation resulted in a statistically significant reduction in the number of residents scoring below the 35th percentile, from 50% to 30.8% (P = .023). One hundred percent of residents initially scoring below the 35th percentile were successfully remediated over the study period. Average overall program ABSITE percentile scores increased from 38.5% to 51.4% over a 2-year period. Conclusion: Structured textbook review and testing combined with a question-writing assignment and a SCORE-focused ILP successfully remediated residents scoring below the 35th percentile and improved general surgery residency ABSITE performance.

Surgical resident learning styles: faculty and resident accuracy at identification of preferences and impact on ABSITE scores.

INTRODUCTION: As a consequence of surgical resident duty hour restrictions, there is a need for faculty to utilize novel teaching methods to convey information in a more efficient manner. The current paradigm of surgical training, which has not changed significantly since the time of Halsted, assumes that all residents assimilate information in a similar fashion. However, recent data has shown that learners have preferences for the ways in which they receive and process information. The VARK model categorizes learners as visual (V), aural (A), read/write (R), and kinesthetic (K). The VARK learning style preferences of surgical residents have not been previously evaluated. In this study, the preferred learning styles of general surgery residents were determined, along with faculty and resident perception of resident learning styles. In addition, we hypothesized that American Board of Surgery In-Training Exam (ABSITE) scores are associated with preference for a read/write (R) learning style. METHODS: The Fleming VARK learning styles inventory was administered to all general surgery residents at a university hospital-based program. Responses on the inventory were scored to determine the preferred learning style for each resident. Faculty members were surveyed to determine their accuracy in identifying the preferred learning style of each resident. All residents were also surveyed to determine their accuracy in identifying their peers' VARK preferences. Resident ABSITE scores were examined for association with preferred learning styles. RESULTS: Twenty-nine residents completed the inventory. Most (18 of 29, 62%) had a multimodal preference, although more than a third (11 of 29, 38%) demonstrated a single-modality preference. Seventy-six percent of all residents (22 of 29) had some degree of kinesthetic (K) learning, while under 50% (14 of 29) were aural (A) learners. Although not significant, dominant (R) learners had the highest mean ABSITE scores. Faculty identified residents' learning styles accurately 41% of the time; more experienced faculty were better than less experienced ones (R(2) = 0.703, P = 0.018). Residents had similar accuracy to faculty in identifying their peers' learning styles. Chief residents were more accurate than junior residents (44% versus 28%, P = 0.009). CONCLUSIONS: Most general surgery residents have a multimodal learning preference. Faculty members are relatively inaccurate at identifying residents' preferred learning styles; however, there is a strong correlation between years of faculty experience and accuracy. Chief residents are more accurate than junior residents at learning style identification. Higher mean ABSITE scores may be a reflection of a dominant read/write learning style.

Can Customized YouTube Videos Improve ABSITE Performance?

INTRODUCTION: The COVID-19 pandemic decreased in-person didactic sessions in our general surgery residency. We piloted a program to develop online tutorials posted to a YouTube channel designed to prepare our residents for the ABSITE examination. METHODS: Study participants were General Surgery residents in our five-year ACGME accredited general surgery residency program who took the ABSITE in-service exam between 2019 and 2021. We compared raw scores and overall programmatic percentile scores for three academic years (2018-2019, 2019-2020, and 2020-2021). RESULTS: From academic year 2018-2019 to 2019-2020, before introducing the YouTube teaching platform, average raw scores and percentile scores among our general surgery residents remained unchanged (raw scores 64 - 65% (p = ns), percentile scores 39 to 37% (p = ns)). However, raw ABSITE scores increased significantly after introducing the YouTube teaching platform. A one-way ANOVA showed a statistically significant difference in raw ABSITE scores for AY 2020-2021 compared to the two prior years (F(2, 30) = [1.193], P < 0.01). Mean percentile ranking scores (program-wide) increased 15% from AY 2018-2019 to AY 2020-2021 (Figure1). CONCLUSION: Teaching via an online platform such as YouTube allows residents to review material at their own pace and schedule. It is other our hypothesis is that the online YouTube material presented in the Ventura Surgery School channel contributed, in part, to this relative improvement compared to resident peers in other institutions. Teaching via an online platform could be a valuable tool to educate surgical learners in our ever-changing teaching environment.

United States medical licensing examination Step 1 scores are a quantifiably better predictor of American Board of Surgery In-Training Examination success compared to Step 2 scores.

BACKGROUND: United States Medical Licensing Examination Step 1 (STEP 1) and Step 2 (STEP 2) scores are used in the selection of surgery residents. The American Board of Surgery In-Training Examination (ABSITE) is employed to assess resident knowledge. We sought to determine whether both STEP 1 and/or STEP 2 were predictive of ABSITE performance. METHODS: We performed a 10-year retrospective analysis of all PGY levels comparing STEP 1 and 2 scores with raw ABSITE scores and percentile rank. Additionally, we analyzed STEP 1 and 2 scores for the likelihood of scoring above the 20th percentile on the ABSITE. RESULTS: When STEP 1 and STEP 2 scores were evaluated separately, both were independent predictors of ABSITE performance. However, when STEP 1 and STEP 2 scores were evaluated together, only STEP 1 scores were predictive of ABSITE performance. Finally, only STEP 1 scores were predictive of scoring above the 20th percentile. CONCLUSION: STEP 1 scores are a better predictor of ABSITE performance than STEP 2.

Just Do It: Participation in Structured Online Curricula Reliably Improves Low ABSITE Scores.

OBJECTIVE: We hypothesized residents enrolled in an Accelerated Clinical Education in Surgery (ACES) program would improve their scores to above the 30th percentile. We analyzed which components of ACES correlated with improvement. DESIGN AND SETTING: A retrospective review of three academic cycles (2018-2021) at an academic general surgery residency. PARTICIPANTS: Residents scoring ≤30th percentile on the ABSITE were enrolled in ACES. Baseline demographics including STEP scores were collected. ACES included: (1) SCORE and DeckerMed assignments (2) Weekly faculty review sessions and (3) Monthly meeting with assigned mentor. Data were analyzed by Student's t-test, one-way ANOVA and Fisher's exact test. RESULTS: Twenty-six surgical residents enrolled in ACES. Compared to residents not in ACES, there was no significant difference females (15 vs. 15; p = 0.19) and STEP 2 scores (241 vs. 246; p = 0.06). Residents in ACES had significantly lower STEP 1 (225 vs. 237; p < 0.001) and STEP 3 (212 vs. 223; p < 0.001) scores. Demographics of ACES residents who subsequently scored >30th percentile were similar to those who didn't, except for STEP 3 scores (216 vs. 204; p = 0.008). For residents in ACES, the completion of assignments between July and January was significantly higher for those who subsequently achieved an ABSITE score >30th percentile: TWIS, 77% vs. 53% (p = 0.022), Decker WC, 80% vs. 49% (p = 0.009) and Decker MR, 53% vs. 29% (p = 0.016). Completion of an online practice exam prior to ABSITE also correlated with score >30th percentile (57% vs. 13%, p = 0.007). There was also no correlation between the number of faculty review sessions and ABSITE (11.5 vs.11.9, p = 0.931). CONCLUSIONS: Participation in a structured online program of reading and quizzes was durably effective in improving ABSITE scores >30th percentile. Completion of online assignments, rather than scores on practice tests or review sessions, appeared to be the most important factor for success.

How to Prepare for the American Board of Surgery In-Training Examination (ABSITE): A Systematic Review.

INTRODUCTION: Performance on ABSITE is an important factor when monitoring resident progress. It predicts future performance and has lasting effects. Understanding the highest-yield preparation strategies can help residents in their study efforts and optimize performance. METHODS: A literature search was conducted searching PubMed, EMBASE and JAMA Network in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Searches were performed for the terms "ABSITE" and "American Board of Surgery In-Training Examination". Only studies discussing individual study habits from May 2011 to May 2021 were included. RESULTS: 19 studies were included in qualitative synthesis. Year-round clinical study failed to show significant correlation to ABSITE performance although year-round ABSITE review was more consistently correlated. During a dedicated study period, increased time and increased total practice questions completed are associated with improved performance. The correlation of individual resources such as ABSITE review books, textbooks, audio podcasts and ABSITE preparatory courses to improved ABSITE performance was not proven. CONCLUSIONS: Residents should optimize study strategies based on methods that have consistently shown to improve performance. Recommendations for best preparation strategies are provided.