Preliminary Development of a Program ABSITE Dashboard (PAD) to Guide Curriculum Innovation.

PURPOSE: Medical Knowledge for general surgery residents' is assessed by the American Board of Surgery In- Training Examination (ABSITE). ASBITE score reports contain many metrics residency directors can utilize to assess resident progress and perform program evaluation. The purpose of this study was to develop a framework to evaluate program effectiveness in teaching specific subtest and subtopic areas of the ABSITE, using ABSITE score reports as an indicator. The aim is to demonstrate the identification of topic areas of weakness in program-wide performance on the ABSITE to guide proposed modification of the general surgery residency program curriculum, and to initiate development of a data visualizing dashboard to communicate these metrics. METHODS: A single institution retrospective study was performed utilizing ABSITE score reports from general surgery residents at a large academic training program from 2017 to 2020. ABSITE performance metrics from 320 unique records were entered into a database; statistical analysis for linear trends and variance were conducted for standard scores, subtest standard scores, and incorrect subtest topics. Deviation from national average scores were calculated by subtracting the national average score from each subtest score for each trainee. Data were displayed as medians or proportions and are displayed to optimize visualization as a proof-of-concept for the development of a program dashboard. RESULTS: Trends and variance in general surgery program and cohort performance on various elements of the ABSITE were visualized using figures and tables that represent a prototype for a program dashboard. Figure A1 demonstrates one example, in which a heatmap displays the median deviation from national average scores for each subtest by program year. Boxplots show the distribution of the deviation from national average, range for national average scores, and the recorded scores for each subtest by program year. Trends in median deviation from the national average scores are displayed for each program year paneled by subtest or for each exam year paneled by cohort. Median change in overall test scores from one program year to another in a cohort is visualized as a table. Bar graphs show the most often missed topics across all program years and heatmaps were generated showing the proportion of times each topic was missed for each subtest and exam year. CONCLUSIONS: We demonstrate use of ABSITE reports to identify specific thematic areas of opportunities for curriculum modification and innovation as an element of program evaluation. In this study we demonstrate, through data analysis and visualization, feasibility for the creation of a Program ABSITE Dashboard (PAD) that enhances the use of ABSITE reports for formative program evaluation and can guide modifications to surgery program curriculum and educational practices.

Implementation of a Holistic Review Process of US Allopathic Medical Students Eliminates Non-Comparable Metrics and Bias in General Surgery Residency Interview Invitations.

OBJECTIVE: Use of traditional scoring metrics for residency recruitment creates racial and gender bias. In addition, widespread use of pass/fail grading has led to noncomparable data. To adjust to these challenges, we developed a holistic review (HR) rubric for scoring residency applicants for interview selection. DESIGN: Single-center observational study comparing the proportion of underrepresented in medicine (URM) students and their United States Medical Licensing Exam (USMLE) scores who were invited for interview before (2015-2020) and after (2022) implementation of a holistic review process. SETTING: General surgery residency program at a tertiary academic center. PARTICIPANTS: US allopathic medical students applying for general surgery residency. RESULTS: After initial screening, a total of 1514 allopathic applicants were narrowed down to 586 (38.7%) for HR. A total of 52% were female and 17% identified as URM. Based on HR score, 20% (118/586) of applicants were invited for an interview. The median HR score was 11 (range 4-19). There was a fourfold higher coefficient of variation of HR scores (22.3; 95% CI 21.0-23.7) compared to USMLE scores (5.1; 95% Cl 4.8-5.3), resulting in greater spread and distinction among applicants. There were no significant differences in HR scores between genders (p = 0.60) or URM vs non-URM (p = 0.08). There were no significant differences in Step 1 (p = 0.60) and 2CK (p = 0.30) scores between those who were invited to interview or not. On multivariable analysis, USMLE scores (OR 1.01; 95% CI 0.98-1.03), URM status (OR 1.71 95% CI 0.98-2.92), and gender (OR 0.94, 95% CI 0.60-1.45) did not predict interview selection (all p > 0.05). There was a meaningful increase in the percentage of URM interviewed after HR implementation (12.9% vs 23.1%, p = 0.016). CONCLUSION: The holistic review process is feasible and eliminates the use of noncomparable metrics for surgical applicant interview invitations and increases the percentage of URM applicants invited to interview.

This Surgical Life - An Exploration of Surgical Department Podcasting.

OBJECTIVE: Surgical departments are increasingly utilizing media to disseminate knowledge, discuss ideas, and mentor future surgeons. Podcasts are a form of media where digitally recorded content can be downloaded or streamed. This study aims (1) to describe the audience reached by a single surgical department podcast and (2) to evaluate what sources of information surgery residency applicants use to formulate a rank list. DESIGN: In Fall 2017, the Rocky Mountain Surgery Podcast (RMSP) was created, produced, and edited by 2 general surgery (GS) residents at a large academic training program. Each episode discussed a topic within GS training and/or educational experiences specific to the program. Interviewing GS applicants for the 2019 match were asked to complete an anonymous voluntary survey on their familiarity and opinion of RMSP and the role of podcasts in the application process. RESULTS: Twenty-two episodes were completed over a 16-month period (October 4, 2017 - February 11, 2019). A total of 7002 individual listens occurred in 644 cities across 46 states. Ninety-eight interviewing applicants responded to the survey (99% response rate), and one-fourth had previously listened to the RMSP. Only half felt that the traditional interview experience provided enough information about a GS program, and a significant majority (97%) stated they would listen to one or more podcast episodes to gain information regarding a GS residency program. CONCLUSIONS: Applicants to GS residency commonly feel inadequate information is gained during the interview process. Podcasts are a tool familiar to applicants that allow for exploration of topics which cannot be adequately addressed in a typical interview day, thus expanding an applicant's knowledge of a GS training program.

Examination of Nondesignated Preliminary Surgery Residents Recruited Since the Inception of Supplementary Offer and Acceptance Program: Lessons Learned From a Large Academic Program.

PURPOSE: The majority of surgery programs roster non-designated preliminary (NDP) residents. We and others have reported on the success of NDP resident mentoring with regard to categorical position placement. Lacking is a focus on the candidates themselves and differences based on initial career of choice. METHODS: NDP residents' files since the institution of SOAP were reviewed for demographics, initial career choice, financial burden, region of medical school of origin, application and interview history, and pre-screen interview candidate score (high score of 22 - data includes USMLE scores, major clerkship grades, and AOA) used for categorical recruitment. RESULTS: From 2012-16 79 NDP residents have been recruited at UCDenver (82% via SOAP). Median age was 28 years, majority were single (73%), and male (82%). Thirty percent belonged to an under-represented minority group. Mean debt was $156,000 but 20% owe over $250,000. 90% attended US medical schools with 65% from the NRMP "South" region. 86% were recruited as fourth year students. NDPs were categorized as failing to match in general surgery (38%), surgical subspecialties (47%), or other (15%). NDPs applied to median of 68 programs (range 7-200) and granted a median of 8 interviews (range 0-24). NDPs had a mean pre-screening interview score of 13 out of 22 and only 9% would have met the standard threshold to obtain a categorical surgery interview. There were no differences in pre-screening scores in the three groups. 95% NDPs (excluding present year) successfully completed their R1 year (three resigned - one obtained a general surgery spot mid-year and two after matching in non-general surgery fields). 68% NDPs placed in categorical positions after the R1 year. The placement was better for the surgical subspecialty group compared to the other two. CONCLUSIONS: The bias is that due to a more competitive applicant pool recruiting NDPs from surgical subspecialties would be optimal. However, those unmatched surgical subspecialty candidates are no better academically than the unmatched general surgery group, often have career interests that do not always align with scheduled rotations, and may not feel compelled to complete the year if they match. They place minimally better when accounting for those unmatched general surgery NDP R1s continuing as NDP R2s. Ultimately the success in a non-designated preliminary R1 surgery program is alignment of clinical educational opportunities with the needs of the trainee.

Selection, mentorship, and subsequent placement of preliminary residents without a designated categorical position in an academic general surgery residency program.

BACKGROUND: Academic general surgery residency programs face a reduction in clinical hours for residents at a time of expanding educational material/clinical programs. To meet these challenges, clinical staffing includes additional faculty, physician extenders, or additional residents. Categorical resident expansion is difficult but there is an ample pool of preliminary first year resident (R1) candidates every match week, and this pool is projected to increase markedly as all medical schools have increased enrollment without any real change to the demographics of available categorical residency positions. Our experience with preliminary R1s without a designated categorical position over the last 8 academic years forms the basis of this report. METHODS: The University of Colorado-Denver general surgery residency program recruits 8-12 preliminary R1s annually. Most of these are recruited on scramble day, as we found that having individual interview dates for these positions was labor-intensive and did not yield better candidates. Preliminary R1s were mentored in a systematic fashion by the program during the year to focus on career planning. Files for preliminary R1s were reviewed for demographics, medical school of origin, original prematch career goal, successful completion of the intern year, and subsequent categorical residency placement/type. RESULTS: We recruited 64 preliminary R1s without a designated categorical position over the past 8 years (16 females and 48 males) with an average age of 30 years old (range 25-45 years). These preliminary R1s attended medical schools throughout the United States (West 24, South 18, Central 11, Northeast 4, or International 7). The average United States Medical Licensing Examination (USMLE) Step 1 test score was 213 (range 185-252). The majority (95%) completed their preliminary year of training or left for a categorical position during the year. Most preliminary R1s (79%) were successfully placed into categorical training programs at the end of the year, with 23% ultimately receiving categorical training at our institution. Four did not complete their preliminary training year due to medical leave or unsatisfactory performance. The categorical positions preliminary R1s obtained include general surgery (n = 7, 15%), surgical subspecialties (n = 31, 65%), and nonsurgical subspecialties (n = 10, 20%). Ultimately, 14 (29%) of preliminary R1s who matched after 1 year did so in their fields of original interest. CONCLUSIONS: A symbiotic relationship can exist between a general surgery residency program and preliminary R1s. An approach that focuses on scramble day recruitment, frequent mentorship meetings, and an active role in secondary placement results in successful placement into categorical positions for the vast majority. In that regard, we would argue that we have provided equal career mentorship to these graduated students in comparison with their medical schools of origin.