Incorporating a situational judgement test in residency selections: clinical, educational and organizational outcomes.

BACKGROUND: Computer-based assessment for sampling personal characteristics (Casper), an online situational judgement test, is a broad measure of personal and professional qualities. We examined the impact of Casper in the residency selection process on professionalism concerns, learning interventions and resource utilization at an institution. METHODS: In 2022, admissions data and information in the files of residents in difficulty (over three years pre- and post- Casper implementation) was used to determine the number of residents in difficulty, CanMEDS roles requiring a learning intervention, types of learning interventions (informal learning plans vs. formal remediation or probation), and impact on the utilization of institutional resource (costs and time). Professionalism concerns were mapped to the 4I domains of a professionalism framework, and their severity was considered in mild, moderate, and major categories. Descriptive statistics and between group comparisons were used for quantitative data. RESULTS: In the pre- and post- Casper cohorts the number of residents in difficulty (16 vs. 15) and the number of learning interventions (18 vs. 16) were similar. Professionalism concerns as an outcome measure decreased by 35% from 12/16 to 6/15 (p < 0.05), were reduced in all 4I domains (involvement, integrity, interaction, introspection) and in their severity. Formal learning interventions (15 vs. 5) and informal learning plans (3 vs. 11) were significantly different in the pre- and post-Casper cohorts respectively (p < 0.05). This reduction in formal learning interventions was associated with a 96% reduction in costs f(rom hundreds to tens of thousands of dollars and a reduction in time for learning interventions (from years to months). CONCLUSIONS: Justifiable from multiple stakeholder perspectives, use of an SJT (Casper) improves a clinical performance measure (professionalism concerns) and permits the institution to redirect its limited resources (cost savings and time) to enhance institutional endeavors and improve learner well-being and quality of programs.

Does Masked Interviewing Encourage Holistic Review in Residency Selection? A Mixed-Methods Study.

Problem: Medical educators increasingly champion holistic review. However, in U.S. residency selection, holistic review has been difficult to implement, hindered by a reliance on standardized academic criteria such as board scores. Masking faculty interviewers to applicants' academic files is a potential means of promoting holistic residency selection by increasing the interview's ability to make a discrete contribution to evaluation. However, little research has directly analyzed the effects of masking on how residency selection committees evaluate applicants. This mixed-methods study examined how masking interviews altered residency selection in an anesthesiology program at a large U.S. academic medical center. Intervention: During the 2019-2020 residency selection season in the University of Pennsylvania's Department of Anesthesiology & Critical Care, we masked interviewers to the major academic components of candidates' application files (board scores, transcripts, letters) on approximately half of interview days. The intent of the masking intervention was to mitigate the tendency of interviewers to form predispositions about candidates based on standardized academic criteria and thereby allow the interview to make a more independent contribution to candidate evaluation. Context: Our examination of the masking intervention used a concurrent, partially mixed, equal-status mixed-methods design guided by a pragmatist approach. We audio-recorded selection committee meetings and qualitatively analyzed them to explore how masking affected the process of candidate evaluation. We also collected independent candidate ratings from interviewers and consensus committee ratings and statistically compared ratings of candidates interviewed on masked days to ratings from conventional days. Impact: In conventional committee meetings, interviewers focused on how to reconcile academic metrics and interviews, and their evaluations of interviews were framed according to predispositions about candidates formed through perusal of application files. In masked meetings, members instead spent considerable effort evaluating candidates' "fit" and whether they came off as tactful. Masked interviewers gave halting opinions of candidates and sometimes pushed for committee leaders to reveal academic information, leading to masking breaches. Higher USMLE Step 1 score and higher medical school ranking were statistically associated with more favorable consensus rating. We found no significant differences in rating outcomes between masked and conventional interview days. Lessons learned: Elimination of academic metrics during the residency interview phase does not straightforwardly promote holistic review. While critical reflection among medical educators about the fairness and utility of such metrics has been productive, research and intervention should focus on the more proximate topic of how programs apply academic and other criteria to evaluate applicants.

Resident Selection Criteria in Veterinary Medicine.

With the continued rise of interest and need for veterinary specialists, information regarding optimal selection criteria for successful residency candidates has been lacking in veterinary medicine. A 28-question online survey was developed to determine prioritized resident selection criteria, the importance of formal interviews, and residency supervisor satisfaction with the current selection process. This survey was sent to all programs listed by the Veterinary Internship and Residency Matching Program (VIRMP) for the 2019-2020 program year. Overall, the most important aspects of the residency application process were (1) letters of recommendation, (2) performance during the interview, (3) personal contact/recommendation from a colleague, (4) personal statement, and (5) demonstrated interest in the residency specialty. While measures of academic performance including GPA and veterinary class rank may play a role in sorting of candidates in more competitive specialties, this does not necessarily exclude them from the ranking process. This information should be helpful to candidates and program directors alike in understanding the success of the current residency candidate selection process.

Evaluating the Whole Applicant: Use of Situational Judgment Testing and Personality Testing to Address Disparities in Resident Selection.

PURPOSE OF REVIEW: Urology program directors are faced with increasing numbers of applications annually, making holistic review of each candidate progressively more difficult. Efforts to streamline evaluation using traditional cognitive metrics have fallen short as these do not predict overall resident performance. Situational judgment tests (SJTs) and personality assessment tools (PATs) have been used in business and industry for decades to evaluate candidates and measure non-cognitive attributes that better predict subsequent performance. The purpose of this review is to describe what these assessments are and the current literature on the use of these metrics in medical education. RECENT FINDINGS: SJTs relative to PATs have more original research. Data suggests that SJTs decrease bias, increase diversity, and may be predictive of performance in residency. PATs are also emerging with data to support use with ability to assess fit to program and certain traits identified more consistently among high-performing residents and correlation to performance on ACGME milestones. PATs may be more coachable than SJTs. SJTs and PATs are emerging as techniques to supplement the current resident application review process. Early evidence supports their use in undergraduate medical education as does some early preliminary results in graduate medical education.

Best Practices for Reducing Bias in the Interview Process.

PURPOSE OF REVIEW: Objective measures of residency applicants do not correlate to success within residency. While industry and business utilize standardized interviews with blinding and structured questions, residency programs have yet to uniformly incorporate these techniques. This review focuses on an in-depth evaluation of these practices and how they impact interview formatting and resident selection. RECENT FINDINGS: Structured interviews use standardized questions that are behaviorally or situationally anchored. This requires careful creation of a scoring rubric and interviewer training, ultimately leading to improved interrater agreements and biases as compared to traditional interviews. Blinded interviews eliminate even further biases, such as halo, horn, and affinity bias. This has also been seen in using multiple interviewers, such as in the multiple mini-interview format, which also contributes to increased diversity in programs. These structured formats can be adopted to the virtual interviews as well. There is growing literature that using structured interviews reduces bias, increases diversity, and recruits successful residents. Further research to measure the extent of incorporating this method into residency interviews will be needed in the future.

Gender bias in the evaluation of interns in different medical specialties: An archival study.

INTRODUCTION: The field of medicine is characterized by within-field gender segregation: Gender ratios vary systematically by subdisciplines. This segregation might be, in part, due to gender bias in the assessment of women and men medical doctors. METHODS: We examined whether the assessments, i.e. overall score, department scores and skills scores, interns receive by their superiors during their internship year, vary as a function of their gender and the representation of women in the field. We analyzed an archival data set from a large hospital in Israel which included 3326 assessments that were given to all interns who completed their internship year between 2015 and 2019. RESULTS: Women received lower department scores and skills scores in fields with a low (versus high) representation of women. Men received higher scores in fields with a high (versus low) representation of men, yet there was no difference in their skills scores. CONCLUSIONS: Women are evaluated more negatively in fields with a low representation of women doctors. Similarly, men are evaluated more negatively in fields with a low representation of men, yet this cannot be explained by their skills. This pattern of results might point to a gender bias in assessments. A better understanding of these differences is important as assessments affect interns' career choices and options.

General Surgery Residency Application Evaluation in a USMLE Step 1 Pass/Fail World: A Retrospective Comparison.

Introduction The United States Medical Licensure Exam (USMLE) Step 1 has been used as both a licensing exam and a way for residency programs to evaluate applicants. It has had significant impact upon the match process over time. With the 2020 decision to make the exam pass/fail due to its unclear validity as an evaluation for future physician performance, programs will go through the match without the Step 1 score. We set out to better understand the effects of the exam score on our selection process, with the hypothesis that without the step 1 score, the ranking of our applicants would be significantly altered. Methods We performed a retrospective analysis of applications to a single General Surgery residency program with 4 categorial residents per year at a physician led, academic, tertiary care medical center from 2017-2020. Important applicant factors including USMLE Step 1 and 2, AOA status, science grades, clerkship scores, audition rotations, volunteer activities, research activities, letters of recommendation, and personal statements were given points and evaluated through our equation, the sum of which was used to create a rank list and offer interviews. The standard deviation of scores was calculated with and without Step 1, and the distribution of scores compared. The range and average of applicants' change in point scores were examined. Results The applications of 653 students were reviewed. After removal of USMLE step 1 points, 40% of all applicants decreased in rank, 35% remained the same, and 24% increased. Specifically, 18.8% of the top third dropped to the middle third, and 11.7% of the bottom third jumped to the middle third, while the middle third changed little (0.2% dropped and 0.9% jumped out of middle third). The points given for USMLE step 1 created a wider distribution of scores with a negative skewness, suggesting there were more applicants below the mean than above. After removing those points, applicants' scores had a narrower distribution and skewness closer to 0, showing fewer upper outliers and more applicants near the mean. Conclusions The USMLE Step 1 score significantly affected the evaluation of applicants, and the removal of it from the recruitment criteria tightened applicant rankings. The elimination of the USMLE Step 1 score in the assessment of applicants will allow for its replacement with variables that better reflect the core values of residency programs.

Delivering CaRMS Transparency: Applicant Review and Selection Process of a Single-Center Diagnostic Radiology Residency Training Program.

PURPOSE: To report the current application review and selection process in our Canadian diagnostic radiology program at the University of Ottawa for both Canadian and international medical graduates. APPLICATION REVIEW AND SELECTION PROCESS: Submitted applications fulfilling institutional requirements were selected for a detailed file review after preliminary screening. A diverse group of file reviewers and interviewers was selected. Interviews were offered based on file review score sheet outcomes. Each interviewer generated a postinterview rank list. Applicants were reviewed and discussed from highest to lowest rank based on a preliminary compiled rank list generated from the average of the postinterview rank lists. Group discussion and a consensus model were used to create a final applicant rank list. CONCLUSIONS: We outlined our systematic, consistent selection process which aligns with current best practices. This description may inform other programs wishing to adopt or optimize strategies to improve candidate assessments and selection processes.

Small Animal Resident Selection Processes at a University Teaching Hospital: An Analysis and Recommendations for Improvement.

Concerns regarding resident performance within a small animal department prompted a review of selection practices, with the intent of improving validity and efficiency. Information was gathered from semi-structured interviews and descriptions of current processes; emphasis was placed on determining how the Veterinary Internship and Residency Matching Program application was used. Processes were found to lack standardization and rely heavily on arbitrary judgments. In addition, faculty members expressed concerns regarding their reliability and the time spent generating candidate rankings. Suggestions for improvement were based on current practices in personnel psychology and human resource management. The need for standardization within and across specialty groups was emphasized, along with a multiple-hurdle approach in which a substantial deficit or red flag in any component results in candidate disqualification. Comprehensive recommendations were made for the selection process as follows: Each application undergoes initial administrative screening for employment eligibility and academic cut-offs; eligible applications are scored by 2-3 faculty members using defined ratings on four equally weighted pre-interview criteria (i.e., veterinary education, post-graduation experiences, personal statement, and standardized letters of reference); phone calls to colleagues with knowledge of the applicant follow specific guidelines and a rating scale; veterinary-situational structured interview questions with appropriate rating scales are used to assess candidates' standing on specified competencies identified as important for success; and the interview score is weighted equally and added to the four pre-interview components to determine the final rank. It is hoped this new approach will take less time and facilitate the selection of successful residents.

Application Factors May Not Be Predictors of Success Among General Surgery Residents as Measured by ACGME Milestones.

BACKGROUND: Can factors within the Electronic Residency Application Service application be used to predict the success of general surgery residents as measured by the Accreditation Council for Graduate Medical Education (ACGME) general surgery milestones? METHODS: This is a retrospective study of 21 residents who completed training at a single general surgery residency program. Electronic Residency Application Service applications were reviewed for objective data, such as age, US Medical Licensing Examination scores, and authorship of academic publications as well as for letters of recommendation, which were scored using a standardized grading system. These factors were correlated to resident success as measured by ACGME general surgery milestone outcomes using univariate and multivariate analyses. This study was conducted at a single academic tertiary care and level 1 trauma facility. Residents who completed general surgery residency training from the years of 2012-2018 were included in the study. RESULTS: There were few correlations between application factors and resident success determined by the ACGME milestones. CONCLUSIONS: Application factors alone do not account for ongoing growth and development throughout residency. Unlike the results presented in the literature for other surgical subspecialties, predicting general surgery resident success based on application factors is not straightforward.

Improving Our Ability to Predict Resident Applicant Performance: Validity Evidence for a Situational Judgment Test.

Construct: We investigated whether a situational judgment test (SJT) designed to measure professionalism in physicians predicts residents' performance on (a) Accreditation Council for Graduate Medical Education (ACGME) competencies and (b) a multisource professionalism assessment (MPA). Background: There is a consensus regarding the importance of assessing professionalism and interpersonal and communication skills in medical students, residents, and practicing physicians. Nonetheless, these noncognitive competencies are not well measured during medical education selection processes. One promising method for measuring these noncognitive competencies is the SJT. In a typical SJT, respondents are presented with written or video-based scenarios and asked to make choices from a set of alternative courses of action. Interpersonally oriented SJTs are commonly used for selection to medical schools in the United Kingdom and Belgium and for postgraduate selection of trainees to medical practice in Belgium, Singapore, Canada, and Australia. However, despite international evidence suggesting that SJTs are useful predictors of in-training performance, end-of-training performance, supervisory ratings of performance, and clinical skills licensing objective structured clinical examinations, the use of interpersonally oriented SJTs in residency settings in the United States has been infrequently investigated. The purpose of this study was to investigate whether residents' performance on an SJT designed to measure professionalism-related competencies-conscientiousness, integrity, accountability, aspiring to excellence, teamwork, stress tolerance, and patient-centered care-predicts both their current and future performance as residents on two important but conceptually distinct criteria: ACGME competencies and the MPA. Approach: We developed an SJT to measure seven dimensions of professionalism. During calendar year 2017, 21 residency programs from 2 institutions administered the SJT. We conducted analyses to determine the validity of SJT and USMLE scores in predicting milestone performance in ACGME core competency domains and the MPA in June 2017 and 3 months later in September 2017 for the MPA and 1 year later, in June 2018, for ACGME domains. Results: At both periods, the SJT score predicted overall ACGME milestone performance (r = .13 and .17, respectively; p < .05) and MPA performance (r = .19 and .21, respectively; p < .05). In addition, the SJT predicted ACGME patient care, systems-based practice, practice-based learning and improvement, interpersonal and communication skills, and professionalism competencies (r = .16, .15, .15, .17, and .16, respectively; p < .05) 1 year later. The SJT score contributed incremental validity over USMLE scores in predicting overall ACGME milestone performance (ΔR = .07) 1 year later and MPA performance (ΔR = .05) 3 months later. Conclusions: SJTs show promise as a method for assessing noncognitive attributes in residency program applicants. The SJT's incremental validity to the USMLE series in this study underscores the importance of moving beyond these standardized tests to a more holistic review of candidates that includes both cognitive and noncognitive measures.

Prosthodontic Resident Selection by Postdoctoral Dental Matching Program: A National Survey by Participating Program Director.

PURPOSE: The purpose of this study was to obtain information about the resident selection and Match process that occurred for the Commission on Dental Accreditation (CODA)-accredited Advanced Education in Prosthodontics program in North America from the program directors' perspective, after the first year of implementation. MATERIALS AND METHODS: The list of Advanced Education in Prosthodontics program in CODA-accredited North American institutions was obtained from the ACP central office, and the directors of each program were identified. The surveys were distributed to 48 program directors online and results were compiled. RESULTS: Out of 48 programs, 43 directors responded to and completed the survey (90.0%), where 35 programs participated in the Match process, 5 did not, and 3 were omitted due to incomplete surveys. While the majority of programs did not see any changes in their applicant pool or their quality, 10 programs (25.0%) observed a number of increases in the application and 13 programs (32.5%) observed an increase in the quality of the applications. Among the 35 programs that participated in the Match process, the main reasons for their participation were "fairness" (48.2%), "did not want to keep moving up interview dates" (23.2%), and "have applicants come to interview as planned" (21.4%). Thirty-three programs reported that 75% to 100% of the invited applicants came to the interview (76.7%). The majority of programs that participated in Match expressed satisfaction with the process (29/33, 80.6%). Sixteen programs (45.7%) reported that they matched with 100% of their first selections. CONCLUSION: Overall response of reintroduction of the Match process for advanced prosthodontic resident selection was very positive for majority of the program directors.

Beyond the USMLE: The STAR Algorithm for Initial Residency Applicant Screening and Interview Selection.

BACKGROUND: Efficient, nonbiased methods for screening residency candidates are lacking. The purpose of this study is to highlight the design, implementation, and impact of the Selection Tool for Applicants to Residency (STAR), an objective approach to selecting candidates to interview for residency selection purposes. MATERIALS AND METHODS: Single-institution retrospective cohort study of medical student applicants and current residents of a single otolaryngology residency program from 2008 to 2015 was performed. STAR was introduced to the selection process in 2013 with no USMLE cutoff score needed to receive an interview. Single-institution review of otolaryngology residency program applications from 2008 to 2015 was performed. STAR was introduced in 2013. In addition to applicants, we analyzed characteristics of residents who successfully matched into our program. Prealgorithm residents (n = 16) and postalgorithm residents (n = 12) were compared to assess the impact of this approach on characteristics of successfully matched residents at the program. RESULTS: Three hundred sixty-five applications were analyzed. Applicant pools before and after algorithm displayed similar characteristics. Interestingly, while there was no USMLE "cutoff," scores significantly increased after algorithm. There was no significant difference in the proportion of women (P = 0.588) or underrepresented minorities (P = 0.587) invited to interview pre- and post-STAR. The algorithm significantly decreased the time needed to review applications and interview residency candidates without impacting the overall composition of the interviewee pool. CONCLUSIONS: Traditional application review methods can be time consuming and may not ensure effective screening of applicants. STAR, or similar objective tools, may be a viable alternative to evaluate applicants, reduce evaluative time, and potentially decrease the impact of unconscious bias.

How Competitive Is the Canadian Diagnostic Radiology Residency Match? Application and Matching Trends From 1991-2014.

PURPOSE: The study sought to evaluate application trends in Canadian diagnostic radiology residency programs and to assess the relative competitiveness of radiology as a specialty. METHODS: The Canadian Residency Matching Service Reports from 1991-2014 for Canadian graduates were used to extract the total residency positions and radiology residency positions, number of applicants to all specialties and to radiology, number of first-choice radiology applicants, number of unmatched radiology positions, and number of positions and applicants to each specialty. Ratios were calculated: radiology positions to applicants and first-choice applicants, first-choice radiology applicants to applicants for all specialties, and training positions to applicants in each specialty. Data trends and correlation coefficients were analysed. RESULTS: The number of radiology residency positions offered increased, with strong positive correlation (r = 0.91, P < .001), while the number of applicants increased with only a moderate positive correlation (r = 0.49, P = .03). Radiology was the most competitive in 1997, with a ratio of 0.32 positions/applicant. There was an increase of positions/applicant over time (decreasing competitiveness; r = 0.76, P < .001) but no change in positions/first-choice applicant (r = 0.11, P = .65). The highest percentage of applicants who ranked radiology as their first choice was in 2003 at 6.5% with a decrease in this percentage over time (r = -0.36, P = .13). Radiology is moderately competitive for positions/overall applicants and very competitive for positions/first-choice applicants compared to other specialties. CONCLUSIONS: The number of radiology residency positions has increased while the number of applicants has not grown commensurately. The match was most competitive in 1997, and decreased in subsequent years. Possible reasons include job market, reimbursement, and work environment.

PROGNOSTIC VALUE OF THE MULTIPLE MINI-INTERVIEW FOR EMERGENCY MEDICINE RESIDENCY PERFORMANCE.

BACKGROUND: The Multiple Mini-Interview (MMI) uses short, structured contacts, and is known to predict medical school success better than traditional interviews and application materials. Its utility in Emergency Medicine residency selection is untested. OBJECTIVES: We investigate whether it provides additional information regarding future first-year resident performance that can be useful in resident selection. METHODS: From three Emergency Medicine residency programs, 71 interns in their first month completed an MMI developed to focus on desirable resident characteristics. Application data were reviewed. First-year resident performance assessments covering the American Council for Graduate Medical Education (ACGME) core competencies, along with professionalism and performance concerns, were obtained. Multiple logistic regressions were employed and MMI correlations were compared with program rank lists and typical selection factors. RESULTS: An individual's score on the MMI correlated with overall performance (p < 0.05) in single logistic regression. MMI correlated with ACGME individual competencies patient care and procedural skills at a less robust level (p < 0.1), but not with any other outcomes. Rank list position correlated with the diagnostic skill competency (p < 0.05), but no others. Traditional selection factors correlated with overall performance, disciplinary action, patient care, medical knowledge, and diagnostic skills (p < 0.05). MMI was not correlated significantly with the outcomes when included in multiple ordinal logistic regression with other selection factors. CONCLUSIONS: MMI scores correlate with overall performance, but are not statistically significant when other traditional selection factors were considered. The MMI process seems potentially superior to program rank list at correlating with first-year performance. The MMI may provide additional benefit when examined using a larger and more diverse sample.

Selecting Residents in Podiatric Medicine and Surgery.

Limited information exists to guide students of podiatric medicine and residency directors through the resident selection process. The present study aimed to evaluate the podiatric medicine and surgery resident selection process using an online survey. Residency directors of podiatric medicine and surgery programs across the United States and fourth-year students across all 9 colleges of podiatric medicine were contacted for participation. Two separate surveys were created, one for the directors and one for the students. The directors and students were asked the relative importance of 21 items considered in resident selection on a 7-point importance scale. Subsequent questions covered an array of related topics. The directors, compared with the students, identified the following items as more important (p < .05): previous disciplinary actions against the student, number of classes failed during school, undergraduate experiences and activities, number of Part I board attempts, class rank, involvement in research, and grade point average during podiatric medical school. The manual dexterity portion of the residency interview was considered significantly more important by the students than the directors. The directors more satisfied with their residents placed greater importance on the following items (p < .05): opinions of current residents, opinions of other attending physicians, and letters of recommendation. Additional trends and differences were also discovered. The results of the present study provide baseline data on the selection of podiatric medicine and surgery residents.

Reimagining general surgery resident selection: Collaborative innovation through design thinking.

Introduction: The process by which surgery residency programs select applicants is complex, opaque, and susceptible to bias. Despite attempts by program directors and educational researchers to address these issues, residents have limited ability to affect change within the process at present. Here, we present the results of a design thinking brainstorm to improve resident selection and propose this technique as a framework for surgical residents to creatively solve problems and generate actionable changes. Methods: Members of the Collaboration of Surgical Education Fellows (CoSEF) used the design thinking framework to brainstorm ways to improve the resident selection process. Members participated in one virtual focus group focused on identifying pain points and developing divergent solutions to those pain points. Pain points and solutions were subsequently organized into themes. Finally, members participated in a second virtual focus group to design prototypes to test the proposed solutions. Results: Sixteen CoSEF members participated in one or both focus groups. Participants identified twelve pain points and 57 potential solutions. Pain points and solutions were grouped into the three themes of transparency, fairness, and applicant experience. Members subsequently developed five prototype ideas that could be rapidly developed and tested to improve resident selection. Conclusions: The design thinking framework can help surgical residents come up with creative ideas to improve pain points within surgical training. Furthermore, this framework can supplement existing quantitative and qualitative methods within surgical education research. Future work will be needed to implement the prototypes devised during our sessions and turn them into complete interventions. Key message: In this paper, we demonstrate the results of a resident-led design thinking brainstorm on improving resident selection in which our team identified twelve pain points in resident selection, ideated 57 solutions, and developed five prototypes for further testing. In addition to sharing our results, we believe design thinking can be a useful framework for creative problem solving within surgical education.

Going Above and Beyond With SJTs: Impact of Applicant Characteristics on Open Response SJT Participation.

BACKGROUND: Situational judgment tests (SJT) have gained popularity as a standardized assessment of nontechnical competencies for applicants to medical school and residency. SJT formats range from rating the effectiveness of potential response options to solely open response. We investigated differences in test-taking patterns between responders and nonresponders to optional open response SJT questions during the application process. METHODS: This was a prospective multi-institutional study of general surgery applicants to seven residency programs. Applicants completed a 32-item SJT designed to measure ten core competencies: adaptability, attention to detail, communication, dependability, feedback receptivity, integrity, professionalism, resilience, self-directed learning, and team orientation. Each SJT item included an optional, nonscored, open response space for applicants to provide a behavioral response if they desired. Trends in applicant gender, race, ethnicity, medical school ranking, and USMLE scores were examined between the responder versus nonresponder group. RESULTS: In total, 1491 general surgery applicants were invited to complete the surgery-specific SJT. Of these, 1454 (97.5%) candidates completed the assessment and 1177 (78.9%) provided additional responses to at least one of the 32 SJT scenario sets. There were no differences in overall SJT performance, USMLE scores (Step 1: 235, SD 14, Step 2: 250, SD 11), race and/or ethnicity between the responder and nonresponder groups. Responders were more likely to be from a top 25 medical school (p < 0.05) compared to the nonresponder group. Among applicants who completed any open response questions, women completed a significantly higher number of questions compared to men (7.21 vs 6.07, p = 0.003). The number of open responses provided correlated with higher scores on SJT items measuring dependability (r = 0.07, p = 0.007). CONCLUSIONS: SJT design and format has the potential to impact test-taker response patterns. SJT developers and adopters should ensure test format and design have no unintended consequences prior to implementation.

The Prevalence and Distribution of Osteopathic Chief Residents in Emergency Medicine.

Background The chief resident position fulfills important administrative and educational functions for emergency medicine (EM) residency programs and has been associated with advanced academic and career opportunities. This study seeks to determine the prevalence and distribution of Doctor of Osteopathic Medicine (D.O.) and allopathic Doctor of Medicine (M.D.) degrees among chief residents within EM residencies. Methodology EM residency program websites, affiliated social media accounts on X (formerly Twitter) and Instagram, and program coordinator surveys were used to collect data, including the number of current residents, chief residents, and the listed medical degrees for residents and program directors during the 2021-2022 and 2022-2023 academic years. A Pearson's chi-square test was used to compare the number of residents, chief residents, and program directors by medical degree. Results A total of 188/229 (82.1%) and 201/229 (87.8%) eligible EM residencies identified their current chief residents and the medical degrees of their residents for the academic year 2021-2022 and 2022-2023, respectively. Of 14,487 EM residents included during the study period, 3,676 (25.4%) were D.O.s, and of the 1,230 chief residents identified, 362 (29.4%) were D.O.s. Conclusions The proportion of D.O. chief residents was higher than the proportion of D.O. residents within EM residencies. However, osteopathic residents were asymmetrically distributed across programs and were most likely to serve as chief residents at programs with a higher proportion of D.O. trainees and at programs with osteopathic program directors.