Changing Perceptions About the Dermatology Residency Application Process Among Applicants and Program Directors: 2020-2022.

Importance Over the last two years, dermatology has undergone significant reforms in the residency application process in efforts to reduce applicant stress, increase equity, and due to the COVID-19 pandemic.  Objective We aimed to determine applicant and program director (PD) perspectives in implementing these changes over the last two application cycles.  Design, setting, and participants Anonymous online surveys were administered by the Association of American Medical Colleges (AAMC) to PDs and applicants from the 2021-2022 dermatology residency application cycle. These results were compared with similar online surveys distributed after the 2020-2021 cycle. Results Coordinated interview release was introduced in the 2020-2021 dermatology application cycle. At that time, 57% of PDs and 84% of applicants wished that more programs participated in the release, compared to 53% and 84%, respectively, in the 2021-2022 cycle. In 2021, 28% of PDs reported matching applicants from their home institution higher on their list compared to 14% in 2022. In 2021 and 2022, 94% of PDs reported that diversity was an explicit goal in their application process. However, in 2021, 33% of PDs reported that they matched no UIMs (underrepresented in medicine) in their cohort, which grew to 39% in 2022. Conclusions This study identifies key trends in applicant and PD perspectives associated with changes in the application process such as coordinated interview release, virtual interviews, and emphasis on diversity. Additional data is needed from subsequent cycles in order to determine the efficacy of these reforms.

Evaluating a Situational Judgment Test for Use in Medical School Admissions: Two Years of AAMC PREview Exam Administration Data.

PURPOSE: To examine the relationship between the Association of American Medical Colleges (AAMC) Professional Readiness Exam (PREview) scores and other admissions data, group differences in mean PREview scores, and whether adding a new assessment tool affected the volume and composition of applicant pools. METHOD: Data from the 2020 and 2021 PREview exam administrations were analyzed. Two U.S. schools participated in the PREview pilot in 2020 and 6 U.S. schools participated in 2021. PREview scores were paired with data from the American Medical College Application Service (undergraduate grade point averages [GPAs], Medical College Admission Test [MCAT] scores, race, and ethnicity) and participating schools (interview ratings). RESULTS: Data included 19,525 PREview scores from 18,549 unique PREview examinees. Correlations between PREview scores and undergraduate GPAs ( r = .16) and MCAT scores ( r = .29) were small and positive. Correlations between PREview scores and interview ratings were also small and positive, ranging between .09 and .14 after correcting for range restriction. Small group differences in mean PREview scores were observed between White and Black or African American and White and Hispanic, Latino, or of Spanish origin examinees. The addition of the PREview exam did not substantially change the volume or composition of participating schools' applicant pools. CONCLUSIONS: Results suggest the PREview exam measures knowledge of competencies that are distinct from those measured by other measures used in medical school admissions. Observed group differences were smaller than group differences observed with traditional academic assessments and evaluations. The addition of the PREview exam did not substantially change the overall volume of applications or the proportions of out-of-state, underrepresented in medicine, or lower socioeconomic status applicants. While more research is needed, these results suggest the PREview exam may provide unique information to the admissions process without adversely affecting applicant pools.

Impact of Preference Signals on Interview Selection Across Multiple Residency Specialties and Programs.

Background Program signaling is an innovation that allows applicants to express interest in specific programs while providing programs the opportunity to review genuinely interested applicants during the interview selection process. Objective To examine the influence of program signaling on "selected to interview" status across specialties in the 2022 Electronic Residency Application Service (ERAS) application cycle. Methods Dermatology, general surgery-categorical (GS), and internal medicine-categorical (IM-C) programs that participated in the signaling section of the 2022 supplemental ERAS application (SuppApp) were included. Applicant signal data was collected from SuppApp, applicant self-reported characteristics collected from the MyERAS Application for Residency Applicants, and 2020 program characteristics collected from the 2020 GME Track Survey. Applicant probability of being selected for interview was analyzed using logistic regression, determined by the selected to interview status in the ERAS Program Director's WorkStation. Results Dermatology had a 62% participation rate (73 of 117 programs), GS a 75% participation rate (174 of 232 programs), and IM-C an 86% participation rate (309 of 361 programs). In all 3 specialties examined, on average, signaling increased the likelihood of being selected to interview compared to applicants who did not signal. This finding held across gender and underrepresented in medicine (UIM) groups in all 3 specialties, across applicant types (MDs, DOs, international medical graduates) for GS and IM-C, and after controlling for United States Medical Licensing Examination Step 1 scores. Conclusions Although there was variability by program, signaling increased likelihood of being selected for interview without negatively affecting any specific gender or UIM group.

Interview Invitations for Otolaryngology Residency Positions Across Demographic Groups Following Implementation of Preference Signaling.

Importance: Preference signaling is a new initiative in the residency application process that has been adopted by 17 specialties that include more than 80% of applicants in the 2023 National Resident Matching cycle. The association of signals with interview selection rate across applicant demographics has not been fully examined. Objective: To assess the validity of survey-based data on the association of preference signals with interview offers and describe the variation across demographic groups. Design, Setting, and Participants: This cross-sectional study examined the interview selection outcomes across demographic groups for applications with and without signals in the 2021 Otolaryngology National Resident Matching cycle. Data were obtained from a post-hoc collaboration between the Association of American Medical Colleges and the Otolaryngology Program Directors Organization evaluating the first preference signaling program used in residency application. Participants included otolaryngology residency applicants in the 2021 application cycle. Data were analyzed from June to July 2022. Exposures: Applicants were provided the option of submitting 5 signals to otolaryngology residency programs to indicate specific interest. Signals were used by programs when selecting candidates to interview. Main Outcomes and Measures: The main outcome of interest was the association of signaling with interview selection. A series of logistic regression analyses were conducted at the individual program level. Each program within the 3 program cohorts (overall, gender, and URM status) was evaluated with 2 models. Results: Of 636 otolaryngology applicants, 548 (86%) participated in preference signaling, including 337 men (61%) and 85 applicants (16%) who identified as underrepresented in medicine, including American Indian or Alaska Native; Black or African American; Hispanic, Latino, or of Spanish origin; or Native Hawaiian or other Pacific Islander. The median interview selection rate for applications with a signal (48% [95% CI, 27%-68%]) was significantly higher than for applications without a signal (10% [95% CI, 7%-13%]). No difference was observed in median interview selection rates with or without signals when comparing male (46% [95% CI, 24%-71%] vs 7% [95% CI, 5%-12%]) and female (50% [95% CI, 20%-80%] vs 12% [95% CI, 8%-18%]) applicants or when comparing applicants who identified as URM (53% [95% CI, 16%-88%] vs 15% [95% CI, 8%-26%]) with those who did not identify as URM (49% [95% CI, 32%-68%] vs 8% [95% CI, 5%-12%]). Conclusions and Relevance: In this cross-sectional study of otolaryngology residency applicants, preference signaling was associated with an increased likelihood of applicants being selected for interview by signaled programs. This correlation was robust and present across the demographic categories of gender and self-identification as URM. Future research should explore the associations of signaling across a broad range of specialties and the associations of signals with inclusion and position on rank order lists and match outcomes.

The AAMC Standardized Video Interview and the Electronic Standardized Letter of Evaluation in Emergency Medicine: A Comparison of Performance Characteristics.

PURPOSE: To compare the performance characteristics of the electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI), a new tool designed by the AAMC to assess interpersonal and communication skills and professionalism knowledge. METHOD: The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships. RESULTS: The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = -0.21) and US-MD applicants (d = 0.23-0.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = -0.20) and white applicants over black applicants (approaching d = 0.40). CONCLUSIONS: Small positive correlations between SVI score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance.

Personality predictors of extreme response style.

Extreme response style (ERS) refers to the tendency to overuse the endpoints of Likert-type scales. This study examined the extent to which ERS is accounted for by measures of personality, specifically, intolerance of ambiguity, simplistic thinking, and decisiveness. One hundred and sixteen pairs of undergraduate students and one of their respective peers completed a battery of questionnaires assessing these personality measures, alongside three measures of extreme responding. Results indicate that peer ratings of intolerance of ambiguity and simplistic thinking interact with the primary participant's time spent on the survey to predict the primary participant's extreme responding. Thus, those who quickly complete surveys and are intolerant of ambiguity or are simplistic thinkers are most likely to exhibit ERS. These results have implications not only for surveys using rating scales, but also illustrate how epistemic personality factors more generally influence the processing of new information.