Identifying Gender and Racial Bias in Pediatric Fellowship Letters of Recommendation: Do Word Choices Influence Interview Decisions?

OBJECTIVES: To describe linguistic differences in letters of recommendation (LORs) for pediatric fellowship candidates based on applicant and letter writer demographics and to examine if these differences influenced the decision to interview a candidate for a fellowship position. STUDY DESIGN: LORs for applicants to 8 pediatric subspecialty fellowships at a single academic center from the 2020 Match were analyzed in this cross-sectional study. Frequency of validated agentic and communal terms in each letter were determined by a language processing web application. Bias was determined as having a >5% surplus of agentic or communal terms. RESULTS: We analyzed 1521 LORs from 409 applicants: 69% were women, 28% were under-represented minorities in medicine (URM), and 50% were invited to interview. Overall, 66% of LORs were agentic biased, 16% communal biased, and 19% neutral. There was no difference in bias in LORs by an applicant's gender (woman 67% agentic vs man 62% agentic; P = .058), race, or ethnicity (non-URM 65% agentic vs URM 67% agentic; P = .660). Despite a lower frequency of agentic terms in LORs for applicants invited for interviews, when accounting for other components of an application and applicant demographics, no significant association was made between language bias in LORs and fellowship interview status. CONCLUSIONS: The frequency of agentic and communal terms in LORs for pediatric subspecialty fellowship candidates were not found to influence the decision to invite a candidate to interview. However, raising awareness of potential areas of bias within the pediatric fellowship selection process might lead to a more equitable and holistic approach to application review.

Words Used in Letters of Recommendation for Pediatric Residency Applicants: Demographic Differences and Impact on Interviews.

OBJECTIVE: To describe differences in agentic (achievement) and communal (relationship) terms in letters of recommendation (LORs) for pediatric residency candidates by applicant and letter writer demographics and to examine if LOR language is associated with interview status. METHODS: A random sample of applicant profiles and LORs submitted to one institution were analyzed from the 2020-21 Match. Letters of recommendation text was inputted into a customized natural language processing application which determined the frequency of agentic and communal words in each LOR. Neutral LORs were defined as having< 5% surplus of agentic or communal terms. RESULTS: We analyzed 2094 LORs from 573 applicants: 78% were women, 24% were under-represented in medicine (URiM), and 39% were invited to interview. Most letter writers were women (55%) and of senior academic rank (49%). Overall, 53% of LORs were agency biased, 25% communal biased, and 23% neutral. There was no difference in agency and communally biased LORs by an applicant's gender (men 53% agentic vs women 53% agentic, P = .424), race or ethnicity (non-URiM 53% agentic vs URiM 51% agentic, P = .631). Male letter writers used significantly more agentic terms (8.5%) compared to women (6.7% agentic) or writers of both genders (3.1% communal) (P = .008). Applicants invited to interview were more likely to have a neutral LOR; however, no significant association existed between language and interview status. CONCLUSIONS: No significant differences in language were found by applicant gender or race among pediatric residency candidates. Identifying potential biases within pediatric residency selection processes is important in creating an equitable approach to application review.