Gender-based Language Differences in Letters of Recommendation.

ABSTRACT

Gender stereotyping is the practice of assigning or ascribing specific characteristics, differences, or identities to a person solely based on their gender. Biased conceptions of gender can create barriers to equality and need to be proactively identified and addressed. In biomedical education, letters of recommendation (LOR) are considered an important source for evaluating candidates' past performance. Because LOR is subjective and has no standard formatting requirements for the writer, potential language bias can be introduced. Natural language processing (NLP) offers a promising solution to detect language bias in LOR through automatic extraction of sensitive language and identification of letters with strong biases. In our study, we developed, evaluated, and deployed four NLP different methods (sublanguage analysis, dictionary-based approach, rule-based approach, and deep learning approach) for the extraction of psycholinguistics and thematic characteristics in LORs from three different physical therapy residency programs (Neurologic, Orthopaedic, and Sport) at Mayo Clinic. The evaluation statistics suggest that both MedTaggerIE model and Bidirectional Encoder Representations from Transformers model achieved moderate-high performance across eight different thematic categories. Through the pilot demonstration study, we learned that male writers were more likely to use the words 'intelligence', 'exceptional', and 'pursue' and male applicants were more likely to have the words 'strength', 'interpersonal skills', 'conversations', and 'pursue' in their letters of recommendation. Thematic analysis suggested that male and female writers have significant differences in expressing doubt, motivation, and recommendation. Findings derived from the study needed to be carefully interpreted based on the context of the study setting, residency programs, and data. A follow-up demonstration study is needed to further evaluate and interpret the findings.