Examining Implicit Bias Differences in Pediatric Surgical Fellowship Letters of Recommendation Using Natural Language Processing.
J Surg Educ
; 80(4): 547-555, 2023 04.
Article
em En
| MEDLINE
| ID: mdl-36529662
ABSTRACT
OBJECTIVE:
We analyzed the prevalence and type of bias in letters of recommendation (LOR) for pediatric surgical fellowship applications from 2016-2021â¯using natural language processing (NLP) at a quaternary careâ¯academic hospital.DESIGN:
Demographics were extracted from submitted applications. Theâ¯Valence Aware Dictionary forâ¯sEntimentâ¯Reasoning (VADER) modelâ¯was used to calculate polarity scores. The National Research Councilâ¯dataset was used for emotion and intensity analysis. â¯The Kruskal-Wallis H-testâ¯was used to determine statistical significance.â¯SETTING:
This study took place at a single, academic, free standing quaternary care children's hospital with an ACGME accredited pediatric surgery fellowship.PARTICIPANTS:
Applicants to a single pediatric surgery fellowship were selected for this study from 2016 to 2021. A total of 182 individual applicants were included and 701 letters of recommendation were analyzed.RESULTS:
Black applicants had the highest meanâ¯polarityâ¯(most positive), while Hispanicâ¯applicants had the lowest. â¯Overall differences between polarity distributions were not statistically significant.â¯â¯ The intensity of emotions showed that differences in "anger" were statistically significant (p=0.03).⯠Mean polarity was higher for applicants that successfully matched in pediatric surgery.DISCUSSION:
This study identified differences in LORs based on racial and gender demographics submitted as part of pediatric surgical fellowship applications to a single training program. The presence of bias in letters of recommendation can lead to inequities in demographics to a given program. While difficult to detect for humans, natural language processing is able to detect bias as well as differences in polarity and emotional intensity. While theâ¯types of emotions identified in this study are highly similar among race and gender groups, the intensity of these emotionsâ¯revealed differences,â¯with "anger" being most significant.CONCLUSION:
From this work, it can be concluded thatâ¯bias in LORs, as reflected asâ¯differences in polarity,â¯which is likely a result of the intensity of the emotions being used and not the types of emotions being expressed.â¯â¯ Natural language processing shows promise in identification of subtle areas of bias that may influence an individual's likelihood of successful matching.Palavras-chave
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Especialidades Cirúrgicas
/
Internato e Residência
Tipo de estudo:
Prognostic_studies
/
Risk_factors_studies
Limite:
Child
/
Humans
Idioma:
En
Ano de publicação:
2023
Tipo de documento:
Article