Effect of a Workshop to Break the Bias Habit for Internal Medicine Faculty: A Multisite Cluster Randomized Controlled Study.

PURPOSE: Recognition that cultural stereotypes can unintentionally perpetuate inequities throughout academic medicine has led to calls for "implicit bias training" without strong evidence to support these recommendations and some evidence of potential harm. The authors sought to determine the effectiveness of a single 3-hour workshop in helping department of medicine faculty overcome implicit stereotype-based bias and in improving the climate in the working environment. METHOD: A multisite cluster randomized controlled study (October 2017 to April 2021) with clustering at the level of divisions within departments and participant-level analysis of survey responses involved 8,657 faculty in 204 divisions in 19 departments of medicine: 4,424 in the intervention group (1,526 attended a workshop) and 4,233 in the control group. Online surveys at baseline (3,764/8,657 = 43.48% response rate) and 3 months after the workshop (2,962/7,715 = 38.39% response rate) assessed bias awareness, bias-reducing intentional behavioral change, and perceptions of division climate. RESULTS: At 3 months, faculty in the intervention vs control divisions showed greater increases in awareness of personal bias vulnerability ( b = 0.190 [95% CI, 0.031 to 0.349], P = .02), bias reduction self-efficacy ( b = 0.097 [95% CI, 0.010 to 0.184], P = .03), and taking action to reduce bias ( b = 0.113 [95% CI, 0.007 to 0.219], P = .04). The workshop had no effect on climate or burnout, but slightly increased perceptions of respectful division meetings ( b = 0.072 [95% CI, 0.0003 to 0.143], P = .049). CONCLUSIONS: Results of this study should give confidence to those designing prodiversity interventions for faculty in academic medical centers that a single workshop which promotes awareness of stereotype-based implicit bias, explains and labels common bias concepts, and provides evidence-based strategies for participants to practice appears to have no harms and may have significant benefits in empowering faculty to break the bias habit.

Engaging faculty in a workshop intervention on overcoming the influence of implicit bias.

INTRODUCTION: To study the effectiveness of any educational intervention for faculty requires first that they attend the training. Using attendance as a measure of faculty engagement, this study examined factors associated with the percentage of faculty in divisions of departments of medicine who attended a workshop as part of a multisite study. METHODS: Between October 2018 and March 2020, 1675 of 4767 faculty in 120 divisions of 14 departments of medicine attended a 3-hour in-person workshop as part of the Bias Reduction in Internal Medicine (BRIM) initiative. This paper describes the workshop development and study design. The number of faculty per division ranged from 5 to 296. Attendance rates varied from 2.7% to 90.1%. Taking a quality improvement approach, the study team brainstormed factors potentially related to variations in workshop attendance, constructed several division- and institution-level variables, and assessed the significance of factors on workshop attendance with hierarchical linear models. RESULTS: The following were positively associated with workshop attendance rate: the division head attended the workshop, the BRIM principal investigator gave Medical Grand Rounds, and the percentage of local workshop presenters who completed training. Workshop attendance rates fell when departments identified more than five on-site study leaders. CONCLUSIONS: Factors associated with higher workshop attendance may have increased the perceived status and value of attending the workshop, leading faculty to choose the workshop over other competing demands. For future investigators studying educational interventions that require participation of faculty in clinical departments at multiple sites, this work offers several valuable lessons.

"This happens all the time": a Qualitative Study of General Internists' Experiences with Discriminatory Patients.

BACKGROUND: Workplace discrimination negatively affects physicians of color personally and professionally. Although the occurrence of discrimination from patients has been visible in social media, popular press, and personal essays, scant research exists on patients as a source of discrimination directed at physicians of color. OBJECTIVE: To explore practicing general internists' experiences observing or interacting with patients exhibiting discriminatory behavior directed at physicians of color. DESIGN: A qualitative study with in-depth, semi-structured, one-on-one telephone interviews conducted and recorded between May and September 2019. PARTICIPANTS: A purposive sample of 24 general internists practicing at 12 academic health centers in the USA: 14 self-identified as White and 10 as a physician of color, which included Asian, Black, Native American, and self-identified other race. APPROACH: Four coders analyzed the transcribed and verified interview text; thematic analysis was used to inductively identify cohesive themes and subthemes. KEY RESULTS: Analyses revealed four major themes: (1) assumption that a legitimate doctor is White, male, and able-bodied; (2) legacy of the Black experience; (3) working through the struggle of discrimination; and (4) ethical dilemma of providing care to discriminatory patients. In addition to discrimination from patients based on a physician's race or ethnicity, participants described experiencing or observing discrimination based on a physician's gender and disability status. Participants generally expressed a need for greater support from colleagues and more guidance from institutional policies. CONCLUSIONS: General internists practicing in academic settings reported observing or experiencing discrimination from patients based on the physician's race, ethnicity, gender (or their intersection), and disability status and the ethical dilemma of providing care to such patients. These results contribute to growing evidence of the need for institutions to better support an increasingly diverse physician workforce with policies and specific guidance to help physicians respond to discrimination from patients while still providing quality care.

Gender Can Influence Student Experiences in MD-PhD Training.

Background: Female physician-scientists have led major advances in medicine broadly and particularly in women's health. Women remain underrepresented in dual MD-PhD degree programs that train many physician-scientists despite gender parity among medical and biomedical research students. Materials and Methods: To explore how the training environment might be experienced differently for male and female students in one MD-PhD program, the authors analyzed gender differences in annual symposium speakers with exact binomial tests, student participation as question-askers at a weekly seminar with logistic regression, and number of publications with quasi-Poisson generalized linear models. They compared male and female students' perceptions of gender-based discrimination using a survey, including qualitative analysis of free text responses. The program consisted of 71 total students in the 2017-2018 and 2018-2019 academic years. Female students comprised 42.0% (81/191) of program matriculants from 1997 to 2019. Results: Male and female students were equally likely to present at the annual program symposium, but faculty (p = 0.001) and keynote (p = 0.012) presenters were more likely to be male. Compared with male counterparts, female students asked fewer seminar questions (p < 0.005) and female speakers received more questions (p = 0.03). Female students perceived less support and differed from men in reasons for asking or not asking seminar questions. Free text responses described repeated small acts of discrimination toward women with cumulative impact. Positive program changes followed presentation of findings to program leaders and students. Conclusions: The authors identified several aspects of one MD-PhD program that could discourage career or training persistence of female students. Increasing awareness of these issues was temporally related to positive programmatic changes.

Will Losing Black Physicians Be a Consequence of the COVID-19 Pandemic?

A compelling case exists that increasing the number of Black physicians trained and practicing in the United States is one effective intervention to promote health equity and reduce the persistent health disparities that have become glaringly evident during the COVID-19 pandemic. However, the U.S. physician workforce has relatively few Black physicians. Blacks comprise approximately 13% of the U.S. population but only 5% of practicing physicians. In this Invited Commentary, the authors caution that the COVID-19 pandemic may erode the meager progress that has been made in increasing the number of Black physicians. This loss of Black physicians may happen because Black patients are overrepresented among cases of COVID-19, Black physicians care for relatively more Black patients often in settings with less access to SARS-CoV-2 testing and personal protective equipment, and Black physicians have more comorbid chronic conditions that increase their own susceptibility to mortality from COVID-19. All organizations in which physicians train and practice must redouble their efforts to recruit, train, and retain Black physicians. If nothing else, the COVID-19 pandemic must make academic health centers and health care systems recognize Black physicians as the precious resource they are and protect and reward them accordingly.

Discrimination Toward Physicians of Color: A Systematic Review.

PURPOSE: To systematically review published research exploring workplace discrimination toward physicians of color with a focus on discrimination from patients. METHOD: The authors searched PubMed, PsycInfo, CINAHL, Scopus, Academic Search Premier, and Web of Science from 1990 through 2017 and performed supplemental manual bibliographic searches. Eligible studies were in English and assessed workplace discrimination experienced by physicians of color practicing in the U.S. including physicians from ethnic/racial groups underrepresented in medicine, Asians, and international medical graduates. Two reviewers independently screened titles and abstracts, 3 reviewers read the full text of eligible studies, and 2 reviewers extracted data and appraised quality using Joanna Briggs Institute checklist for qualitative research or the AXIS tool for quality of cross-sectional studies. RESULTS: Of the 19 eligible studies, 6 conducted surveys and 13 analyzed data from interviews and/or focus groups; most were medium quality. All provided evidence to support the high prevalence of workplace discrimination experienced by physicians of color, particularly black physicians and women of color. Discrimination was associated with adverse effects on career, work environment, and health. In the few studies inquiring about patient interactions, discrimination was predominantly refusal of care. No study evaluated an intervention to reduce workplace discrimination experienced by physicians of color. Ethnic/racial groups were inconsistent across studies, and some samples included physicians in Canada, non-physician faculty, or trainees. CONCLUSION: With physicians of color comprising a growing percentage of the U.S. physician workforce, healthcare organizations must examine and implement effective ways to ensure a healthy and supportive work environment.

Gender, Race, and Grant Reviews: Translating and Responding to Research Feedback.

Grant-writing and grant-getting are key to success in many academic disciplines, but research points to gender gaps in both, especially as careers progress. Using a sample of National Institutes of Health (NIH) K-Awardees-Principal Investigators of Mentored Career Development Awards-we examined gender and race effects in response to imagined negative grant reviews that emphasized either promise or inadequacy. Women translated both forms of feedback into worse NIH priority scores than did men and showed reduced motivation to reapply for funding following the review highlighting inadequacy. Translation of feedback mediated the effects of gender on motivation, changing one's research focus, and advice-seeking. Race effects were less consistent, and race did not moderate effects of gender. We suggest that gender bias in grant reviews (i.e., greater likelihood of highlighting inadequacy in reviews of women's grants), along with gender differences in responsiveness to feedback, may contribute to women's underrepresentation in academic medicine.

Low agreement among reviewers evaluating the same NIH grant applications.

Obtaining grant funding from the National Institutes of Health (NIH) is increasingly competitive, as funding success rates have declined over the past decade. To allocate relatively scarce funds, scientific peer reviewers must differentiate the very best applications from comparatively weaker ones. Despite the importance of this determination, little research has explored how reviewers assign ratings to the applications they review and whether there is consistency in the reviewers' evaluation of the same application. Replicating all aspects of the NIH peer-review process, we examined 43 individual reviewers' ratings and written critiques of the same group of 25 NIH grant applications. Results showed no agreement among reviewers regarding the quality of the applications in either their qualitative or quantitative evaluations. Although all reviewers received the same instructions on how to rate applications and format their written critiques, we also found no agreement in how reviewers "translated" a given number of strengths and weaknesses into a numeric rating. It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant. This research replicates the NIH peer-review process to examine in detail the qualitative and quantitative judgments of different reviewers examining the same application, and our results have broad relevance for scientific grant peer review.

The More Things Change, the More They Stay the Same: A Study to Evaluate Compliance With Inclusion and Assessment of Women and Minorities in Randomized Controlled Trials.

PURPOSE: The National Institutes of Health (NIH) Revitalization Act of 1993 requires NIH-funded clinical trials to include women and minorities as participants and assess outcomes by sex and race or ethnicity. The objective of this study was to investigate current levels of compliance with these guidelines for inclusion, analysis, and reporting in NIH-funded randomized controlled trials (RCTs) and compare the results with those from 2009 and 2004, which the authors reported previously. METHOD: The authors identified 782 RCTs published in 14 leading U.S. medical journals in 2015 with a PubMed search. Of those, 142 were the primary report of an NIH-funded RCT, conducted in the United States, and eligible for analysis. The authors reviewed abstract, text, and tables of each eligible study as well as any follow-up published commentary to determine compliance with NIH guidelines. RESULTS: Thirty-five studies limited enrollment to one sex. The median enrollment of women in the remaining 107 studies was 46%, but 16 (15.0%) enrolled less than 30% women. Twenty-eight of the 107 (26%) reported at least one outcome by sex or explicitly included sex as a covariate in statistical analysis. Of the 142 studies, 19 (13.4%) analyzed or reported outcomes by race or ethnicity. There were no statistically significant changes in inclusion, analysis, or reporting by sex, race, or ethnicity compared with the previous studies. CONCLUSIONS: NIH policies have not resulted in significant increases in reporting results by sex, race, or ethnicity. The authors recommend strong journal policies to increase compliance with NIH policies.

Fair Play: A Study of Scientific Workforce Trainers' Experience Playing an Educational Video Game about Racial Bias.

Explicit racial bias has decreased in the United States, but racial stereotypes still exist and conspire in multiple ways to perpetuate the underparticipation of Blacks in science careers. Capitalizing on the potential effectiveness of role-playing video games to promote the type of active learning required to increase awareness of and reduce subtle racial bias, we developed the video game Fair Play, in which players take on the role of Jamal, a Black male graduate student in science, who experiences discrimination in his PhD program. We describe a mixed-methods evaluation of the experience of scientific workforce trainers who played Fair Play at the National Institutes of Health Division of Training Workforce Development and Diversity program directors' meeting in 2013 (n = 47; 76% female, n = 34; 53% nonwhite, n = 26). The evaluation findings suggest that Fair Play can promote perspective taking and increase bias literacy, which are steps toward reducing racial bias and affording Blacks equal opportunities to excel in science.

Write More Articles, Get More Grants: The Impact of Department Climate on Faculty Research Productivity.

BACKGROUND: Many studies find that female faculty in academic medicine, science, and engineering experience adverse workplace climates. This study longitudinally investigates whether department climate is associated with future research productivity and whether the associations are stronger for female than male faculty. METHOD: Two waves of a faculty climate survey, institutional grant records, and publication records were collected for 789 faculties in academic medicine, science, and engineering at the University of Wisconsin-Madison between 2000 and 2010. Research productivity was measured as Number of Publications and Number of Grants awarded, and department climate was measured with scales for professional interactions, department decision-making practices, climate for underrepresented groups, and work/life balance. Ordinary least squares and negative binomial regression methods were used to assess gender differences in productivity, influences of department climate on productivity, and gender differences in effects of climate on productivity. RESULTS: Female faculty published fewer articles and were awarded fewer grants in the baseline period, but their productivity did not differ from male faculty on these measures in subsequent years. Number of Publications was positively affected by professional interactions, but negatively affected by positive work/life balance. Number of Grants awarded was positively affected by climate for underrepresented groups. These main effects did not differ by gender; however, some three-way interactions illuminated how different aspects of department climate affected productivity differently for men and women in specific situations. CONCLUSIONS: In perhaps the first study to assess the longitudinal impact of department climate on faculty research productivity, positive department climate is associated with significantly greater productivity for all faculty-women and men. However, some positive aspects of climate (specifically, work/life balance) may be associated with lower productivity for some female faculty at specific career periods. These findings suggest that departments that wish to increase grants and publications would be wise to foster a positive workplace climate.

Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers' Critiques.

BACKGROUND: Women are less successful than men in renewing R01 grants from the National Institutes of Health. Continuing to probe text mining as a tool to identify gender bias in peer review, we used algorithmic text mining and qualitative analysis to examine a sample of critiques from men's and women's R01 renewal applications previously analyzed by counting and comparing word categories. METHODS: We analyzed 241 critiques from 79 Summary Statements for 51 R01 renewals awarded to 45 investigators (64% male, 89% white, 80% PhD) at the University of Wisconsin-Madison between 2010 and 2014. We used latent Dirichlet allocation to discover evaluative "topics" (i.e., words that co-occur with high probability). We then qualitatively examined the context in which evaluative words occurred for male and female investigators. We also examined sex differences in assigned scores controlling for investigator productivity. RESULTS: Text analysis results showed that male investigators were described as "leaders" and "pioneers" in their "fields," with "highly innovative" and "highly significant research." By comparison, female investigators were characterized as having "expertise" and working in "excellent" environments. Applications from men received significantly better priority, approach, and significance scores, which could not be accounted for by differences in productivity. CONCLUSIONS: Results confirm our previous analyses suggesting that gender stereotypes operate in R01 grant peer review. Reviewers may more easily view male than female investigators as scientific leaders with significant and innovative research, and score their applications more competitively. Such implicit bias may contribute to sex differences in award rates for R01 renewals.

Analysis of National Institutes of Health R01 Application Critiques, Impact, and Criteria Scores: Does the Sex of the Principal Investigator Make a Difference?

PURPOSE: Prior text analysis of R01 critiques suggested that female applicants may be disadvantaged in National Institutes of Health (NIH) peer review, particularly for renewals. NIH altered its review format in 2009. The authors examined R01 critiques and scoring in the new format for differences due to principal investigator (PI) sex. METHOD: The authors analyzed 739 critiques-268 from 88 unfunded and 471 from 153 funded applications for grants awarded to 125 PIs (76 males, 49 females) at the University of Wisconsin-Madison between 2010 and 2014. The authors used seven word categories for text analysis: ability, achievement, agentic, negative evaluation, positive evaluation, research, and standout adjectives. The authors used regression models to compare priority and criteria scores, and results from text analysis for differences due to PI sex and whether the application was for a new (Type 1) or renewal (Type 2) R01. RESULTS: Approach scores predicted priority scores for all PIs' applications (P < .001), but scores and critiques differed significantly for male and female PIs' Type 2 applications. Reviewers assigned significantly worse priority, approach, and significance scores to female than male PIs' Type 2 applications, despite using standout adjectives (e.g., "outstanding," "excellent") and making references to ability in more critiques (P < .05 for all comparisons). CONCLUSIONS: The authors' analyses suggest that subtle gender bias may continue to operate in the post-2009 NIH review format in ways that could lead reviewers to implicitly hold male and female applicants to different standards of evaluation, particularly for R01 renewals.

Patterns of Feedback on the Bridge to Independence: A Qualitative Thematic Analysis of NIH Mentored Career Development Award Application Critiques.

BACKGROUND: NIH Mentored Career Development (K) Awards bridge investigators from mentored to independent research. A smaller proportion of women than men succeed in this transition. The aim of this qualitative study was to analyze reviewers' narrative critiques of K award applications and explore thematic content of feedback provided to male and female applicants. METHOD: We collected 88 critiques, 34 from 9 unfunded and 54 from 18 funded applications, from 70% (n = 26) of investigators at the University of Wisconsin-Madison with K awards funded between 2005 and 2009 on the first submission or after revision. We qualitatively analyzed text in the 5 critique sections: candidate, career development plan, research plan, mentors, and environment and institutional commitment. We explored thematic content within these sections for male and female applicants and for applicants who had received a subsequent independent research award by 2014. RESULTS: Themes revealed consistent areas of criticism for unfunded applications and praise for funded applications. Subtle variations in thematic content appeared for male and female applicants: For male applicants criticism was often followed by advice but for female applicants it was followed by questions about ability; praise recurrently characterized male but not female applicants' research as highly significant with optimism for future independence. Female K awardees that obtained subsequent independent awards stood out as having track records described as "outstanding." CONCLUSION: This exploratory study suggests that K award reviewer feedback, particularly for female applicants, should be investigated as a potential contributor to research persistence and success in crossing the bridge to independence.

Afraid of being "witchy with a 'b'": a qualitative study of how gender influences residents' experiences leading cardiopulmonary resuscitation.

PURPOSE: Ineffective leadership during cardiopulmonary resuscitation ("code") can negatively affect a patient's likelihood of survival. In most teaching hospitals, internal medicine residents lead codes. In this study, the authors explored internal medicine residents' experiences leading codes, with a particular focus on how gender influences the code leadership experience. METHOD: The authors conducted individual, semistructured telephone or in-person interviews with 25 residents (May 2012 to February 2013) from 9 U.S. internal medicine residency programs. They audio recorded and transcribed the interviews and then thematically analyzed the transcribed text. RESULTS: Participants viewed a successful code as one with effective leadership. They agreed that the ideal code leader was an authoritative presence; spoke with a deep, loud voice; used clear, direct communication; and appeared calm. Although equally able to lead codes as their male colleagues, female participants described feeling stress from having to violate gender behavioral norms in the role of code leader. In response, some female participants adopted rituals to signal the suspension of gender norms while leading a code. Others apologized afterwards for their counternormative behavior. CONCLUSIONS: Ideal code leadership embodies highly agentic, stereotypical male behaviors. Female residents employed strategies to better integrate the competing identities of code leader and female gender. In the future, residency training should acknowledge how female gender stereotypes may conflict with the behaviors required to enact code leadership and offer some strategies, such as those used by the female residents in this study, to help women integrate these dual identities.