Effect of a Workshop to Break the Bias Habit for Internal Medicine Faculty: A Multisite Cluster Randomized Controlled Study.

PURPOSE: Recognition that cultural stereotypes can unintentionally perpetuate inequities throughout academic medicine has led to calls for "implicit bias training" without strong evidence to support these recommendations and some evidence of potential harm. The authors sought to determine the effectiveness of a single 3-hour workshop in helping department of medicine faculty overcome implicit stereotype-based bias and in improving the climate in the working environment. METHOD: A multisite cluster randomized controlled study (October 2017 to April 2021) with clustering at the level of divisions within departments and participant-level analysis of survey responses involved 8,657 faculty in 204 divisions in 19 departments of medicine: 4,424 in the intervention group (1,526 attended a workshop) and 4,233 in the control group. Online surveys at baseline (3,764/8,657 = 43.48% response rate) and 3 months after the workshop (2,962/7,715 = 38.39% response rate) assessed bias awareness, bias-reducing intentional behavioral change, and perceptions of division climate. RESULTS: At 3 months, faculty in the intervention vs control divisions showed greater increases in awareness of personal bias vulnerability ( b = 0.190 [95% CI, 0.031 to 0.349], P = .02), bias reduction self-efficacy ( b = 0.097 [95% CI, 0.010 to 0.184], P = .03), and taking action to reduce bias ( b = 0.113 [95% CI, 0.007 to 0.219], P = .04). The workshop had no effect on climate or burnout, but slightly increased perceptions of respectful division meetings ( b = 0.072 [95% CI, 0.0003 to 0.143], P = .049). CONCLUSIONS: Results of this study should give confidence to those designing prodiversity interventions for faculty in academic medical centers that a single workshop which promotes awareness of stereotype-based implicit bias, explains and labels common bias concepts, and provides evidence-based strategies for participants to practice appears to have no harms and may have significant benefits in empowering faculty to break the bias habit.

Research Conducted in Women Was Deemed More Impactful but Less Publishable than the Same Research Conducted in Men.

Background: Female scientists, who are more likely than their male counterparts to study women and report findings by sex/gender, fare worse in the article peer review process. It is unknown whether the gender of research participants influences the recommendation to publish an article describing the study. Materials and Methods: Reviewers were randomly assigned to evaluate one of three versions of an article abstract describing a clinical study conducted in men, women, or individuals. Reviewers assessed the study's scientific rigor, its level of contribution to medical science, and whether they would recommend publishing the full article. Responses were analyzed with logistic regression controlling for reviewer background variables, including sex and experience level. Results: There was no significant difference in perceived research rigor by abstract condition; contribution to medical science was perceived to be greater for research conducted in women than men (odds ratio = 1.7; p = 0.030). Nevertheless, reviewers were almost twice as likely to recommend publication for research conducted in men than the same research conducted in women (predicted probability 0.606 vs. 0.322; p = 0.000). Conclusions: These results are consistent with abundant data from multiple sources showing a lower societal value placed on women than men. Because female investigators are more likely than male investigators to study women, our findings suggest a previously unrecognized bias that could contribute to gender asymmetries in the publication outcomes of peer review. This pro-male publication bias could be an additional barrier to leadership attainment for women in academic medicine and the advancement of women's health.

Engaging faculty in a workshop intervention on overcoming the influence of implicit bias.

INTRODUCTION: To study the effectiveness of any educational intervention for faculty requires first that they attend the training. Using attendance as a measure of faculty engagement, this study examined factors associated with the percentage of faculty in divisions of departments of medicine who attended a workshop as part of a multisite study. METHODS: Between October 2018 and March 2020, 1675 of 4767 faculty in 120 divisions of 14 departments of medicine attended a 3-hour in-person workshop as part of the Bias Reduction in Internal Medicine (BRIM) initiative. This paper describes the workshop development and study design. The number of faculty per division ranged from 5 to 296. Attendance rates varied from 2.7% to 90.1%. Taking a quality improvement approach, the study team brainstormed factors potentially related to variations in workshop attendance, constructed several division- and institution-level variables, and assessed the significance of factors on workshop attendance with hierarchical linear models. RESULTS: The following were positively associated with workshop attendance rate: the division head attended the workshop, the BRIM principal investigator gave Medical Grand Rounds, and the percentage of local workshop presenters who completed training. Workshop attendance rates fell when departments identified more than five on-site study leaders. CONCLUSIONS: Factors associated with higher workshop attendance may have increased the perceived status and value of attending the workshop, leading faculty to choose the workshop over other competing demands. For future investigators studying educational interventions that require participation of faculty in clinical departments at multiple sites, this work offers several valuable lessons.

Improving Department Climate Through Bias Literacy: One College's Experience.

Many institutions of higher education are investing in "implicit bias training" as a mechanism to improve diversity and inclusion on their campuses. In this study, we describe an effort to implement this training in the form of a 3-hour workshop delivered to faculty members in the College of Engineering at the University of Wisconsin-Madison. Evaluation form data collected immediately post-workshop, and in-person interviews and survey data collected 6-12 months post-workshop, were used to measure the effectiveness of the intervention. These data show that faculty awareness of implicit bias in their workplace environments increased significantly, although individual motivation and self-efficacy to act without bias, and self-reported bias-reduction actions, did not increase. At the same time, we found evidence of improved department climates and bias-reduction actions at the department level, which increase our confidence that the workshops were having a positive impact. Importantly, women and faculty of color in the College did not report increases in negative behavior after the workshop, and reported that their departments were engaging in explicit discussions of potential biases in departmental processes more often. These findings support the continued implementation of the "Breaking the Bias Habit®" workshops along with measurement of their success.

The Effects of Doctoral Teaching Development on Early-Career STEM Scholars' College Teaching Self-efficacy.

To help prepare future faculty in science, technology, engineering, and mathematics (STEM) to teach undergraduates, more research universities are offering teaching development (TD) programs to doctoral students who aspire to academic careers. Using social cognitive career theory, we examine the effects of TD programs on early-career STEM scholars' sense of self-efficacy as postsecondary teachers. In 2011, a survey questionnaire was administered to 2156 people who in 2009 were doctoral students in STEM departments at three U.S. research universities; 1445 responded (67%). Regression analysis revealed positive relationships between TD participation and participants' college teaching self-efficacy and positive interaction effects for women. These findings may be used to improve the quality and quantity of TD offerings and help them gain wider acceptance.

Write More Articles, Get More Grants: The Impact of Department Climate on Faculty Research Productivity.

BACKGROUND: Many studies find that female faculty in academic medicine, science, and engineering experience adverse workplace climates. This study longitudinally investigates whether department climate is associated with future research productivity and whether the associations are stronger for female than male faculty. METHOD: Two waves of a faculty climate survey, institutional grant records, and publication records were collected for 789 faculties in academic medicine, science, and engineering at the University of Wisconsin-Madison between 2000 and 2010. Research productivity was measured as Number of Publications and Number of Grants awarded, and department climate was measured with scales for professional interactions, department decision-making practices, climate for underrepresented groups, and work/life balance. Ordinary least squares and negative binomial regression methods were used to assess gender differences in productivity, influences of department climate on productivity, and gender differences in effects of climate on productivity. RESULTS: Female faculty published fewer articles and were awarded fewer grants in the baseline period, but their productivity did not differ from male faculty on these measures in subsequent years. Number of Publications was positively affected by professional interactions, but negatively affected by positive work/life balance. Number of Grants awarded was positively affected by climate for underrepresented groups. These main effects did not differ by gender; however, some three-way interactions illuminated how different aspects of department climate affected productivity differently for men and women in specific situations. CONCLUSIONS: In perhaps the first study to assess the longitudinal impact of department climate on faculty research productivity, positive department climate is associated with significantly greater productivity for all faculty-women and men. However, some positive aspects of climate (specifically, work/life balance) may be associated with lower productivity for some female faculty at specific career periods. These findings suggest that departments that wish to increase grants and publications would be wise to foster a positive workplace climate.

Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers' Critiques.

BACKGROUND: Women are less successful than men in renewing R01 grants from the National Institutes of Health. Continuing to probe text mining as a tool to identify gender bias in peer review, we used algorithmic text mining and qualitative analysis to examine a sample of critiques from men's and women's R01 renewal applications previously analyzed by counting and comparing word categories. METHODS: We analyzed 241 critiques from 79 Summary Statements for 51 R01 renewals awarded to 45 investigators (64% male, 89% white, 80% PhD) at the University of Wisconsin-Madison between 2010 and 2014. We used latent Dirichlet allocation to discover evaluative "topics" (i.e., words that co-occur with high probability). We then qualitatively examined the context in which evaluative words occurred for male and female investigators. We also examined sex differences in assigned scores controlling for investigator productivity. RESULTS: Text analysis results showed that male investigators were described as "leaders" and "pioneers" in their "fields," with "highly innovative" and "highly significant research." By comparison, female investigators were characterized as having "expertise" and working in "excellent" environments. Applications from men received significantly better priority, approach, and significance scores, which could not be accounted for by differences in productivity. CONCLUSIONS: Results confirm our previous analyses suggesting that gender stereotypes operate in R01 grant peer review. Reviewers may more easily view male than female investigators as scientific leaders with significant and innovative research, and score their applications more competitively. Such implicit bias may contribute to sex differences in award rates for R01 renewals.

Analysis of National Institutes of Health R01 Application Critiques, Impact, and Criteria Scores: Does the Sex of the Principal Investigator Make a Difference?

PURPOSE: Prior text analysis of R01 critiques suggested that female applicants may be disadvantaged in National Institutes of Health (NIH) peer review, particularly for renewals. NIH altered its review format in 2009. The authors examined R01 critiques and scoring in the new format for differences due to principal investigator (PI) sex. METHOD: The authors analyzed 739 critiques-268 from 88 unfunded and 471 from 153 funded applications for grants awarded to 125 PIs (76 males, 49 females) at the University of Wisconsin-Madison between 2010 and 2014. The authors used seven word categories for text analysis: ability, achievement, agentic, negative evaluation, positive evaluation, research, and standout adjectives. The authors used regression models to compare priority and criteria scores, and results from text analysis for differences due to PI sex and whether the application was for a new (Type 1) or renewal (Type 2) R01. RESULTS: Approach scores predicted priority scores for all PIs' applications (P < .001), but scores and critiques differed significantly for male and female PIs' Type 2 applications. Reviewers assigned significantly worse priority, approach, and significance scores to female than male PIs' Type 2 applications, despite using standout adjectives (e.g., "outstanding," "excellent") and making references to ability in more critiques (P < .05 for all comparisons). CONCLUSIONS: The authors' analyses suggest that subtle gender bias may continue to operate in the post-2009 NIH review format in ways that could lead reviewers to implicitly hold male and female applicants to different standards of evaluation, particularly for R01 renewals.