Effect of the ASCENT Intervention to Increase Knowledge of Kidney Allocation Policy Changes Among Dialysis Providers.

INTRODUCTION: The Allocation System Changes for Equity in Kidney Transplantation (ASCENT) trial was a cluster-randomized pragmatic, effectiveness-implementation study designed to test whether a multicomponent educational intervention targeting leadership, clinic staff, and patients in dialysis facilities improved knowledge and awareness of the 2014 Kidney Allocation System (KAS) change. METHODS: Participants included 690 dialysis facility medical directors, nephrologists, social workers, and other staff within 655 US dialysis facilities, with 51% (n = 334) in the intervention group and 49% (n = 321) in the control group. Intervention activities included a webinar targeting medical directors and facility staff, an approximately 10-minute educational video targeting dialysis staff, an approximately 10-minute educational video targeting patients, and a facility-specific audit and feedback report of transplant performance. The control group received a standard United Network for Organ Sharing brochure. Provider knowledge was a secondary outcome of the ASCENT trial and the primary outcome of this study; knowledge was assessed as a cumulative score on a 5-point Likert scale (higher score = greater knowledge). Intention-to-treat analysis was used. RESULTS: At baseline, nonintervention providers had a higher mean knowledge score (mean ± SD, 2.45 ± 1.43) than intervention providers (mean ± SD, 2.31 ± 1.46). After 3 months, the average knowledge score was slightly higher in the intervention (mean ± SD, 3.14 ± 1.28) versus nonintervention providers (mean ± SD, 3.07 ± 1.24), and the estimated mean difference in knowledge scores between the groups at follow-up minus the mean difference at baseline was 0.25 (95% confidence interval [CI], 0.11-0.48; P = 0.039). The effect size (0.41) was low to moderate. CONCLUSION: Dialysis facility provider education could help extend the impact of a national policy change in organ allocation.

Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers' Critiques.

BACKGROUND: Women are less successful than men in renewing R01 grants from the National Institutes of Health. Continuing to probe text mining as a tool to identify gender bias in peer review, we used algorithmic text mining and qualitative analysis to examine a sample of critiques from men's and women's R01 renewal applications previously analyzed by counting and comparing word categories. METHODS: We analyzed 241 critiques from 79 Summary Statements for 51 R01 renewals awarded to 45 investigators (64% male, 89% white, 80% PhD) at the University of Wisconsin-Madison between 2010 and 2014. We used latent Dirichlet allocation to discover evaluative "topics" (i.e., words that co-occur with high probability). We then qualitatively examined the context in which evaluative words occurred for male and female investigators. We also examined sex differences in assigned scores controlling for investigator productivity. RESULTS: Text analysis results showed that male investigators were described as "leaders" and "pioneers" in their "fields," with "highly innovative" and "highly significant research." By comparison, female investigators were characterized as having "expertise" and working in "excellent" environments. Applications from men received significantly better priority, approach, and significance scores, which could not be accounted for by differences in productivity. CONCLUSIONS: Results confirm our previous analyses suggesting that gender stereotypes operate in R01 grant peer review. Reviewers may more easily view male than female investigators as scientific leaders with significant and innovative research, and score their applications more competitively. Such implicit bias may contribute to sex differences in award rates for R01 renewals.

A quantitative linguistic analysis of National Institutes of Health R01 application critiques from investigators at one institution.

PURPOSE: Career advancement in academic medicine often hinges on the ability to garner research funds. The National Institutes of Health's (NIH's) R01 award is the "gold standard" of an independent research program. Studies show inconsistencies in R01 reviewers' scoring and in award outcomes for certain applicant groups. Consistent with the NIH recommendation to examine potential bias in R01 peer review, the authors performed a text analysis of R01 reviewers' critiques. METHOD: The authors collected 454 critiques (262 from 91 unfunded and 192 from 67 funded applications) from 67 of 76 (88%) R01 investigators at the University of Wisconsin-Madison with initially unfunded applications subsequently funded between December 2007 and May 2009. To analyze critiques, the authors developed positive and negative grant application evaluation word categories and selected five existing categories relevant to grant review. They analyzed results with linear mixed-effects models for differences due to applicant and application characteristics. RESULTS: Critiques of funded applications contained more positive descriptors and superlatives and fewer negative evaluation words than critiques of unfunded applications. Experienced investigators' critiques contained more references to competence. Critiques showed differences due to applicant sex despite similar application scores or funding outcomes: more praise for applications from female investigators, greater reference to competence/ability for funded applications from female experienced investigators, and more negative evaluation words for applications from male investigators (all P<.05). CONCLUSIONS: Results suggest that text analysis is a promising tool for assessing consistency in R01 reviewers' judgments, and gender stereotypes may operate in R01 review.