An experimental test of the effects of redacting grant applicant identifiers on peer review outcomes.

Background: Blinding reviewers to applicant identity has been proposed to reduce bias in peer review. Methods: This experimental test used 1200 NIH grant applications, 400 from Black investigators, 400 matched applications from White investigators, and 400 randomly selected applications from White investigators. Applications were reviewed by mail in standard and redacted formats. Results: Redaction reduced, but did not eliminate, reviewers' ability to correctly guess features of identity. The primary, preregistered analysis hypothesized a differential effect of redaction according to investigator race in the matched applications. A set of secondary analyses (not preregistered) used the randomly selected applications from White scientists and tested the same interaction. Both analyses revealed similar effects: Standard format applications from White investigators scored better than those from Black investigators. Redaction cut the size of the difference by about half (e.g. from a Cohen's d of 0.20-0.10 in matched applications); redaction caused applications from White scientists to score worse but had no effect on scores for Black applications. Conclusions: Grant-writing considerations and halo effects are discussed as competing explanations for this pattern. The findings support further evaluation of peer review models that diminish the influence of applicant identity. Funding: Funding was provided by the NIH.

NIH peer review: Criterion scores completely account for racial disparities in overall impact scores.

Previous research has found that funding disparities are driven by applications' final impact scores and that only a portion of the black/white funding gap can be explained by bibliometrics and topic choice. Using National Institutes of Health R01 applications for council years 2014-2016, we examine assigned reviewers' preliminary overall impact and criterion scores to evaluate whether racial disparities in impact scores can be explained by application and applicant characteristics. We hypothesize that differences in commensuration-the process of combining criterion scores into overall impact scores-disadvantage black applicants. Using multilevel models and matching on key variables including career stage, gender, and area of science, we find little evidence for racial disparities emerging in the process of combining preliminary criterion scores into preliminary overall impact scores. Instead, preliminary criterion scores fully account for racial disparities-yet do not explain all of the variability-in preliminary overall impact scores.

Reviewing Peer Review at the NIH.

Recent reports suggest that peer reviews of National Institutes of Health grant applications are at best imprecise predictors of research projects' scientific impact. But these findings may not mean that peer review is failing.

Examining the Predictive Validity of NIH Peer Review Scores.

The predictive validity of peer review at the National Institutes of Health (NIH) has not yet been demonstrated empirically. It might be assumed that the most efficient and expedient test of the predictive validity of NIH peer review would be an examination of the correlation between percentile scores from peer review and bibliometric indices of the publications produced from funded projects. The present study used a large dataset to examine the rationale for such a study, to determine if it would satisfy the requirements for a test of predictive validity. The results show significant restriction of range in the applications selected for funding. Furthermore, those few applications that are funded with slightly worse peer review scores are not selected at random or representative of other applications in the same range. The funding institutes also negotiate with applicants to address issues identified during peer review. Therefore, the peer review scores assigned to the submitted applications, especially for those few funded applications with slightly worse peer review scores, do not reflect the changed and improved projects that are eventually funded. In addition, citation metrics by themselves are not valid or appropriate measures of scientific impact. The use of bibliometric indices on their own to measure scientific impact would likely increase the inefficiencies and problems with replicability already largely attributed to the current over-emphasis on bibliometric indices. Therefore, retrospective analyses of the correlation between percentile scores from peer review and bibliometric indices of the publications resulting from funded grant applications are not valid tests of the predictive validity of peer review at the NIH.

Large-scale visuomotor integration in the cerebral cortex.

Efficient visuomotor behavior depends on integrated processing by the visual and motor systems of the cerebral cortex. Yet, many previous cortical neurophysiology studies have examined the visual and motor modalities in isolation, largely ignoring questions of large-scale cross-modal integration. To address this issue, we analyzed event-related local field potentials simultaneously recorded from multiple visual, motor, and executive cortical sites in monkeys performing a visuomotor pattern discrimination task. The timing and cortical location of four aspects of event-related activities were examined: stimulus-evoked activation onset, stimulus-specific processing, stimulus category-specific processing, and response-specific processing. Activations appeared earliest in striate cortex and rapidly thereafter in other visual areas. Stimulus-specific processing began early in most visual cortical areas, some at activation onset. Early onset latencies were also observed in motor, premotor, and prefrontal areas, some as early as in striate cortex, but these early-activating frontal sites did not show early stimulus-specific processing. Response-specific processing began around 150 ms poststimulus in widespread cortical areas, suggesting that perceptual decision formation and response selection arose through concurrent processes of visual, motor, and executive areas. The occurrence of stimulus-specific and stimulus category-specific differences after the onset of response-specific processing suggests that sensory and motor stages of visuomotor processing overlapped in time.

Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality.

Previous studies have shown that synchronized beta frequency (14-30 Hz) oscillations in the primary motor cortex are involved in maintaining steady contractions of contralateral arm and hand muscles. However, little is known about the role of postcentral cortical areas in motor maintenance and their patterns of interaction with motor cortex. We investigated the functional relations of beta-synchronized neuronal assemblies in pre- and postcentral areas of two monkeys as they pressed a hand lever during the wait period of a visual discrimination task. By using power and coherence spectral analysis, we identified a beta-synchronized large-scale network linking pre- and postcentral areas. We then used Granger causality spectra to measure directional influences among recording sites. In both monkeys, strong Granger causal influences were observed from primary somatosensory cortex to both motor cortex and inferior posterior parietal cortex, with the latter area also exerting Granger causal influences on motor cortex. Granger causal influences from motor cortex to postcentral sites, however, were weak in one monkey and not observed in the other. These results are the first, to our knowledge, to demonstrate in awake monkeys that synchronized beta oscillations bind multiple sensorimotor areas into a large-scale network during motor maintenance behavior and carry Granger causal influences from primary somatosensory and inferior posterior parietal cortices to motor cortex.

Including ethnic minorities in mental health intervention research: a practical approach to a long-standing problem.

This paper examines a controversy that arose while developing a supplement to Mental Health: A Report of the Surgeon General that was focused on ethnic minority mental health. The controversy involved whether and how to make recommendations about ethnic minorities seeking mental health care. We found that few studies provided information on outcomes of mental health care for ethnic minorities. In this paper, we discuss outcomes of mental health care for ethnic minorities and how to proceed in developing an evidence base for understanding mental health care and minorities. We conclude that entering representative (based on population) numbers of ethnic minorities in efficacy trials is unlikely to produce useful information on outcomes of care because the numbers will be too small to produce reliable findings. We also conclude that while conducting randomized efficacy trials for all mental health interventions for each ethnic group would be impractical, innovative and theoretically informed studies that focus on specific cultural groups are needed to advance the knowledge base. We call for theory-driven research focused on mental health disparities that has the potential for understanding disparities and improving outcomes for ethnic minority populations.

Synchronized activity in prefrontal cortex during anticipation of visuomotor processing.

It is commonly presumed, though not well established, that the prefrontal cortex exerts top-down control of sensory processing. One aspect of this control is thought to be a facilitation of sensory pathways in anticipation of such processing. To investigate the possible involvement of prefrontal cortex in anticipatory top-down control, we studied the statistical relations between prefrontal activity, recorded while a macaque monkey waited for presentation of a visual stimulus, and subsequent sensory and motor events. Local field potentials were simultaneously recorded from prefrontal, motor, occipital and temporal cortical sites in the left cerebral hemisphere. Spectral power and coherence analysis revealed that during stimulus anticipation three of five prefrontal sites participated in a coherent oscillatory network synchronized in the beta-frequency range. Pre-stimulus network power and coherence were highly correlated with the amplitude and latency of early visual evoked potential components in visual cortical areas, and with response time. The results suggest that synchronized oscillatory networks in prefrontal cortex are involved in top-down anticipatory mechanisms that facilitate subsequent sensory processing in visual cortex. They further imply that stronger top-down control leads to larger and faster sensory responses, and a subsequently faster motor response.

Trial-to-trial variability of cortical evoked responses: implications for the analysis of functional connectivity.

OBJECTIVES: The time series of single trial cortical evoked potentials typically have a random appearance, and their trial-to-trial variability is commonly explained by a model in which random ongoing background noise activity is linearly combined with a stereotyped evoked response. In this paper, we demonstrate that more realistic models, incorporating amplitude and latency variability of the evoked response itself, can explain statistical properties of cortical potentials that have often been attributed to stimulus-related changes in functional connectivity or other intrinsic neural parameters. METHODS: Implications of trial-to-trial evoked potential variability for variance, power spectrum, and interdependence measures like cross-correlation and spectral coherence, are first derived analytically. These implications are then illustrated using model simulations and verified experimentally by the analysis of intracortical local field potentials recorded from monkeys performing a visual pattern discrimination task. To further investigate the effects of trial-to-trial variability on the aforementioned statistical measures, a Bayesian inference technique is used to separate single-trial evoked responses from the ongoing background activity. RESULTS: We show that, when the average event-related potential (AERP) is subtracted from single-trial local field potential time series, a stimulus phase-locked component remains in the residual time series, in stark contrast to the assumption of the common model that no such phase-locked component should exist. Two main consequences of this observation are demonstrated for statistical measures that are computed on the residual time series. First, even though the AERP has been subtracted, the power spectral density, computed as a function of time with a short sliding window, can nonetheless show signs of modulation by the AERP waveform. Second, if the residual time series of two channels co-vary, then their cross-correlation and spectral coherence time functions can also be modulated according to the shape of the AERP waveform. Bayesian estimation of single-trial evoked responses provides further proof that these time-dependent statistical changes are due to remnants of the evoked phase-locked component in the residual time series. CONCLUSIONS: Because trial-to-trial variability of the evoked response is commonly ignored as a contributing factor in evoked potential studies, stimulus-related modulations of power spectral density, cross-correlation, and spectral coherence measures is often attributed to dynamic changes of the connectivity within and among neural populations. This work demonstrates that trial-to-trial variability of the evoked response must be considered as a possible explanation of such modulation.