Grant Review Feedback: Appropriateness and Usefulness.

The primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56-60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents' feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.

The Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey.

Scientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. The American Institute of Biological Sciences (AIBS) developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBS's proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2-5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56-87% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

Negative life events and incident alcohol use disorders among ethnic minorities.

The association of negative life events (NLEs) with incident alcohol use disorders (AUDs) was examined among Blacks, Hispanics, and Whites in the second wave of the National Comorbidity Survey (NCS) among 3,679 participants without AUDs at the first wave. The number of past-year NLEs at NCS-2 was higher for Black than White participants, but the rates of incident AUDs did not differ by racial/ethnic group (14.2% among all participants). Past-year NLEs were associated with increased odds of incident AUDs for Whites and Hispanics but not Blacks. The implications of racial/ethnic differences in life events and AUDs are discussed.

Psychologists in Academic Administration: A Call to Action and Service.

Academic psychologists' backgrounds may prepare them for many aspects of academic administration such as: understanding and working with people; prioritizing others' needs and institutional needs; and managing projects and budgets, e.g., for research grants or training programs. Contemporary academic health centers also may provide opportunities for psychologists to serve in academic health administration. This article encourages psychologists to consider preparing for and seeking administrative and higher-level leadership roles. Six psychologists serving diverse administrative roles-from vice chairs in medical school departments to presidents of universities with academic health centers-reflected on: their paths to administration; their preparation for administrative roles; and the commonalities and differences between the work and skills sets of psychologist health service providers and the work and skill sets required for higher level administrative and leadership roles.

Neurocognitive complaints and functional status among patients with chronic fatigue syndrome and fibromyalgia.

PURPOSE: The purpose of this study was to conduct a longitudinal examination of cognitive complaints and functional status in patients with chronic fatigue syndrome (CFS) alone and those who also had fibromyalgia (CFS/FM). METHODS: A total of 93 patients from a tertiary care fatigue clinic were evaluated on four occasions, each 6 months apart. Each evaluation included a tender point assessment, and self-reported functional status and cognitive complaints. RESULTS: Patients with CFS/FM reported significantly worse physical functioning, more bodily pain, and more cognitive difficulties (visuo-perceptual ability and verbal memory) than patients with CFS alone. Over time, bodily pain decreased only for participants with CFS alone. Verbal memory problems were associated with more bodily pain for both patient groups, whereas visuo-perceptual problems were associated with worse functional status for patients with CFS alone. CONCLUSIONS: This study adds to the literature on functional status, longitudinal course, and cognitive difficulties among patients with CFS and those with CFS and FM. The results suggest that patients with CFS/FM are more disabled, have more cognitive complaints, and improve more slowly over time than patients with CFS alone. Specific cognitive difficulties are related to worse functional status, which supports the addition of cognitive difficulties to the FM case criteria.

The privilege and burden of peer review.

OBJECTIVES: Peer review is a core value and method of quality control in psychological research, academic psychology, and other disciplines, but little is known about the peer-reviewing behavior of ethnic minority reviewers in particular. The purpose of this study was to examine the self-identified ethnicity of those invited to peer review articles for 76 journals that utilized the American Psychological Association's Journals Back Office (JBO) system from 2003 to 2012. It was hypothesized that a modest increase in the ratio of requests for reviews from self-identified ethnic minority reviewers would be observed over time, that self-identified ethnic minority reviewers would be less likely to refuse a review request than those who do not self-identify as an ethnic minority, and finally that increases in reviewer burden would be evident in significant increases in declines to requests by all reviewers. METHOD: Reviewer requests and responses were examined among the 76 journals that used the JBO system over a 10-year period. RESULTS: Using hierarchical linear models, the percentage of review invitations extended to ethnic minorities was found to significantly increase over time: Initially, an estimated 8.34% of review requests were made to ethnic minority reviewers, and that percentage increased an average of 0.41% per year. CONCLUSIONS: Ethnic minority reviewers were significantly less likely to refuse a review request than ethnic majority reviewers. Results are discussed in terms of perceived pressure to demonstrate scholastic impact and the disproportionate service burden often borne by ethnic minority psychologists.

Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis.

BACKGROUND: Differential participation and success in grant applications may contribute to women's lesser representation in the sciences. This study's objective was to conduct a systematic review and meta-analysis to address the question of gender differences in grant award acceptance rates and reapplication award acceptance rates (potential bias in peer review outcomes) and other grant outcomes. METHODS: The review was registered on PROSPERO (CRD42021232153) and conducted in accordance with PRISMA 2020 standards. We searched Academic Search Complete, PubMed, and Web of Science for the timeframe 1 January 2005 to 31 December 2020, and forward and backward citations. Studies were included that reported data, by gender, on any of the following: grant applications or reapplications, awards, award amounts, award acceptance rates, or reapplication award acceptance rates. Studies that duplicated data reported in another study were excluded. Gender differences were investigated by meta-analyses and generalized linear mixed models. Doi plots and LFK indices were used to assess reporting bias. RESULTS: The searches identified 199 records, of which 13 were eligible. An additional 42 sources from forward and backward searches were eligible, for a total of 55 sources with data on one or more outcomes. The data from these studies ranged from 1975 to 2020: 49 sources were published papers and six were funders' reports (the latter were identified by forwards and backwards searches). Twenty-nine studies reported person-level data, 25 reported application-level data, and one study reported both: person-level data were used in analyses. Award acceptance rates were 1% higher for men, which was not significantly different from women (95% CI 3% more for men to 1% more for women, k = 36, n = 303,795 awards and 1,277,442 applications, I2 = 84%). Reapplication award acceptance rates were significantly higher for men (9%, 95% CI 18% to 1%, k = 7, n = 7319 applications and 3324 awards, I2 = 63%). Women received smaller award amounts (g = -2.28, 95% CI -4.92 to 0.36, k = 13, n = 212,935, I2 = 100%). CONCLUSIONS: The proportions of women that applied for grants, re-applied, accepted awards, and accepted awards after reapplication were less than the proportion of eligible women. However, the award acceptance rate was similar for women and men, implying no gender bias in this peer reviewed grant outcome. Women received smaller awards and fewer awards after re-applying, which may negatively affect continued scientific productivity. Greater transparency is needed to monitor and verify these data globally.

Depression trial results: A cross-sectional study of ClinicalTrials.gov.

Published clinical trials represent a subsample of the objective information needed to appraise treatments for depression. We characterize the extent of selective and delayed reporting in a systematic review (PROSPERO #CRD42020173606) of depression trial results registered on ClinicalTrials.gov. Inclusion criteria were studies registered on ClinicalTrials.gov with depression as the condition, had enrolled ages 18 and over, were completed between January 1, 2008 and May 1, 2019, and had posted results by February 1, 2022. Cox regression analyses of time to result posting from registration and from study completion included enrollment as a covariate. Among 442 protocols, median result posting occurred over two years after study completion and five years after registration. Among protocols with incomplete results, effect sizes (d or W) were calculated for 134 protocols. Median effect sizes for protocols with incomplete results were small (0.16, 95% CI 0.08, 0.21). For 28% of protocols, observed effects were contrary to the expected direction. Between-group effect size calculations were based on post-treatment data as pre-treatment data were inconsistently provided. Although drug and device trials in the U.S. are required to register on ClinicalTrials.gov, compliance is imperfect, and submissions are not peer reviewed. For depression treatment trials, long intervals between study completion and posting of results are common. Further, investigators often fail to report the results of statistical tests. Failure to post trial results in a timely manner and omission of statistical test reporting may lead to overestimates of treatment effects in systematic literature reviews.

Couples and COVID-19 vaccination: Frequency and reasons for discordance.

The purpose of this study was to determine the frequency of concordance and discordance in the COVID-19 vaccination status of intimate couples. Partners are well known to influence each other's health behaviors, but previous reports of couples' COVID-19 vaccination status were not found. Among 1305 respondents to an on-line survey who were living with their partners, couples' COVID-19 vaccination status was 15.63% discordant and 84.37% concordant. Different potentially modifiable reasons for being unvaccinated were emphasized by unvaccinated participants with vaccinated partners and by vaccinated participants with unvaccinated partners. This study provides an estimate of the proportion of couples with discordant COVID-19 vaccine status. The development of tailored, scalable interventions potentially involving the partner might increase the COVID-19 vaccine concordance rate.

Peer review: Risk and risk tolerance.

Peer review, commonly used in grant funding decisions, relies on scientists' ability to evaluate research proposals' quality. Such judgments are sometimes beyond reviewers' discriminatory power and could lead to a reliance on subjective biases, including preferences for lower risk, incremental projects. However, peer reviewers' risk tolerance has not been well studied. We conducted a cross-sectional experiment of peer reviewers' evaluations of mock primary reviewers' comments in which the level and sources of risks and weaknesses were manipulated. Here we show that proposal risks more strongly predicted reviewers' scores than proposal strengths based on mock proposal evaluations. Risk tolerance was not predictive of scores but reviewer scoring leniency was predictive of overall and criteria scores. The evaluation of risks dominates reviewers' evaluation of research proposals and is a source of inter-reviewer variability. These results suggest that reviewer scoring variability may be attributed to the interpretation of proposal risks, and could benefit from intervention to improve the reliability of reviews. Additionally, the valuation of risk drives proposal evaluations and may reduce the chances that risky, but highly impactful science, is supported.

Transparency of results reporting for depression treatment studies in ClinicalTrials.gov: a cross-sectional study.

OBJECTIVES: Depression affects an estimated 7% of the adult population at an estimated cost of over US$200 billion/year in the USA. Complete, transparent reporting of clinical trial data facilitates valid estimates of treatment efficacy. In the USA, ClinicalTrials.gov increases transparency through mandatory prospective trial registration and outcome reporting. We examined characteristics of the transparent reporting of depression treatment studies registered in ClinicalTrials.gov. DESIGN: Cross sectional. SETTING AND PARTICIPANTS: US-based studies identified in a search of ClinicalTrials.gov with depression as the condition, enrolling ages 18 and older, and completed between 1 January 2008 and 1 May 2019. INTERVENTIONS: All interventions were included. MAIN OUTCOMES AND MEASURES: The main outcome was whether any results were reported prior to 1 May 2020. Data were extracted regarding inclusion and exclusion criteria, publications related to the study and specification of hypotheses. RESULTS: 725 studies involving 156 634 patients met inclusion criteria. 416 (57.4%) of the studies posted some results. However, statistical test results were not included in 230 studies (55.3%). Most studies had data that could have been analysed and reported. Compared with studies without results, studies with any results were more likely to have hypotheses, include drug treatment conditions, and to have publications related to the study. CONCLUSIONS: Required study registration does not always result in transparent outcome reporting. Better compliance with mandated reporting and improved reporting standards would facilitate a more comprehensive representation of depression treatment research.

Treatment for Borderline Personality Disorder and Secondary Effects on Somatization.

Borderline personality disorder (BPD) is often accompanied by other diagnoses. Some comorbidities have received a good deal of attention, but others, including somatization, have not. As effective treatments for BPD are identified, the secondary effects of these treatments on BPD comorbidities are of clinical relevance. Secondary data were used from a randomized clinical trial of dialectical behavior therapy (DBT) and community treatment by experts among 101 women with BPD in order to characterize change in somatization with treatment for BPD, and to examine emotional avoidance as a mediator. Somatization decreased significantly over time, which did not differ by treatment condition. DBT had an indirect effect on somatization through less emotional avoidance. These results suggest that the comorbid presence of significant somatization should be evaluated among those with BPD, especially in the context of emotional avoidance; treatments could be enhanced by addressing emotion regulation skills.

Stress and alcohol use among soldiers assessed at mobilization and demobilization.

Excessive alcohol use may have negative consequences, including less force readiness among military personnel. The identification of variables associated with alcohol use may inform early intervention efforts to decrease negative consequences. This longitudinal prospective study examined the associations of demographic and stress variables with alcohol use among 876 soldiers that were mobilized and demobilized through an Army installation during a 9-month period in 2003. Participants reported a moderate level of general stress at mobilization and demobilization, and a minority reported significant combat stress. Alcohol use in the 2 weeks before the demobilization evaluation was associated with younger age, nonactive duty status before mobilization, and more general stress. Male gender was associated with more drinks per drinking day. The results suggest that younger, nonactive duty male personnel experiencing stress may be a group at risk for increased drinking after deployment and for whom intervention may be helpful.

Longitudinal outcomes associated with significant other responses to chronic fatigue and pain.

This study investigated significant others' behavior associated with fatigue, pain, and mental health outcomes among 68 individuals with chronic fatigue (43% also had fibromyalgia) over 18 months. More negative significant others' responses were associated with more pain, poorer physical and mental health, and more fatigue-related symptoms over time. More fibromyalgia tender points covaried with more solicitous significant others' responses over time. Better mental health covaried with more distracting significant others' responses over time. The results are discussed in terms of theoretical models of the role of perceived significant others' responses on patient outcomes and recommendations for future research.

Grant reviewer perceptions of the quality, effectiveness, and influence of panel discussion.

BACKGROUND: Funding agencies have long used panel discussion in the peer review of research grant proposals as a way to utilize a set of expertise and perspectives in making funding decisions. Little research has examined the quality of panel discussions and how effectively they are facilitated. METHODS: Here, we present a mixed-method analysis of data from a survey of reviewers focused on their perceptions of the quality, effectiveness, and influence of panel discussion from their last peer review experience. RESULTS: Reviewers indicated that panel discussions were viewed favorably in terms of participation, clarifying differing opinions, informing unassigned reviewers, and chair facilitation. However, some reviewers mentioned issues with panel discussions, including an uneven focus, limited participation from unassigned reviewers, and short discussion times. Most reviewers felt the discussions affected the review outcome, helped in choosing the best science, and were generally fair and balanced. However, those who felt the discussion did not affect the outcome were also more likely to evaluate panel communication negatively, and several reviewers mentioned potential sources of bias related to the discussion. While respondents strongly acknowledged the importance of the chair in ensuring appropriate facilitation of the discussion to influence scoring and to limit the influence of potential sources of bias from the discussion on scoring, nearly a third of respondents did not find the chair of their most recent panel to have performed these roles effectively. CONCLUSIONS: It is likely that improving chair training in the management of discussion as well as creating review procedures that are informed by the science of leadership and team communication would improve review processes and proposal review reliability.

Illness behaviors in patients with unexplained chronic fatigue are associated with significant other responses.

Chronic fatigue syndrome (CFS) and unexplained chronic fatigue (CF) are characterized by compromised functional status and physical disability. Prior research on chronic pain has suggested that social factors may contribute to disability. This study examined the relationship between significant other responses and patient outcomes in patients with unexplained CF. Questionnaire data were collected from 117 patients on physical function, fatigue, pain, illness behaviors and responses of significant others to them, and depression. Ninety-four SOs reported their perceptions of patient illness behavior and their responses. Thirty-seven of these dyads also completed a series of household activities while being videotaped. Dyadic interactions were coded and analyzed. Both reported and observed solicitous responses by the significant other were associated with reported and observed patient illness behavior. Negative responses to patient illness behavior by significant others were associated with higher levels of patient depressive symptoms. The findings provide support for the role of operant behavioral factors in the context of chronic fatigue. They also suggest that further research on the relationship between dysfunction and significant other responses in patients with CFS or CF appears warranted and may have implications for treatment development.

Change in airflow among patients with asthma discussing relationship problems with their partners.

This study examined the covariation of negative emotions with airflow among 48 persons with asthma and their partners as they discussed relationship problems. Measures included self-reported questionnaires, airflow and behavior coded from videotaped discussions. Significantly increased self-reported hostility and statistically but not clinically significant declines in airflow were found post- versus pre-discussion. Self-reported responses to asthma symptoms of more anger and less loneliness predicted lower post-discussion airflow after accounting for pre-discussion airflow. The use of effort-independent measures of airflow and autonomic nervous system monitoring may inform future research regarding the physiological mechanisms through which mood and behavior affect airflow.

Moderators of Outcome in Problem-Solving Therapy for Depression in Primary Care.

OBJECTIVE: The purpose of this study was to conduct a secondary analysis of data from trials of problem-solving therapy in primary care (PST-PC) to examine differential outcomes by gender or race-ethnicity. METHODS: The participants were 352 patients with depression treated with PST-PC in multiple primary care sites in the United States. RESULTS: Women's depressive symptoms improved more over time than men's. Hopkins Symptom Checklist depression scale scores decreased significantly (t=-10.12, df=692, p<0.001) (estimate=-0.02, 95% confidence interval=-0.03, -0.02). Although patients from racial-ethnic minority groups had more depressive symptoms over time than patients from nonminority groups, there was no evidence of differential change by racial-ethnic group. CONCLUSIONS: These data add to the literature supporting the usefulness of PST-PC without evidence of differential effects for patients from racial-ethnic minority or nonminority groups.

The association of major life events with chronic fatigue.

OBJECTIVE: Major life events have been associated with the onset of unexplained chronic fatigue (CF) and with variability in illness course. The purpose of this study was to characterize the associations of major life events with illness status over time. METHOD: Sentinel measures of clinical status were collected four times over 18 months in a cohort of 99 persons with CF; participants also were interviewed regarding major life events, which were independently rated by observers. RESULTS: Over time, more major life events attributed to CF predicted a worse clinical course, suggesting that illness-related major life events rather than stressful life events in general are associated with worse clinical status for persons with CF. CONCLUSION: This study adds to the literature regarding specific types of life events' relationship to illness course, which has implications for the management of CF.