Peer review reduces spin in PCORI research reports.

BACKGROUND: The Patient-Centered Outcomes Research Institute (PCORI) is obligated to peer review and to post publicly "Final Research Reports" of all funded projects. PCORI peer review emphasizes adherence to PCORI's Methodology Standards and principles of ethical scientific communication. During the peer review process, reviewers and editors seek to ensure that results are presented objectively and interpreted appropriately, e.g., free of spin. METHODS: Two independent raters assessed PCORI peer review feedback sent to authors. We calculated the proportion of reports in which spin was identified during peer review, and the types of spin identified. We included reports submitted by April 2018 with at least one associated journal article. The same raters then assessed whether authors addressed reviewers' comments about spin. The raters also assessed whether spin identified during PCORI peer review was present in related journal articles. RESULTS: We included 64 PCORI-funded projects. Peer reviewers or editors identified spin in 55/64 (86%) submitted research reports. Types of spin included reporting bias (46/55; 84%), inappropriate interpretation (40/55; 73%), inappropriate extrapolation of results (15/55; 27%), and inappropriate attribution of causality (5/55; 9%). Authors addressed comments about spin related to 47/55 (85%) of the reports. Of 110 associated journal articles, PCORI comments about spin were potentially applicable to 44/110 (40%) articles, of which 27/44 (61%) contained the same spin that was identified in the PCORI research report. The proportion of articles with spin was similar for articles accepted before and after PCORI peer review (63% vs 58%). DISCUSSION: Just as spin is common in journal articles and press releases, we found that most reports submitted to PCORI included spin. While most spin was mitigated during the funder's peer review process, we found no evidence that review of PCORI reports influenced spin in journal articles. Funders could explore interventions aimed at reducing spin in published articles of studies they support.

Training patients to review scientific reports for the Patient-Centered Outcomes Research Institute: an observational study.

OBJECTIVE: The peer review of completed Patient-Centered Outcomes Research Institute (PCORI) funded research includes reviews from patient reviewers (patients, caregivers, and patient advocates). Very little is known about how best to support these reviewers in writing helpful comments from a patient-centred perspective. This study aimed to evaluate the effect of a new training in peer review for patient reviewers. DESIGN: Observational study. SETTING: Online. PARTICIPANTS: Adults registered in the PCORI Reviewer Database as a patient stakeholder. INTERVENTION: A new online training in peer review. MAIN OUTCOME MEASURES: Changes in reviewers' knowledge and skills; change in self-efficacy and attitudes, satisfaction with the training and perceived benefits and relevance of the training. RESULTS: Before-after training survey data were analysed for 37 (29.4% of 126) patient reviewers invited to participate in an online training as part of a quality improvement effort or as part of a PCORI peer review. The reviewers improved their answers to the knowledge questions (p<0.001, median number of answers improved 4 (95% CI 3 to 5), large effect size (ES) Cohen's w=0.94) after the training, particularly in the questions targeting the specifics of PCORI peer review. Reviewers improved their skills in recognising helpful review comments, but those without peer-review background improved proportionally more (p=0.008, median number of answers improved 2 (95% CI 1 to 3), medium ES w=0.60). The use of training modestly increased reviewers' confidence in completing a high-quality peer review (p=0.005, mean increase in 5-point Likert rating 0.51 (95% CI 0.17 to 0.86), small-to-medium ES Cliff's delta=0.32) and their excitement about providing a review slightly increased (p=0.019, mean increase in 5-point Likert rating 0.35 (95% CI 0.03 to 0.68), small ES delta=0.19). All reviewers were satisfied with the training and would recommend it to other reviewers. CONCLUSIONS: Training improved knowledge, skills and self-efficacy and slightly increased enthusiasm for completing a PCORI peer review.

A shortened psychophysical task to quantify irritability: the Reactive Irritability Scale II (RIS-II).

The Reactive Irritability Scale (RIS) uses magnitude estimation to measure reactive irritability. Respondents rate target sounds in comparison to a neutral reference sound. The RIS proved more sensitive than self-report measures to detect irritability associated with withdrawal from cigarette smoking and with exposure to a stressful environment, but was too long (30 min) for routine use. We report here on a shortened version (13 min)--RIS-II. The RIS-II exhibited robust test-retest reliability and correlated strongly with the original RIS (Study 1). In Study 2, the RIS-II was administered to subjects who experienced psychological stress and then were exposed to progressive relaxation, music, cognitive tasks, or silence; the Progressive Relaxation group was the most irritable. In Study 3, the RIS-II was administered to chronic pain patients. Pain patients found the sounds less irritating than did controls with the exception of the reference sound; repeated presentation of the reference sound increased irritability. These studies indicate that the RIS-II is a reliable instrument that may have utility for the measurement of irritability in laboratory and clinical settings. In addition, these studies indicate that the RIS-II is understandable by individuals of different ages who are from educationally- and culturally-diverse backgrounds and individuals who are healthy as well as individuals suffering from chronic medical conditions who are on multiple medications.