Unique Review Criteria and Patient and Stakeholder Reviewers: Analysis of PCORI's Approach to Research Funding.

OBJECTIVE: The Patient-Centered Outcomes Research Institute (PCORI) uses a unique approach to Merit Review that includes patients and stakeholders as reviewers with scientists, and includes unique review criteria (patient-centeredness and active engagement of end users in the research). This study assessed the extent to which different reviewer types influence review scores and funding outcomes, the emphasis placed on technical merit compared to other criteria by a multistakeholder panel, and the impact of the in-person discussion on agreement among different reviewer types. METHODS: Cross-sectional analysis of administrative data from PCORI online and in-person Merit Review (N = 1312 applications from the five funding cycles from November 2013 to August 2015). Linear and logistic regression models were used to analyze the data. RESULTS: For all reviewer types, final review scores were associated with at least one review criterion score from each of the three reviewer types. The strongest predictor of final overall scores for all reviewer types was scientists' prediscussion ratings of technical merit. All reviewers' prediscussion ratings of the potential to improve health care and outcomes, and scientists' ratings of technical merit and patient-centeredness, were associated with funding success. For each reviewer type, overall impact scores from the online scoring were changed on at least half of the applications at the in-person panel discussion. Score agreement across reviewer types was greater after panel discussion. CONCLUSIONS: Scientist, patient, and stakeholder views all contribute to PCORI Merit Review of applications for research funding. Technical merit is critical to funding success but patient and stakeholder ratings of other criteria also influence application disposition.

Researchers, Patients, and Stakeholders Evaluating Comparative-Effectiveness Research: A Mixed-Methods Study of the PCORI Reviewer Experience.

OBJECTIVES: The Patient-Centered Outcomes Research Institute (PCORI) includes patients and stakeholders alongside scientists in reviewing research applications using unique review criteria including patient-centeredness and patient and/or stakeholder engagement. To support extension of this unique collaborative model to other funders, information from the reviewers on the review process is needed to understand how scientists and nonscientists evaluate research proposals together. Thus, this study aimed to describe reviewers' perspectives of the interactions during the in-person review panel; to examine the value and challenges of including scientists, patients, and stakeholders together; and to understand the perceived importance of PCORI's review criteria. METHODS: This study utilized anonymous, cross-sectional surveys (N = 925 respondents from 5 funding cycles: 470 scientists, 217 patients, 238 stakeholders; survey completion rates by cycle: 70-89%) and group interviews (N = 18). RESULTS: Reviewers of all types describe PCORI Merit Review as respectful, balanced, and one of reciprocal influence among different reviewer types. Reviewers indicate strong support and value of input from all reviewer types, receptivity to input from others, and the panel chair's incorporation of all views. Patients and stakeholders provide real-world perspectives on importance to patients, research partnership plans, and study feasibility. Challenges included concerns about a lack of technical expertise of patient/stakeholder reviewers and about scientists dominating conversations. The most important criterion for assigning final review scores was technical merit-either alone or in conjunction with patient-centeredness or patient/ stakeholder engagement. CONCLUSIONS: PCORI Merit Reviewers' self-reports indicate that the perspectives of different reviewer types are influential in panel discussions and Merit Review outcomes.

Commentary on "Health economics and the value of therapy in Alzheimer's disease." Report from the Alzheimer's Association Research Roundtable on patient-reported outcomes and dementia research.

The release of the Food and Drug Administration's draft guidance on the patient-reported outcomes (PROs) measurement in February 2006 has specific implication for outcomes research in dementia. Advances in treatments for probable Alzheimer's disease present unique challenges to the measurement of clinical outcomes. Standards adapted for PROs measurement can be applied to provide an alternative perspective on efficacy, and to access a more comprehensive range of outcomes than is currently assessed in dementia trials. The draft guidance provides only limited direction with regard to proxy measurements, but the principles of PROs assessment should be applied to proxy respondent measures. Taking a broad view of outcome measurements can offset limitations related to utility assessment and the use of data in pharmacoeconomic evaluations of dementia treatments. Measurements that formally encompass patients and caregivers can enhance our understanding of the disorder, and improve conclusions about treatment effectiveness.

Health researcher views on comparative effectiveness research and research engagement.

AIM: To understand researcher capability for and interest in patient-centered comparative effectiveness research (PC-CER), particularly related to engaging with patients/caregivers. MATERIALS & METHODS: Web-based survey of 508 health researchers recruited via professional health research organizations. RESULTS: Most respondents (94%) were familiar with CER and many (69%) reported having previously conducting some form of CER. Most respondents were familiar with (81%) and interested in (87%) partnering with patients and/or caregivers in research. Resources to assist in training, coordination of partners, guidance in apply for funding and improved infrastructure were commonly cited factors that would help researchers conduct PC-CER. CONCLUSION: There is a significant opportunity for researchers to engage patients and caregivers as partners in CER. Researchers recognize the need for additional training and expertise to leverage those opportunities.

Patient, caregiver and clinician use of comparative effectiveness research findings in care decisions: results from a national study.

AIM: To assess awareness, use and attitudes concerning comparative effectiveness research (CER) findings. MATERIALS & METHODS: Online surveys of patients and caregivers managing rare (n = 560 patients, n = 609 caregivers) or chronic conditions (n = 762 patients, n = 776 caregivers), and practicing clinicians (n = 638). RESULTS: Less than half of patients and caregivers reported exposure to any type of CER findings in the past 12 months. Of those, over half identify healthcare professionals as the information source, yet only 10% of clinicians indicated they were 'very familiar' with CER. Clinicians were concerned about the time required to find relevant evidence and the appropriateness of comparisons in available health research. CONCLUSION: Clinicians, patients and caregivers indicate unmet need for incorporating CER research findings into clinical decision-making.

Patient, caregiver and clinician views on engagement in comparative effectiveness research.

AIM: Describe patient, caregiver and clinician views toward engagement as partners in health research. MATERIALS & METHODS: Online surveys of patients and caregivers managing rare (n = 560 patients, n = 609 caregivers) or chronic conditions (n = 762 patients, n = 776 caregivers) and practicing clinicians (n = 638). RESULTS: Over half of respondents were unfamiliar with the concept of partnering with researchers but most expressed interest in working in a research partnership. Potential facilitators endorsed were ensuring research is meaningful, applying results in an understandable way, and sharing results. Lack of time is a potential barrier. Clinicians were most interested in helping researchers decide on intervention comparisons and identifying implications for clinical practice. CONCLUSION: Patients, caregivers and clinicians are interested in research roles that emphasize usefulness and understandability of research.

Depression and health-related quality of life for low-income African-American women in the U.S.

The health-related quality of life (HRQL) impact of depression for low-income young African-American women has not been quantified. Baseline scores on a generic HRQL measure, the SF-36, from a randomized controlled trial of depression treatments were used as a basis for describing the HRQL of depressed (n = 124) and non-depressed (n = 44) low-income African-American young women. Results were compared to U.S. normative values for well adults and for depressed adults. Relationship between SF-36 scores and demographic variables were examined, and SF-36 scores were compared for those with depression only and those with comorbid anxiety. SF-36 scores were lower for all eight subscales relative to the U.S. norms (p < 0.05) and lower on two subscales relative to a general U.S. depressed sample (p < 0.05). Higher age and higher number of children was associated with poorer scores. Comorbid anxiety was present in 66% of the depressed sample; all SF-36 scores for the sample with comorbid anxiety were significantly lower than scores for subjects with depression alone. Results indicate the substantial HRQL impact of depression among low-income young African-American women with depression.