Public involvement in health outcomes research: lessons learnt from the development of the recovering quality of life (ReQoL) measures.

BACKGROUND: To provide a model for Public involvement (PI) in instrument development and other research based on lessons learnt in the co-production of a recently developed mental health patient reported outcome measure called Recovering Quality of Life (ReQoL). While service users contributed to the project as research participants, this paper focuses on the role of expert service users as research partners, hence referred to as expert service users or PI. METHODS: At every stage of the development, service users influenced the design, content and face validity of the measure, collaborating with other researchers, clinicians and stakeholders who were central to this research. Expert service users were integral to the Scientific Group which was the main decision-making body, and also provided advice through the Expert Service User Group. RESULTS: During the theme and item generation phase (stage 1) expert service users affirmed the appropriateness of the seven domains of the Patient Reported Outcome Measure (activity, hope, belonging and relationships, self-perception, wellbeing, autonomy, and physical health). Expert service users added an extra 58 items to the pool of 180 items and commented on the results from the face and content validity testing (stage 2) of a refined pool of 88. In the item reduction and scale generation phase (stage 3), expert service users contributed to discussions concerning the ordering and clustering of the themes and items and finalised the measures. Expert service users were also involved in the implementation and dissemination of ReQoL (stage 4). Expert service users contributed to the interpretation of findings, provided inputs at every stage of the project and were key decision-makers. The challenges include additional work to make the technical materials accessible, extra time to the project timescales, including time to achieve consensus from different opinions, sometimes strongly held, and extra costs. CONCLUSION: This study demonstrates a successful example of how PI can be embedded in research, namely in instrument development. The rewards of doing so cannot be emphasised enough but there are challenges, albeit surmountable ones. Researchers should anticipate and address those challenges during the planning stage of the project.

Integrating Qualitative and Quantitative Data in the Development of Outcome Measures: The Case of the Recovering Quality of Life (ReQoL) Measures in Mental Health Populations.

While it is important to treat symptoms, there is growing recognition that in order to help people with mental health problems lead meaningful and fulfilling lives, it is crucial to capture the impact of their conditions on wider aspects of their social lives. We constructed two versions of the Recovering Quality of Life (ReQoL) measure—ReQoL-10 and ReQoL-20—for use in routine settings and clinical trials from a larger pool of items by combining qualitative and quantitative evidence covering six domains. Qualitative evidence was gathered through interviews and focus groups with over 76 service users, clinicians, and a translatability assessment. Psychometric evidence generated from data from over 6200 service users was obtained from confirmatory factor models and item response theory analyses. In this paper we present an approach based on a traffic light pictorial format that was developed to present qualitative and quantitative evidence to a group of service users, clinicians, and researchers to help to make the final selection. This work provides a pragmatic yet rigorous approach to combining qualitative and quantitative evidence to ensure that ReQoL is psychometrically robust and has high relevance to service users and clinicians. This approach can be extended to the development of patient reported outcome measures in general.

Reaching consensus on reporting patient and public involvement (PPI) in research: methods and lessons learned from the development of reporting guidelines.

INTRODUCTION: Patient and public involvement (PPI) is inconsistently reported in health and social care research. Improving the quality of how PPI is reported is critical in developing a higher quality evidence base to gain a better insight into the methods and impact of PPI. This paper describes the methods used to develop and gain consensus on guidelines for reporting PPI in research studies (updated version of the Guidance for Reporting Patient and Public Involvement (GRIPP2)). METHODS: There were three key stages in the development of GRIPP2: identification of key items for the guideline from systematic review evidence of the impact of PPI on health research and health services, a three-phase online Delphi survey with a diverse sample of experts in PPI to gain consensus on included items and a face-to-face consensus meeting to finalise and reach definitive agreement on GRIPP2. Challenges and lessons learnt during the development of the reporting guidelines are reported. DISCUSSION: The process of reaching consensus is vital within the development of guidelines and policy directions, although debate around how best to reach consensus is still needed. This paper discusses the critical stages of consensus development as applied to the development of consensus for GRIPP2 and discusses the benefits and challenges of consensus development.

Evaluating public involvement in research design and grant development: Using a qualitative document analysis method to analyse an award scheme for researchers.

PLAIN ENGLISH SUMMARY: The National Institute for Health Research (NIHR) Research Design Service (RDS) for Yorkshire and Humber has been running a public involvement funding scheme since 2008. This scheme awards researchers a small amount of money to help them get involvement from patients and/or the public. Involvement activities take place at the time when researchers are planning studies, and when they are completing application forms to request funding for a proposed research project. After the public involvement activities researchers are asked to write a report for the RDS describing what they did with the public involvement funding. This study analysed those reports using an approach which included members of a public involvement panel in the data analysis process. The aim of the work was to see what the views and experiences of researchers who received funding were, and what might be learned for the future of the scheme. Twenty five reports were analysed. Four main themes were identified, these described: the added value of public involvement; aspects to consider when planning and designing public involvement; different roles of public contributors; and aspects of valuing public member contributions. The group approach to analysis was successful in enabling involvement of a variety of individuals in the process. The findings of the study provide evidence of the value of public involvement during the development of applications for research funding. The results also indicate that researchers recognise the variety in potential roles for the public in research, and acknowledge how involvement adds value to studies. ABSTRACT: Background A regional Research Design Service, funded by the National Institute for Health Research, introduced a small grant in 2008, to support public involvement (often known as patient and public involvement [PPI]) activities during the development of applications for research funding. Successful applicants are requested to submit a report detailing how the grant money was used, including a description of the aims and outcomes of the public involvement activities. The purpose of this study was to analyse the content of these reports. We aimed to find out what researcher views and experiences of public involvement activities were, and what lessons might be learned. Methods We used an innovative method of data analysis, drawing on group participatory approaches, qualitative content analysis, and Framework Analysis to sort and label the content of the reports. We developed a framework of categories and sub-categories (or themes and sub-themes) from this process. Results Twenty five documents were analysed. Four main themes were identified in the data: the added value of public involvement; planning and designing involvement; the role of public members; and valuing public member contributions. Within these themes, sub-themes related to the timing of involvement (prior to the research study/intended during the research study), and also specific benefits of public involvement such as: validating ideas; ensuring appropriate outcomes; ensuring the acceptability of data collection methods/tools and advice regarding research processes. Other sub-themes related to: finding and approaching public members; timing of events; training/support; the format of sessions; setting up public involvement panels: use of public contributors in analysis and interpretation of data; and using public members to assist with dissemination and translation into practice. Conclusions The analysis of reports submitted by researchers following involvement events provides evidence of the value of public involvement during the development of applications for research funding, and details a method for involving members of the public in data analysis which could be of value to other researchers The findings of the analysis indicate recognition amongst researchers of the variety in potential roles for public members in research, and also an acknowledgement of how involvement adds value to studies.

Exploring the impact of patient and public involvement in a cancer research setting.

An enduring theme in the literature exploring patient and public involvement (PPI) in research has been the focus on evaluating impact, defined usually in terms of participants' practical contribution to enhancing research processes. By contrast, there has been less emphasis on the perspectives and experiences of those involved in PPI. Drawing on qualitative data with people involved in the National Cancer Research Network in the United Kingdom, we report on what motivated participants to get involved and their experiences of involvement in this setting. We highlight how those involved in PPI often espoused the notion of the "good citizen," with PPI in research being a natural extension of their wider civic interests. However, our findings also highlight how PPI was an important resource, utilized by participants to make sense of living with chronic illness. We suggest that PPI in research also offers spaces for the reconfiguration of self and identity.

Credibility and the 'professionalized' lay expert: reflections on the dilemmas and opportunities of public involvement in health research.

Contemporary health policy in England places increasing emphasis on patient and public involvement (PPI) in health and health research. With regard to the latter, it has been suggested that PPI brings 'different' perspectives to research decision-making spaces, based on what has been referred to as 'experiential expertise'. This article presents findings from a qualitative study of PPI in cancer research settings in England. We argue that participants highlighted specific forms of expertise in their accounts about involvement, above and beyond experiential expertise, which they felt legitimated their claims to be credible participants within cancer research settings. We report here on the various strategies by which participants sought to accomplish this and highlight, in particular, a concomitant process of 'professionalization' of some within our group of participants. We discuss the significance of these findings in the context of recent debates around the status of experiential expertise.

Can the impact of public involvement on research be evaluated? A mixed methods study.

BACKGROUND: Public involvement is central to health and social research policies, yet few systematic evaluations of its impact have been carried out, raising questions about the feasibility of evaluating the impact of public involvement. OBJECTIVE: To investigate whether it is feasible to evaluate the impact of public involvement on health and social research. METHODS: Mixed methods including a two-round Delphi study with pre-specified 80% consensus criterion, with follow-up interviews. UK and international panellists came from different settings, including universities, health and social care institutions and charitable organizations. They comprised researchers, members of the public, research managers, commissioners and policy makers, self-selected as having knowledge and/or experience of public involvement in health and/or social research; 124 completed both rounds of the Delphi process. A purposive sample of 14 panellists was interviewed. RESULTS: Consensus was reached that it is feasible to evaluate the impact of public involvement on 5 of 16 impact issues: identifying and prioritizing research topics, disseminating research findings and on key stakeholders. Qualitative analysis revealed the complexities of evaluating a process that is subjective and socially constructed. While many panellists believed that it is morally right to involve the public in research, they also considered that it is appropriate to evaluate the impact of public involvement. CONCLUSIONS: This study found consensus among panellists that it is feasible to evaluate the impact of public involvement on some research processes, outcomes and on key stakeholders. The value of public involvement and the importance of evaluating its impact were endorsed.

The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research.

OBJECTIVES: The aim of this study was to develop the GRIPP (Guidance for Reporting Involvement of Patients and Public) checklist to enhance the quality of PPI reporting. METHODS: Thematic analysis was used to synthesize key issues relating to patient and public involvement (PPI) identified in the PIRICOM and PAPIRIS systematic reviews. These issues informed the development of the GRIPP checklist. RESULTS: The key issues identified included limited conceptualization of PPI, poor quality of methods reporting, unclear content validity of studies, poor reporting of context and process, enormous variability in the way impact is reported, little formal evaluation of the quality of involvement, limited focus on negative impacts, and little robust measurement of impact. The GRIPP checklist addresses these key issues. CONCLUSION: The reporting of patient and public involvement in health research needs significant enhancement. The GRIPP checklist represents the first international attempt to enhance the quality of PPI reporting. Better reporting will strengthen the PPI evidence-base and so enable more effective evaluation of what PPI works, for whom, in what circumstances and why.

Selecting outcome measures in mental health: the views of service users.

BACKGROUND: Little is known about service users' views of measures used to evaluate treatments in mental health. AIMS: To identify the views of people with psychosis and affective disorder about the relevance and acceptability of commonly used outcome measures. METHODS: Twenty-four widely used outcome measures were presented to expert groups of service users. Nominal group methods were used to develop consensus about the appropriateness of each measure. Comments made by service users about how outcomes should be assessed were also recorded. RESULTS: Group members expressed concern about the ability of some outcome measures to capture their experiences. Patient-rated measures were assessed as more relevant and appropriate than staff-rated measures, and the need to examine negative as well as the positive effects of treatments was emphasised. Specific concerns were raised about some widely used measures including the Global Assessment of Functioning and the European Quality of Life scale. CONCLUSIONS: We consider it essential that service users' views are taken into account when selecting measures to evaluate treatment outcomes. Providing insight into views of users of mental health services, our findings serve as a starting point for discussion.

Health researchers' attitudes towards public involvement in health research.

OBJECTIVE: To investigate health researchers' attitudes to involving the public in research. BACKGROUND: Public involvement in research is encouraged by the Department of Health in the UK. Despite this, the number of health researchers actively involving the public in research appears to be limited. There is little research specifically addressing the attitudes of health researchers towards involving the public: how they interpret the policy, what motivates and de-motivates them and what their experiences have been to date. DESIGN: A qualitative research design, using semi-structured telephone interviews. SETTING AND PARTICIPANTS: Fifteen purposively sampled UK-based University health researchers were the participants. Interviews were conducted over the telephone. FINDINGS: The participants suggested varying constructions of public involvement in research. Arguments based on moral and political principles and consequentialist arguments for involving the public in research were offered and most participants highlighted the potential benefits of involving the public. However, feelings of apprehension expressed by some participants imply that a number of researchers may still be uncomfortable with involving the public, as it presents a different way of working.

Involving consumers successfully in NHS research: a national survey.

OBJECTIVES: To investigate how far and in what way consumers are involved in NHS research. BACKGROUND: There is guidance from the UK Department of Health on involving consumers in research, but it is not known how these policies have been implemented. DESIGN: A national postal survey was conducted of 884 researchers selected randomly from the National Research Register, 16 researchers registered on the INVOLVE database and 15 consumers nominated by researchers who collaborated in the same research projects. SETTING: The survey participants were drawn from diverse settings including NHS organizations and universities. PARTICIPANTS: Researchers and consumers collaborating in the same projects. MAIN OUTCOME MEASURES: Details of how consumers were involved and the number of projects that met previously developed consensus-derived indicators of successful consumer involvement in NHS research. RESULTS: Of the 900 researchers who were sent a postal questionnaire, 518 responded, giving a response rate of 58%. Nine of the 15 consumers responded. Eighty-eight (17%) researchers reported involving consumers, mainly as members of a project steering group, designing research instruments and/or planning or designing the research methods. Most projects met between one and four indicators. CONCLUSIONS: This national survey revealed that only a small proportion of NHS researchers were actively involving consumers. This study provides a useful marker of how far the Department of Health's policy on consumer involvement in NHS research has been implemented and in what way.

Principles and indicators of successful consumer involvement in NHS research: results of a Delphi study and subgroup analysis.

Consumer involvement in NHS research is Department of Health policy within the UK. Despite the existence of policy directives and guidance, until recently there has been no consensus among consumers and researchers about what it means to involve consumers successfully in NHS research. This paper discusses the value of consensus research in this policy area, and presents the detailed findings of a Delphi study carried out to reach consensus on principles and indicators of successful consumer involvement in NHS research. Study participants, comprising consumers, researchers and consumer-researchers, were identified using a purposive sampling strategy. Consensus was reached on eight clear and valid principles of successful consumer involvement in NHS research, with each principle having at least one clear and valid indicator. Subgroup analysis revealed few significant differences in how consumers, researchers and consumer-researchers rated the principles and indicators. The implications and limitations of the study are discussed. Further research is needed to assess: (1) the usefulness of the principles and indicators for differing models of consumer involvement, health research methodologies, and subject areas within health research; and (2) the impact of 'successful' consumer involvement on health research processes and outcomes.