Assessing the impact of healthcare research: A systematic review of methodological frameworks.

BACKGROUND: Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix. METHODS AND FINDINGS: Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) 'primary research-related impact', (2) 'influence on policy making', (3) 'health and health systems impact', (4) 'health-related and societal impact', and (5) 'broader economic impact'. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched. CONCLUSIONS: The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

A review of patient and carer participation and the use of qualitative research in the development of core outcome sets.

BACKGROUND: To be meaningful, a core outcome set (COS) should be relevant to all stakeholders including patients and carers. This review aimed to explore the methods by which patients and carers have been included as participants in COS development exercises and, in particular, the use and reporting of qualitative methods. METHODS: In August 2015, a search of the Core Outcomes Measures in Effectiveness Trials (COMET) database was undertaken to identify papers involving patients and carers in COS development. Data were extracted to identify the data collection methods used in COS development, the number of health professionals, patients and carers participating in these, and the reported details of qualitative research undertaken. RESULTS: Fifty-nine papers reporting patient and carer participation were included in the review, ten of which reported using qualitative methods. Although patients and carers participated in outcome elicitation for inclusion in COS processes, health professionals tended to dominate the prioritisation exercises. Of the ten qualitative papers, only three were reported as a clear pre-designed part of a COS process. Qualitative data were collected using interviews, focus groups or a combination of these. None of the qualitative papers reported an underpinning methodological framework and details regarding data saturation, reflexivity and resource use associated with data collection were often poorly reported. Five papers reported difficulty in achieving a diverse sample of participants and two reported that a large and varied range of outcomes were often identified by participants making subsequent rating and ranking difficult. CONCLUSIONS: Consideration of the best way to include patients and carers throughout the COS development process is needed. Additionally, further work is required to assess the potential role of qualitative methods in COS, to explore the knowledge produced by different qualitative data collection methods, and to evaluate the time and resources required to incorporate qualitative methods into COS development.