An evidence-based framework for identifying technologies of no or low-added value (NLVT).

OBJECTIVE: To synthetize the state of the art of methods for identifying candidate technologies for disinvestment and propose an evidence-based framework for executing this task. METHODS: An interpretative review was conducted. A systematic literature search was performed to identify secondary or tertiary research related to disinvestment initiatives and/or any type of research that specifically described one or more methods for identifying potential candidates technologies, services, or practices for disinvestment. An iterative and critical analysis of the methods described alongside the disinvestment initiatives was performed. RESULTS: Seventeen systematic reviews on disinvestment or related terms (health technology reassessment or medical reversal) were retrieved and methods of 45 disinvestment initiatives were compared. On the basis of this evidence, we proposed a new framework for identifying these technologies based on the wide definition of evidence provided by Lomas et al. The framework comprises seven basic approaches, eleven triggers and thirteen methods for applying these triggers, which were grouped in embedded and ad hoc methods. CONCLUSIONS: Although identification methods have been described in the literature and tested in different contexts, the proliferation of terms and concepts used to describe this process creates considerable confusion. The proposed framework is a rigorous and flexible tool that could guide the implementation of strategies for identifying potential candidates for disinvestment.

Comparatively Ineffective? PCORI and the Uphill Battle to Make Evidence Count in US Medicine.

The Patient-Centered Outcomes Research Institute (PCORI) was established as part of the Affordable Care Act to promote research on the comparative effectiveness of treatment options. Advocates hoped this information would help reduce wasteful spending by identifying low-value treatments, but many conservatives and industry groups feared PCORI would ration care and threaten physicians' autonomy. PCORI faced three challenges during its first decade of operation: overcoming the controversy of its birth and escaping early termination, shaping medical practice, and building a public reputation for relevance. While PCORI has won reauthorization, it has not yet had a major impact on the decisions of clinicians or payers. PCORI's modest footprint reflects not only the challenges of getting a new organization off the ground but also the larger political, financial, and cultural barriers to the uptake of medical evidence in the US health care system. The growing attention among policymakers and researchers to provider prices (rather than utilization) as the driver of health care spending could be helpful to the political prospects of the evidence-based medicine project by making it appear to be less as rationing driven by costs and more as an effort to improve quality and uphold medical professionalism.

A Culture-Centered Community-Grounded Approach to Disseminating Health Information among African Americans.

This study highlights the role of local communities in creating culturally rooted health information resources based on comparative effectiveness research (CER), depicting the role of culture in creating entry points for building community-grounded communication structures for evidence-based health knowledge. We report the results from running a year-long culture-centered campaign that was carried out among African American communities in two counties, Lake and Marion County, in Indiana addressing basic evidence-based knowledge on four areas of cardiovascular disease (CVD). Campaign effectiveness was tested through an experimental design with post-test knowledge of CER among African Americans in these counties compared to CER knowledge among African Americans in a comparable control county (Allen). Our campaign, based on the principles of the culture-centered approach (CCA), increased community CER knowledge in the experimental communities relative to a community that did not receive the culturally centered health information campaign. The CCA-based campaign developed by community members and distributed through the mass media, community wide channels such as health fairs and church meetings, postcards, and face-to-face interventions explaining the postcards improved CER knowledge in specific areas (ACE-I/ARBs, atrial fibrillation, and renal artery stenosis) in the CCA communities as compared to the control community.

A Multilevel Approach to Stakeholder Engagement in the Formulation of a Clinical Data Research Network.

OBJECTIVES: To ensure meaningful engagement of stakeholders (patients, clinicians, and communities) in developing the Mid-South Clinical Data Research Network (MS-CDRN), we implemented a comprehensive, multilevel approach: (1) identify barriers to involving stakeholders in governance, network design, and implementation; (2) engage stakeholders in priority setting and research topic generation; (3) develop strategies to fully integrate stakeholders in CDRN governance and oversight; and (4) solicit guidance on patient-centered tools and strategies for recruiting research participants. METHODS: We engaged stakeholders: (1) as integral research team members; (2) on oversight and advisory committees; (3) as consultants (using Community Engagement Studios); and (4) through interviews and surveys. We recruited stakeholders from community health centers, churches, barbershops, health fairs, a volunteer registry, and a patient portal. We prioritized recruitment from populations often underrepresented in research. RESULTS: During the first 18 months, we engaged 5670 stakeholders in developing the MS-CDRN. These were research team members and on governance committees (N=10), consultants (N=58), survey respondents (N=5543), and interviewees (N=59). Stakeholders identified important barriers and facilitators to engagement, developed stakeholder-informed policies, provided feedback on priority topics and research questions, and developed an intake process for data requests and interventional studies that included reviewing for appropriate patient-centeredness, patient engagement, and dissemination. DISCUSSION: Multilevel stakeholder engagement is a novel systematic approach to developing a meaningful patient-centered and patient-engaged research program. This approach allows ongoing input from highly engaged stakeholders while leveraging focused input from larger, more diverse groups to enhance the patient-centeredness of research and increase relevance to broader audiences.

Engaging Stakeholders to Develop a Patient-centered Research Agenda: Lessons Learned From the Research Action for Health Network (REACHnet).

BACKGROUND: Patient-centered research requires a focus on the needs and priorities of patients. Because patient engagement can result in the discovery of important topics not currently prioritized by research programs, topic generation, and prioritization activities conducted with patients, caregivers, and other stakeholders are essential. To develop patient-centered research agendas for obesity and diabetes, the Research Action for Health Network conducted topic generation and prioritization activities with multistakeholder research advisory groups. OBJECTIVES: The purpose of this case study was to demonstrate how methods for engaging patients in topic generation and prioritization can be implemented in practice for the development of a patient-centered research agenda. RESEARCH DESIGN: Four multistakeholder groups comprising patients, clinicians, and researchers met 4-5 times between November 2014 and July 2015 to generate and prioritize topics for obesity and diabetes research. Topics were prioritized using an iterative engagement process, in which themes were identified and resulting topics were refined and ranked over multiple meetings. PARTICIPANTS: Sixty-four patients, clinicians, and researchers participated in 2 obesity and 2 diabetes advisory groups. The majority of participants (64.0%) were patients, followed by clinicians (23.4%), researchers (9.4%), and parents of children with diabetes (3.1%). RESULTS: Ten and 12 priority topics were identified for obesity and diabetes, respectively. The resulting research agendas were disseminated to patients, researchers, and clinicians. CONCLUSIONS: Patient engagement has the potential to enrich our understanding of patient priorities for research. The results from this process suggest that convening in-person multistakeholder groups can be an effective way to generate research topics that reflect patients' priorities. Engagement strategies should be focused not only on the development of patient-centered research topics but also on the implementation of these topics into research studies.

Patient vs. Community Engagement: Emerging Issues.

BACKGROUND: The value proposition of including patients at each step of the research process is that patient perspectives and preferences can have a positive impact on both the science and the outcomes of comparative effectiveness research. How to accomplish engagement and the extent to which approaches to community engagement inform strategies for effective patient engagement need to be examined to address conducting and accelerating comparative effectiveness research. OBJECTIVES: To examine how various perspectives and diverse training lead investigators and patients to conflicting positions on how best to advance patient engagement. RESEARCH DESIGN: Qualitative methods were used to collect perspectives and models of engagement from a diverse group of patients, researchers and clinicians. The project culminated with a workshop involving these stakeholders. The workshop used a novel approach, combining World Café and Future Search techniques, to compare and contrast aspects of patient engagement and community engagement. SUBJECTS: Participants included patients, researchers, and clinicians. MEASURES: Group and workshop discussions provided the consensus on topics related to patient and community engagement. RESULTS: Participants developed and refined a framework that compares and contrasts features associated with patient and community engagement. CONCLUSIONS: Although patient and community engagement may share a similar approach to engagement based on trust and mutual benefit, there may be distinctive aspects that require a unique lexicon, strategies, tactics, and activities.

Building Meaningful Patient Engagement in Research: Case Study From ADVANCE Clinical Data Research Network.

BACKGROUND: Strategies to engage patients to improve and enhance research and clinical care are increasingly being implemented in the United States, yet little is known about best practices for or the impacts of meaningful patient engagement. OBJECTIVE: We describe and reflect on our patient stakeholder groups, engagement framework, experiences, and lessons learned in engaging patients in research, from generating proposal ideas to disseminating findings. SETTING: The ADVANCE (Accelerating Data Value Across a National Community Health Center Network) clinical data research network is the nation's largest clinical dataset on the safety net, with outpatient clinical data from 122 health systems (1109 clinics) in 23 states. RESULTS: Patients stakeholders codeveloped the ADVANCE engagement framework and its implementation in partnership with network leaders. In phase I of ADVANCE, patients were involved with designing studies (input on primary outcome measures and methods) and usability testing (of the patient portal). In phase II, the network is prioritizing research training, dissemination opportunities, an "ambassador" program to pair more experienced patient stakeholders with those less experienced, and evaluation of engagement activities and impacts. DISCUSSION: The ADVANCE framework for patient engagement has successfully involved a diverse group of patients in the design, implementation, and interpretation of comparative effectiveness research. Our experience and framework can be used by other organizations and research networks to support patient engagement activities.

MoodNetwork: An Innovative Approach to Patient-centered Research.

INTRODUCTION: Research has often been viewed as a passive process by which participants enroll in studies developed by researchers. It is becoming clearer that to understand the nuances of mood episodes and how to prevent them, we need to conduct large clinical trials that have the power to investigate moderators and mediators, or catalysts and mechanisms of change. MoodNetwork, the first online, patient-centered research community for individuals with mood disorders, aims to change the way that traditional research has been conducted by involving patients, their caregivers, and advocates in the process of research. The aim of this report is to share lessons learned from developing MoodNetwork. METHODS: Participants enroll by completing a demographic survey and consent form. Once enrolled, participants are encouraged to complete optional surveys about their mood disorders and areas of research priority. Stakeholder and advocacy partners developed the website, web-based surveys, and recruitment materials. RESULTS: MoodNetwork has enrolled 4103 participants to date. Of this sample, 96.9% report experiencing depression and 79.7% endorse symptoms of mania or hypomania. Participants rated reducing stigma and alleviating symptoms as their 2 largest research priorities. Recruitment has been slower than expected. Recruiting a diverse sample has been challenging, and this impacts the Network's ability to conduct comparative effectiveness research studies. DISCUSSION: We discuss lessons learned from recruiting individuals with mood disorders to MoodNetwork, an innovative approach to conducting clinical trials. We identify and review 5 strategies for increasing enrollment as well as future directions.

Crohn's and Colitis Foundation of America Partners Patient-Powered Research Network: Patient Perspectives on Facilitators and Barriers to Building an Impactful Patient-Powered Research Network.

BACKGROUND: To build a Patient-Powered Research Networks (PPRN) that prioritizes the needs of its members who have inflammatory bowel diseases (IBD), we sought to better understand patients' preferences for what are the essential features that will facilitate and sustain engagement. METHODS: We conducted a two-phase qualitative study. Seven focus groups involving 62 participants with IBD were conducted (phase 1). Focus group results informed the phase 2 cognitive interviews, which included 13 phone interviews. Topics included experiences with IBD and research, PPRN engagement, patient-generated health data, and resources/tools to facilitate self-management. All focus groups and interviews were digitally recorded, transcribed verbatim, and analyzed in ATLAS.ti 7.5. Thematic categories were derived from the data, and codes were grouped into emergent themes and relationships. RESULTS: Four major themes emerged through inductive coding: (1) the impact of knowing; (2) participation barriers and challenges; (3) engagement and collaboration; and (4) customizable patient portal features/functionalities. Participants were motivated to participate in the PPRN because the knowledge gained from research studies would benefit both society and the individual. Main concerns included credibility of online resources, pharmaceutical industry profiting from their data, data security, and participation expectations. Participants wanted a true and equal partnership in every phase of building a PPRN. Participants felt it was important to have access to personal health records and be able to track health status and symptoms. CONCLUSION: Partnering with participants throughout PPRN development was critical to understanding the needs and preferences of patients with IBDs and for shaping engagement strategies and the portal's design.

A Novel Stakeholder Engagement Approach for Patient-centered Outcomes Research.

INTRODUCTION/OBJECTIVES: The engagement of patients and other stakeholders is a critical element in the design of patient-centered outcomes research studies. However, methodology for scalable engagement in research management particularly activities such as operationalization of principles and setting of priorities is not well-developed. The objective of this study is to describe a novel approach for scalable stakeholder engagement in research aligned with the Patient-Centered Outcomes Research Institute (PCORI) engagement principles, which was evaluated in a national clinical data research network. MATERIALS AND METHODS: Patient, patient advocate, clinician, and researcher stakeholders were recruited from clinical sites, as well as social media sites related to the 3 conditions of focus, heart failure, obesity, and Kawasaki disease. The engagement strategy was designed, implemented, and mapped to the PCORI engagement principles. Evaluation included internal assessment and quantitative measures of online engagement. RESULTS: We operationalized the PCORI principles with 12 stakeholder engagement strategies and convened stakeholder advisory boards and online research prioritization panels to determine research priorities in a rigorous, deliberative process. A total of 46 advisors (20 patients) and 339 panelists (159 patients) actively participated. There were not significant differences between patients and clinicians in level of online engagement. Nonetheless, while patients reported a slightly greater challenge with following online discussion, they overall had a more favorable opinion about use of the online format. DISCUSSION/CONCLUSION: An efficient way to engage large numbers of representative stakeholders in research is a necessary first step to assure the public of trustworthy use of data networks for health research. This paper describes a comprehensive approach to engagement in patient-centered outcomes research management that informs ongoing development of rigorous methodologies in this area.

Experiences of Structured Elicitation for Model-Based Cost-Effectiveness Analyses.

BACKGROUND: Empirical evidence supporting the cost-effectiveness estimates of particular health care technologies may be limited, or it may even be missing entirely. In these situations, additional information, often in the form of expert judgments, is needed to reach a decision. There are formal methods to quantify experts' beliefs, termed as structured expert elicitation (SEE), but only limited research is available in support of methodological choices. Perhaps as a consequence, the use of SEE in the context of cost-effectiveness modelling is limited. OBJECTIVES: This article reviews applications of SEE in cost-effectiveness modelling with the aim of summarizing the basis for methodological choices made in each application and recording the difficulties and challenges reported by the authors in the design, conduct, and analyses. METHODS: The methods used in each application were extracted along with the criteria used to support methodological and practical choices and any issues or challenges discussed in the text. Issues and challenges were extracted using an open field, and then categorised and grouped for reporting. RESULTS: The review demonstrates considerable heterogeneity in methods used, and authors acknowledge great methodological uncertainty in justifying their choices. Specificities of the context area emerging as potentially important in determining further methodological research in elicitation are between- expert variation and its interpretation, the fact that substantive experts in the area may not be trained in quantitative subjects, that judgments are often needed on various parameter types, the need for some form of assessment of validity, and the need for more integration with behavioural research to devise relevant debiasing strategies. CONCLUSIONS: This review of experiences of SEE highlights a number of specificities/constraints that can shape the development of guidance and target future research efforts in this area.

DEFINING INNOVATION WITH RESPECT TO NEW MEDICINES: A SYSTEMATIC REVIEW FROM A PAYER PERSPECTIVE.

OBJECTIVES: The aim of this study was to investigate how innovation is defined with respect to new medicines. METHODS: MEDLINE, Embase, and EconLit databases were searched for articles published between January 1, 2010 and May 25, 2016 that described a relevant definition of innovation. Identified definitions were analyzed by mapping the concepts described onto a set of ten dimensions of innovation. RESULTS: In total, thirty-six articles were included, and described a total of twenty-five different definitions of innovation. The most commonly occurring dimension was therapeutic benefit, with novelty and the availability of existing treatments the second and third most common dimensions. Overall, there was little agreement in the published literature on what characteristics of new medicines constitute rewardable innovation. CONCLUSIONS: Alignment across countries and among regulators, health technology assessment bodies and payers would help manufacturers define research policies that can drive innovation, but may be challenging, as judgements about what aspects of innovation should be rewarded vary among stakeholders, and depend on political and societal factors.

HEALTH TECHNOLOGY ASSESSMENT, DELIBERATIVE PROCESS, AND ETHICALLY CONTESTED ISSUES.

Healthcare technology assessment (HTA) aims to support decisions as to which technologies should be used in which situations to optimize value. Because such decisions will create winners and losers, they are bound to be controversial. HTA, then, faces a dilemma: should it stay away from such controversies, remaining a source of incomplete advice and risking an important kind of marginalization, or should it enter the controversy? The question is a challenging one, because we lack agreement on principles that are fine grained enough to tell us what choices we should make. In this study, we will argue that HTA should take a stand on ethical issues raised by the technology that is being investigated. To do so, we propose adding a form of procedural justice to HTA to arrive at decisions that the public can regard as legitimate and fair. A fair process involves deliberation about the reasons, evidence, and rationales that are considered relevant to meeting population-health needs fairly. One important way to make sure that there is real deliberation about relevant reasons is to include a range of stakeholders in the deliberative process. To illustrate how such deliberation might work, we use the case of cochlear implants for deaf children.

Comparative Effectiveness Research and Children With Cerebral Palsy: Identifying a Conceptual Framework and Specifying Measures.

PURPOSE: A step toward advancing research about rehabilitation service associated with positive outcomes for children with cerebral palsy is consensus about a conceptual framework and measures. METHODS: A Delphi process was used to establish consensus among clinicians and researchers in North America. RESULTS: Directors of large pediatric rehabilitation centers, clinicians from large hospitals, and researchers with expertise in outcomes participated (N = 18). Andersen's model of health care utilization framed outcomes: consumer satisfaction, activity, participation, quality of life, and pain. Measures agreed upon included Participation and Environment Measure for Children and Youth, Measure of Processes of Care, PEDI-CAT, KIDSCREEN-10, PROMIS Pediatric Pain Interference Scale, Visual Analog Scale for pain intensity, PROMIS Global Health Short Form, Family Environment Scale, Family Support Scale, and functional classification levels for gross motor, manual ability, and communication. CONCLUSIONS: Universal forms for documenting service use are needed. Findings inform clinicians and researchers concerned with outcome assessment.

The patient-centered outcomes research institute: research done differently.

BACKGROUND: The Patient-Centered Outcomes Research Institute (PCORI) was authorized by the Patient Protection and Affordable Care Act of 2010 to fund comparative clinical effectiveness research to provide reliable evidence to help patients and their healthcare providers make informed decisions. OBJECTIVE: The aim of this paper is to describe the synergy between nursing research and PCORI funding priorities, discuss unique aspects of PCORI funding, and identify the implications for nurse researchers. DISCUSSION: Goals of nursing research are well aligned with PCORI interests. Given this synergy, many phenomena that nursing scientists study could become topics of a competitive proposal for PCORI funding. Major aspects of PCORI funding with implications for nurse researchers include the nature of the questions asked, funding priorities, engagement of patients and stakeholders, compliance with the PCORI's methodology standards, and the proposal review criteria and process.

Development, implementation and evaluation of an evidence-based program for introduction of new health technologies and clinical practices in a local healthcare setting.

BACKGROUND: This paper reports the process of establishing a transparent, accountable, evidence-based program for introduction of new technologies and clinical practices (TCPs) in a large Australian healthcare network. Many countries have robust evidence-based processes for assessment of new TCPs at national level. However many decisions are made by local health services where the resources and expertise to undertake health technology assessment (HTA) are limited and a lack of structure, process and transparency has been reported. METHODS: An evidence-based model for process change was used to establish the program. Evidence from research and local data, experience of health service staff and consumer perspectives were incorporated at each of four steps: identifying the need for change, developing a proposal, implementation and evaluation. Checklists assessing characteristics of success, factors for sustainability and barriers and enablers were applied and implementation strategies were based on these findings. Quantitative and qualitative methods were used for process and outcome evaluation. An action research approach underpinned ongoing refinement to systems, processes and resources. RESULTS: A Best Practice Guide developed from the literature and stakeholder consultation identified seven program components: Governance, Decision-Making, Application Process, Monitoring and Reporting, Resources, Administration, and Evaluation and Quality Improvement. The aims of transparency and accountability were achieved. The processes are explicit, decisions published, outcomes recorded and activities reported. The aim of ascertaining rigorous evidence-based information for decision-making was not achieved in all cases. Applicants proposing new TCPs provided the evidence from research literature and local data however the information was often incorrect or inadequate, overestimating benefits and underestimating costs. Due to these limitations the initial application process was replaced by an Expression of Interest from applicants followed by a rigorous HTA by independent in-house experts. CONCLUSION: The program is generalisable to most health care organisations. With one exception, the components would be achievable with minimal additional resources; the lack of skills and resources required for HTA will limit effective application in many settings. A toolkit containing details of the processes and sample materials is provided to facilitate replication or local adaptation by those wishing to establish a similar program.

PAYER PERSPECTIVES ON FUTURE ACCEPTABILITY OF COMPARATIVE EFFECTIVENESS AND RELATIVE EFFECTIVENESS RESEARCH.

OBJECTIVES: Our objective was to gather perspectives from payers on how comparative effectiveness research (CER) in the United States and relative effectiveness (RE) research in Europe will impact evidentiary standards for access decisions of new drugs by 2020. METHODS: We conducted semi-structured interviews with fourteen senior officials representing public and private payers, health technology assessment groups, and pricing and reimbursement bodies in the United States and Europe. An online survey assessed current use of CER/RE evidence and potential trends that might influence its use for decision making by 2020. A semi-structured interview elicited payers' definitions of CER/RE and was structured around four hypothetical cases resembling drugs expected to be more common or poised to create policy challenges by 2020. Topics included acceptance of study designs and analytic methods associated with CER/RE. A systematic content review was done to extract relevant information. RESULTS: According to key informants, randomization will remain an essential component for assessing comparative or relative effectiveness. They anticipate greater use of policy levers such as conditional reimbursement or prior authorization to manage diffusion of new drugs. Case studies provided important insights into situations when certain types of CER evidence may be acceptable (e.g., observational data when differences between drugs are largely convenience). CONCLUSIONS: Industry perceptions that CER/RE will change payers' evidentiary requirements in the future are consistent with our findings. Growing investment in payers' own data and increased reliance on policy tools to control diffusion of new drugs may also influence the type of evidence industry will be required to produce by 2020.

Approaches to comparative effectiveness research in multimorbid populations.

There is an urgent need for an evidence base to guide care for patients with multiple chronic medical conditions (MCC). Comparative effectiveness research (CER) has been touted as 1 solution to generating such evidence. However, the majority of CER topics and methods are designed to generate evidence applicable to single diseases. Generating evidence to guide the care of MCC populations requires thoughtful, and often alternative, approaches to using the existing armamentarium of CER methods. To initiate a dialog about appropriate methods for CER in MCC populations, we discuss advantages and disadvantages of experimental and quasi-experimental study designs for CER in MCC populations, estimating heterogeneity of treatment effects, developing meaningful outcome measures, and aligning morbidity measurement with relevant outcomes. Through an engaged dialog with clinicians, methodologists, and patients, evidence about strengths and limitations of alternative approaches, recommendations about preferred methods for CER in MCC can be developed to ensure that knowledge gaps are filled by valid evidence.

How to set up a departmental comparative effectiveness research unit: one department's experience.

OBJECTIVE: Comparative effectiveness research (CER) is the comparison of clinical interventions in real-world settings. The purpose of this article is to discuss the experiences of a CER unit created within the radiology department of one medical institution to provide an example of how to pursue CER within the field of radiology. CONCLUSION: Medical institutions would benefit from investing in CER by creating research groups specifically devoted to this evolving field.