Protocol for the development of guidance for collaborator and partner engagement in health care evidence syntheses.

BACKGROUND: Involving collaborators and partners in research may increase relevance and uptake, while reducing health and social inequities. Collaborators and partners include people and groups interested in health research: health care providers, patients and caregivers, payers of health research, payers of health services, publishers, policymakers, researchers, product makers, program managers, and the public. Evidence syntheses inform decisions about health care services, treatments, and practice, which ultimately affect health outcomes. Our objectives are to: A. Identify, map, and synthesize qualitative and quantitative findings related to engagement in evidence syntheses B. Explore how engagement in evidence synthesis promotes health equity C. Develop equity-oriented guidance on methods for conducting, evaluating, and reporting engagement in evidence syntheses METHODS: Our diverse, international team will develop guidance for engagement with collaborators and partners throughout multiple sequential steps using an integrated knowledge translation approach: 1. Reviews. We will co-produce 1 scoping review, 3 systematic reviews and 1 evidence map focusing on (a) methods, (b) barriers and facilitators, (c) conflict of interest considerations, (d) impacts, and (e) equity considerations of engagement in evidence synthesis. 2. Methods study, interviews, and survey. We will contextualise the findings of step 1 by assessing a sample of evidence syntheses reporting on engagement with collaborators and partners and through conducting interviews with collaborators and partners who have been involved in producing evidence syntheses. We will use these findings to develop draft guidance checklists and will assess agreement with each item through an international survey. 3. CONSENSUS: The guidance checklists will be co-produced and finalised at a consensus meeting with collaborators and partners. 4. DISSEMINATION: We will develop a dissemination plan with our collaborators and partners and work collaboratively to improve adoption of our guidance by key organizations. CONCLUSION: Our international team will develop guidance for collaborator and partner engagement in health care evidence syntheses. Incorporating partnership values and expectations may result in better uptake, potentially reducing health inequities.

Treatment goals for rheumatoid arthritis: patient engagement and goal collection.

Aim: We developed the Patient-Engaged Health Technology Assessment strategy for survey-based goal collection from patients to yield patient-important outcomes suitable for use in multi-criteria decision analysis. Methods: Rheumatoid arthritis patients were recruited from online patient networks for proof-of-concept testing of goal collection and prioritization using a survey. A Project Steering Committee and Expert Panel rated the feasibility of scaling to larger samples. Results: Survey respondents (n = 47) completed the goal collection exercise. Finding effective treatments was rated by respondents as the most important goal, and reducing stiffness was rated as the least important. Feedback from our steering committee and expert panel support the approach's feasibility for goal identification and ranking. Conclusion: Goals relevant for treatment evaluation can be identified and rated for importance by patients to permit wide input from patients with lived experience of disease.

Use of Real-World Evidence for Regulatory Approval and Coverage of Medical Devices: A Landscape Assessment.

OBJECTIVES: To enhance the generalizability of the evidence it reviews, the US Food and Drug Administration (FDA) has encouraged manufacturers to expand the submission of real-world evidence (RWE). The extent to which this evidence, which is generated outside of research settings, can support decision making remains unclear. We described the current use of RWE for medical devices, assessed manufacturers' challenges in generating and using it for regulatory and coverage decisions, and identified opportunities to expand its use. METHODS: We conducted 27 solo and group interviews with FDA officials and representatives of device manufacturers, payers, and health technology assessment organizations. All interviews used a semistructured protocol and were transcribed to allow thematic analysis. RESULTS: Accessing and linking real-world data sources, identifying unique devices, capturing longitudinal data, limited staff expertise, and uncertain return on investment have hampered efforts to use real-world data. Many companies in our sample were conducting research using real-world data, but none had submitted RWE as the primary evidence supporting a premarket approval. FDA guidance was helpful, but regulatory requirements remained ambiguous and examples of successful regulatory decisions based on RWE were limited. Payers mainly used RWE to supplement experimental evidence in coverage decisions, evaluated both types of evidence in similar ways, and had concerns about the rigor of RWE. CONCLUSIONS: Technical challenges may slow efforts to generate and use RWE in the near term. Additional regulatory guidance and examples, greater use of rigorous study designs and analytic methods, and continued stakeholder engagement could accelerate the use of RWE.

Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.

BACKGROUND: Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation. METHODS: This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation. DISCUSSION: We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers. Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health.

Engaging Patients and Other Non-Researchers in Health Research: Defining Research Engagement.

With the increase in patient and consumer activism through the late twentieth century and into this century, patient roles in research evolved into a new model of research engagement, with patients serving as active advisors and co-leading or leading clinical research. By requiring active engagement of patients and other stakeholders, several government research funders have advanced this model, particularly in Canada, the United States (US), United Kingdom (UK), and Australia. A consortium of individuals from these countries formed a Multi-Stakeholder Engagement (MuSE) consortium to examine critical issues in engaged research, establish consensus on definitions, and provide guidance for the field, beginning with an overview of how to involve stakeholders in health research (Concannon et al. J Gen Intern Med. 2019;34(3):458-463) and continuing here with an examination of definitions of research engagement. The political and advocacy roots of engaged research are reflected in definitions. Engagement is conceptualized with reference to research project goals, from informing specific clinical decisions to informing health-system level decisions. Political and cultural differences across countries are evident. Some of these government funders focus on empirical rather than ethical rationales. In countries with centralized health technology assessment, the link between societal values and engaged research is explicit. Ethical rationales for engagement are explicit in most of the published literature on research engagement. Harmonization of definitions is recommended so that research engagement elements, methods, and outcomes and impacts can be clearly examined and understood, and so that the field of research engagement can proceed from a clear conceptual foundation. Specific recommendations for terminology definitions are provided. Placing engaged research on a continuum from specific clinical decisions to more global public and social justice concerns clarifies the type of engaged research, supports appropriate comparisons, and improves the rigor of engaged research methods. The results help identify knowledge gaps in this growing field.

Are Publicly Funded Health Databases Geographically Detailed and Timely Enough to Support Patient-Centered Outcomes Research?

Emerging health care research paradigms such as comparative effectiveness research (CER), patient-centered outcome research (PCOR), and precision medicine (PM) share one ultimate goal: constructing evidence to provide the right treatment to the right patient at the right time. We argue that to succeed at this goal, it is crucial to have both timely access to individual-level data and fine geographic granularity in the data. Existing data will continue to be an important resource for observational studies as new data sources are developed. We examined widely used publicly funded health databases and population-based survey systems and found four ways they could be improved to better support the new research paradigms: (1) finer and more consistent geographic granularity, (2) more complete geographic coverage of the US population, (3) shorter time from data collection to data release, and (4) improved environments for restricted data access. We believe that existing data sources, if utilized optimally, and newly developed data infrastructures will both play a key role in expanding our insight into what treatments, at what time, work for each patient.

Does framing of cancer survival affect perceived value of care? A willingness-to-pay survey of US residents.

AIMS: To investigate the relationship between the framing of survival gains and the perceived value of cancer care. METHODS: Through a population-based survey of 2040 US adults, respondents were randomized to one of the two sets of hypothetical scenarios, each of which described the survival benefit for a new treatment as either an increase in median survival time (median survival), or an increase in the probability of survival for a given length of time (landmark survival). Each respondent was presented with two randomly selected scenarios with different prognosis and survival improvements, and asked about their willingness to pay (WTP) for the new treatments. RESULTS: Predicted WTP increased with survival benefits and respondents' income, regardless of how survival benefits were described. Framing therapeutic benefits as improvements in landmark rather than median time survival increased the proportion of the population willing to pay for that gain by 11-35%, and the mean WTP amount by 42-72% in the scenarios we compared. CONCLUSION: How survival benefits are described may influence the value people place on cancer care.

Evidence of systematic duplication by new percutaneous coronary intervention programs.

BACKGROUND: Evidence suggests that recent and projected future investments in percutaneous coronary intervention (PCI) programs at US hospitals fail to increase access to timely reperfusion for patients with ST-segment elevation myocardial infarction. METHODS AND RESULTS: We set out to estimate the annual number and costs of new PCI programs in US hospitals from 2004 to 2008 and identify the characteristics of hospitals, neighborhoods, and states where new PCI programs have been introduced. We estimated a discrete-time hazard model to measure the influence of these characteristics on the decision of a hospital to introduce a new PCI program. In 2008, 1739 US hospitals were capable of performing PCI, a relative increase of 16.5% (251 hospitals) over 2004. The percentage of the US population with projected access to timely PCI grew by 1.8%. New PCI programs were more likely to be introduced in areas that already had a PCI program with more competition for market share, near populations with higher rates of private insurance, in states that had weak or no regulation of new cardiac catheterization laboratories, and in wealthier and larger hospitals. CONCLUSIONS: Our data show that new PCI programs were systematically duplicative of existing programs and did not help patients gain access to timely PCI. The total cost of recent US investments in new PCI programs is large and of questionable value for patients.

High priority future research needs for obstructive sleep apnea diagnosis and treatment.

STUDY OBJECTIVES: To identify and prioritize future research needs (FRN) topics for diagnosis and treatment of obstructive sleep apnea (OSA). METHODS: Twenty-one panel members represented six stake-holder categories: patients and the public, providers; purchasers of health care, payers, policymakers, and principal investigators. Building on a recently completed comparative effectiveness review, stakeholders nominated and discussed potential FRN topics. Stakeholders then nominated their top priority FRN topics based on the Agency for Healthcare Research and Quality Effective Health Care Program Selection Criteria. From these nominations, the highest priority FRN topics were determined and were elaborated upon to include possible study designs to address the topics. RESULTS: Thirty-seven topics were discussed and prioritized. The nine highest priority FRN topics included: cost-effectiveness of management strategies, defining age- and sex-specific criteria for OSA, evaluating routine preoperative screening for OSA, evaluating involvement of a sleep medicine specialist in diagnosis of OSA, evaluating clinical prediction rules, assessing the effect of treating sleep disordered breathing and long-term clinical outcomes, comparing treatments for patients who do not tolerate positive airway pressure, evaluating strategies to improve treatment compliance, and evaluating the association between sleep apnea severity and long-term clinical outcomes. CONCLUSIONS: While there are numerous specific research questions with low or insufficient strength of evidence for OSA management, OSA patients, their healthcare providers, and society at large would benefit from refocusing research efforts into the prioritized research questions and away from simple comparisons of short-term outcomes between specific interventions.

Willingness-to-pay for predictive tests with no immediate treatment implications: a survey of US residents.

We assessed how much, if anything, people would pay for a laboratory test that predicted their future disease status. A questionnaire was administered via an internet-based survey to a random sample of adult US respondents. Each respondent answered questions about two different scenarios, each of which specified: one of four randomly selected diseases (Alzheimer's, arthritis, breast cancer, or prostate cancer); an ex ante risk of developing the disease (randomly designated 10 or 25%); and test accuracy (randomly designated perfect or 'not perfectly accurate'). Willingness-to-pay (WTP) was elicited with a double-bounded, dichotomous-choice approach. Of 1463 respondents who completed the survey, most (70-88%, depending on the scenario) were inclined to take the test. Inclination to take the test was lower for Alzheimer's and higher for prostate cancer compared with arthritis, and rose somewhat with disease prevalence and for the perfect versus imperfect test [Correction made here after initial online publication.]. Median WTP varied from $109 for the imperfect arthritis test to $263 for the perfect prostate cancer test. Respondents' preferences for predictive testing, even in the absence of direct treatment consequences, reflected health and non-health related factors, and suggests that conventional cost-effectiveness analyses may underestimate the value of testing.

What do researchers say about proposed Medicare claims public use files?

BACKGROUND: Medicare claims data sets have been used widely in outcomes research over the last decade. In 2010, the Centers for Medicare & Medicaid Services established a program to create de-identified basic standalone public use files (PUFs) from Medicare claims data, each one containing claims information from a 5% sample of beneficiaries. METHODS: We conducted a series of structured key informant interviews with research stakeholders to compile recommendations that would guide the creation of these PUFs. In this paper, we describe the interview methodology and present our findings. Fifteen researchers, representing a range of clinical health services and health policy expertise, were interviewed. RESULTS: All respondents supported the use of Medicare claims in comparative effectiveness research and responded favorably to the creation of PUFs for this purpose. The interviews resulted in administrative-, technical- and content-related recommendations, some of which led to important changes in the PUFs. DISCUSSION: A primary trade-off in the development of the proposed PUFs involved assuring maximum research utility of the files while assuring security of beneficiaries' protected health information. Protection of protected health information was considered a requirement. Given this constraint, the proposed PUFs may be most useful for two primary activities in comparative effectiveness research: first, working through the beginning stages of a research project; and second, examining high-level questions.

Comparative effectiveness of ST-segment-elevation myocardial infarction regionalization strategies.

BACKGROUND: Primary percutaneous coronary intervention (PCI) is more effective on average than fibrinolytic therapy in the treatment of ST-segment-elevation myocardial infarction. Yet, most US hospitals are not equipped for PCI, and fibrinolytic therapy is still widely used. This study evaluated the comparative effectiveness of ST-segment-elevation myocardial infarction regionalization strategies to increase the use of PCI against standard emergency transport and care. METHODS AND RESULTS: We estimated incremental treatment costs and quality-adjusted life expectancies of 2000 patients with ST-segment-elevation myocardial infarction who received PCI or fibrinolytic therapy in simulations of emergency care in a regional hospital system. To increase access to PCI across the system, we compared a base case strategy with 12 hospital-based strategies of building new PCI laboratories or extending the hours of existing laboratories and 1 emergency medical services-based strategy of transporting all patients with ST-segment-elevation myocardial infarction to existing PCI-capable hospitals. The base case resulted in 609 (95% CI, 569-647) patients getting PCI. Hospital-based strategies increased the number of patients receiving PCI, the costs of care, and quality-adjusted life years saved and were cost-effective under a variety of conditions. An emergency medical services-based strategy of transporting every patient to an existing PCI facility was less costly and more effective than all hospital expansion options. CONCLUSION: Our results suggest that new construction and staffing of PCI laboratories may not be warranted if an emergency medical services strategy is both available and feasible.

Elapsed time in emergency medical services for patients with cardiac complaints: are some patients at greater risk for delay?

BACKGROUND: In patients with a major cardiac event, the first priority is to minimize time to treatment. For many patients, first contact with the health system is through emergency medical services (EMS). We set out to identify patient-level and neighborhood-level factors that were associated with elapsed time in EMS. METHODS AND RESULTS: A retrospective cohort study was conducted in 10 municipalities in Dallas County, Tex, from January 1 through December 31, 2004. The data set included 5887 patients with suspected cardiac-related symptoms. The region was served by 29 hospitals and 98 EMS depots. Multivariate models included measures of distance traveled, time of day, day of week, and patient and neighborhood characteristics. The main outcomes were elapsed time in EMS (continuous; in minutes) and delay in EMS (dichotomous; >15 minutes beyond median elapsed time). We found positive associations between patient characteristics and both average elapsed time and delay in EMS care. Variation in average elapsed time was not large enough to be clinically meaningful. However, approximately 11% (n=647) of patients were delayed >or=15 minutes. Women were more likely to be delayed (adjusted odds ratio, 1.52; 95% confidence interval, 1.32 to 1.74), and this association did not change after adjusting for other characteristics, including neighborhood socioeconomic composition. CONCLUSIONS: Compared with otherwise similar men, women have 50% greater odds of being delayed in the EMS setting. The determinants of delay should be a special focus of EMS studies in which time to treatment is a priority.