Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of 15 members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to 2 rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by 6 phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?

Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of fifteen members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to two rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by six phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Examining the impact of different country processes for appraising rare disease treatments: a case study analysis.

BACKGROUND: Conventional appraisal and reimbursement processes are being challenged by the increasing number of rare disease treatments (RDTs) with a small evidence base and often a high price. Processes to appraise RDTs vary across countries; some use standard processes, others have separate processes or adapted processes that explicitly deal with rare disease specificities. The objective of this study was to examine the impacts of different appraisal processes for two RDTs. METHODS: A case study analysis was conducted using countries with different forms of appraisal processes for RDTs for which public health technology assessment (HTA) reports were available. Two contrasting RDTs were chosen according to the criteria: rare versus ultra-rare treatment, affecting child versus adult, life-threatening versus disabling. Information from public HTA reports for each country's RDT appraisal was extracted into templates, allowing a systematic comparison of the appraisals across countries and identification of the impact of the different processes in practice. RESULTS: Reports from Belgium, England, France, Germany, Italy, Netherlands, Norway, Scotland, Sweden, and the USA were selected for nusinersen (for spinal muscular atrophy) and voretigene neparvovec (for inherited retinal disorders). Countries with separate or adapted processes had more consistent approaches for managing RDT-related issues during appraisal, such as stakeholder involvement and criteria to address the specificities of RDTs, creating more transparency in decision-making. CONCLUSIONS: Findings suggest that separate or adapted approaches for RDT appraisal may facilitate more structured, consistent decision-making and better management of RDT specificities.

A framework for action to improve patient and public involvement in health technology assessment.

BACKGROUND: Patient and public involvement (PPI) in the Brazilian Health Technology Assessment (HTA) process occurs in response to a legislative mandate for "social participation." This resulted in some limited patient participation activities, and, therefore, a more systematic approach was needed. The study describes the development of a suggested framework for action to improve PPI in HTA. METHODS: This work used formal methodology to develop a PPI framework based on three-phase mixed-methods research with desktop review of Brazilian PPI activities in HTA; workshop, survey, and interviews with Brazilian stakeholders; and a rapid review of international practices to enact effective patient involvement. Patient partners reviewed the draft framework. RESULTS: According to patient group representatives, their involvement in the Brazilian HTA process is important but could be improved. Different stakeholders perceived barriers, identified values, and made suggestions for improvement, such as expansion of communication, capacity building, and transparency, to support more meaningful patient involvement. The international practices identified opportunities for earlier, more active, and collaborative PPI during all HTA stages, based on values and principles that are relevant for Brazilian patients and the public. These findings were synthesized to design a framework that defines and systematizes actions to support PPI in Brazil, highlighting the importance of evaluating these strategies. CONCLUSIONS: Since the publication of this framework, some of its suggestions are being implemented in the Brazilian HTA process to improve PPI. We encourage other HTA organizations to consider a systematic and planned approach with regular evaluation when pursuing or strengthening involvement practices.

Real-world evidence to support Payer/HTA decisions about highly innovative technologies in the EU-actions for stakeholders.

OBJECTIVES: There are divergent views on the potential of real-world data (RWD) to inform decisions made by regulators, health technology assessment (HTA) bodies, payers, clinicians, and patients. This RWE4Decisions initiative explored the particularly challenging setting of highly innovative technologies, which require Payers/HTAs to make decisions on a small evidence base with major uncertainties. The aim was to go beyond strategic intent to consider actions that each stakeholder could take to improve use of RWD in this setting. RESULTS: Case studies of recent Payer/HTA decisions about highly innovative technologies were considered in light of recent international initiatives about RWD. This showed a lack of clarity about the Payer/HTA questions that could be answered by RWD and how the quality of real-world evidence (RWE) could be assessed. All stakeholders worked together to create a vision whereby stakeholders agree what RWD can be collected for highly innovative technologies based on principles of collaboration and transparency. For each stakeholder group, recommended actions to support the generation, analysis, and interpretation of RWD to inform decision making were developed. For HTA bodies, this includes cross border HTA/regulatory collaboration to agree RWD requirements over the technology life cycle to inform initial recommendations and reassessment, data analytics methods development for HTA, and promotion of transparency in RWE studies. RECOMMENDATIONS: Stakeholders need to collaborate on demonstration projects to consider how RWE can be developed to inform healthcare decisions and contribute to a learning network that can develop systems to support a learning health system and improve patient outcomes through best use of RWD.

Stories of Patient Involvement Impact in Health Technology Assessments: A Discussion Paper.

OBJECTIVES: As more health technology assessment (HTA) bodies seek to implement patient involvement, there is a desire to learn from other HTA bodies about their experiences and understand what approaches can be used and which ones make a real difference to HTA. This is difficult, as the impact of patient involvement in HTA is not well documented. This study aims to promote further discussion about the ways in which patient involvement can impact HTAs by studying stories of impact. METHODS: In a multi-stakeholder workshop, experts leading patient involvement in four HTA bodies shared examples of HTAs where they believed patient involvement made a difference, then they reflected on these impact stories within the wider context of impact evaluation. RESULTS: The HTA bodies drew on patient input and patient-based evidence to inform their HTAs. The patient involvement was observed to elucidate patients' experiences, needs and preferences which, in turn, was observed to influence the HTA recommendations about optimal use of technologies, including taking account of issues for sub-groups, outcomes that matter to patients and educational needs. CONCLUSIONS: Personal stories of patient involvement may enable a wider understanding of different approaches to and impact of patient involvement. The examples relate to both patient input and patient-based evidence and highlight the role that patient involvement can play in reducing uncertainties and complementing the clinical and economic evidence in HTA. They suggest that impact can be seen in recommendations about how and when a technology is used.

EVALUATION OF PATIENT AND PUBLIC INVOLVEMENT INITIATIVES IN HEALTH TECHNOLOGY ASSESSMENT: A SURVEY OF INTERNATIONAL AGENCIES.

OBJECTIVES: Although there is increased awareness of patient and public involvement (PPI) among health technology assessment (HTA) organizations, evaluations of PPI initiatives are relatively scarce. Our objective as members of Health Technology Assessment International's (HTAi's) Patient and Citizen Involvement Group (PCIG) was to advance understanding of the range of evaluation strategies adopted by HTA organizations and their potential usefulness. METHODS: In March 2016, a survey was sent to fifty-four HTA organizations through the International Network of Agencies for Health Technology Assessment (INAHTA) and contacts of members of HTAi's PCIG. Respondents were asked about their organizational structure; how patients and members of the public are involved; whether and how PPI initiatives have been evaluated, and, if so, which facilitators and challenges to evaluation were found and how results were used and disseminated. RESULTS: Fifteen (n = 15) programs from twelve countries responded (response rate 27.8 percent) that involved patients (14/15) and members of the public (10/15) in HTA activities. Seven programs evaluated their PPI activities, including participant satisfaction (5/7), process (5/7) and impact evaluations (4/7). Evaluation results were used to improve PPI activities, identify education and training needs, and direct strategic priorities. Facilitators and challenges revolved around the need for stakeholder buy-in, sufficient resources, senior leadership, and including patients in evaluations. CONCLUSIONS: A small but diverse set of HTA organizations evaluate their PPI activities using a range of strategies that reflect the range of rationales and approaches to PPI in HTA. It will be important for HTA organizations to draw on evaluation theories and methods.

IMPROVING THE EFFECTIVENESS AND EFFICIENCY OF EVIDENCE PRODUCTION FOR HEALTH TECHNOLOGY ASSESSMENT.

OBJECTIVES: Health Technology Assessment (HTA) needs to address the challenges posed by high cost, effective technologies, expedited regulatory approaches, and the opportunities provided by collaborative real-world evaluation of technologies. The Health Technology Assessment International (HTAi) Policy Forum met to consider these issues and the implications for evidence production to inform HTA. This paper shares their discussion to stimulate further debate. METHODS: A background paper, presentations, group discussions, and stakeholder role play at the 2015 HTAi Policy Forum meeting informed this paper. RESULTS: HTA has an important role to play in helping improve evidence production and ensuring that the health service is ready to adopt effective technologies. It needs to move from simply informing health system decisions to also working actively to align stakeholder expectations about realistic evidence requirements. Processes to support dialogue over the health technology life cycle need to be developed that are mindful of limited resources, operate across jurisdictions and learn from past processes. Collaborations between health technology developers and health systems in different countries should be encouraged to develop evidence that will inform decision making. New analytical techniques emerging for real-world data should be harnessed to support modeling for HTA. CONCLUSIONS: A paradigm shift (to "Health Innovation System 2.0") is suggested where HTA adopts a more central, proactive role to support alignment within and amongst stakeholders over the whole life cycle of the technology. This could help ensure that evidence production is better aligned with patient and health system needs and so is more effective and efficient.

Generating health technology assessment evidence for rare diseases.

OBJECTIVES: Rare diseases are often heterogeneous in their progression and response to treatment, with only a small population for study. This provides challenges for evidence generation to support HTA, so novel research methods are required. METHODS: Discussion with an expert panel was augmented with references and case studies to explore robust approaches for HTA evidence generation for rare disease treatments. RESULTS: Traditional RCTs can be modified using sequential, three-stage or adaptive designs to gain more power from a small patient population or to focus trial design. However, such designs need to maintain important design aspects such as randomization and blinding and be analyzed to take account of the multiple analyses performed. N-of-1 trials use within-patient randomization to test repeat periods of treatment and control until a response is clear. Such trials could be particularly valuable for rare diseases and when prospectively planned across several patients and analyzed using Bayesian techniques, a population effect can be estimated that might be of value to HTA. When the optimal outcome is unclear in a rare disease, disease specific patient reported outcomes can elucidate impacts on patients' functioning and wellbeing. Likewise, qualitative research can be used to elicit patients' perspectives, with just a small number of patients. CONCLUSIONS: International consensus is needed on ways to improve evidence collection and assessment of technologies for rare diseases, which recognize the value of novel study designs and analyses in a setting where the outcomes and effects of importance are yet to be agreed.

Improving Interpretation of Evidence Relating to Quality of Life in Health Technology Assessments of Rare Disease Treatments.

Rare diseases are often severe, debilitating, life-limiting conditions, many of which occur in childhood. These complex conditions have a wide range of clinical manifestations that have a substantial impact on the lives of patients, carers and families and often produce heterogeneous clinical outcomes. Therefore, the evaluation of quality-of-life (QoL) impacts is important. In health technology assessment (HTA), patient-reported outcome measures (PROMs) and/or health state utility values (HSUVs) are used to determine QoL impacts of new treatments, but their use in rare diseases is challenging due to small and heterogeneous populations and limited disease knowledge. This paper describes challenges associated with the use of patient-reported outcomes (PROs)/HSUVs to evaluate QoL in HTA of rare disease treatments (RDTs) and identifies five recommendations to ensure appropriate interpretation of QoL impacts. These were derived from mixed methods research (literature reviews, appraisal document analyses, appraisal committee observations and interviews) examining the use of PROs/HSUVs in HTA of RDTs. They highlight that HTAs of RDTs must (1) understand the QoL impacts of the disease and of treatments; (2) critically assess PRO data, recognising the nuances in development and administration of PROMs/HSUVs, considering what is feasible and what matters most to the patient population; (3) recognise that lack of significant effect on a PRO does not imply no QoL benefit; (4) use different forms of evidence to understand QoL impacts, such as patient input; and (5) provide methodological guidance to capture QoL impacts on patients/carers.

Advancing the use of real world evidence in health technology assessment: insights from a multi-stakeholder workshop.

Introduction: Real-world evidence (RWE) in health technology assessment (HTA) holds significant potential for informing healthcare decision-making. A multistakeholder workshop was organised by the European Health Data and Evidence Network (EHDEN) and the GetReal Institute to explore the status, challenges, and opportunities in incorporating RWE into HTA, with a focus on learning from regulatory initiatives such as the European Medicines Agency (EMA) Data Analysis and Real World Interrogation Network (DARWIN EU®). Methods: The workshop gathered key stakeholders from regulatory agencies, HTA organizations, academia, and industry for three panel discussions on RWE and HTA integration. Insights and recommendations were collected through panel discussions and audience polls. The workshop outcomes were reviewed by authors to identify key themes, challenges, and recommendations. Results: The workshop discussions revealed several important findings relating to the use of RWE in HTA. Compared with regulatory processes, its adoption in HTA to date has been slow. Barriers include limited trust in RWE, data quality concerns, and uncertainty about best practices. Facilitators include multidisciplinary training, educational initiatives, and stakeholder collaboration, which could be facilitated by initiatives like EHDEN and the GetReal Institute. Demonstrating the impact of "driver projects" could promote RWE adoption in HTA. Conclusion: To enhance the integration of RWE in HTA, it is crucial to address known barriers through comprehensive training, stakeholder collaboration, and impactful exemplar research projects. By upskilling users and beneficiaries of RWE and those that generate it, promoting collaboration, and conducting "driver projects," can strengthen the HTA evidence base for more informed healthcare decisions.

Consideration of quality of life in the health technology assessments of rare disease treatments.

OBJECTIVES: Challenges with patient-reported outcome (PRO) evidence and health state utility values (HSUVs) in rare diseases exist due to small, heterogeneous populations, lack of disease knowledge and early onset. To better incorporate quality of life (QoL) into Health Technology Assessment, a clearer understanding of these challenges is needed. METHODS: NICE appraisals of non-oncology treatments with an EMA orphan designation (n = 24), and corresponding appraisals in the Netherlands, France, and Germany were included. Document analysis of appraisal reports investigated how PROs/HSUVs influenced decision-making and was representative of QoL impact of condition and treatment. RESULTS: PRO evidence was not included in 6/24 NICE appraisals. When included, it either failed to demonstrate change, capture domains important for patients, or was uncertain. In the other countries, little information was reported and evidence largely did not demonstrate change. In NICE appraisals, HSUVs were derived through the collection of EQ-5D data (7/24 cases), mapping (6/24), vignettes (5/24), and published literature or other techniques (6/24). The majority did not use data collected alongside clinical trials. Few measures demonstrated significant change due to lack of sensitivity or face validity, short-term data, or implausible health states. In 8/24 NICE appraisals, patient surveys or input during appraisal committee meetings supported the interpretation of uncertainty or provided evidence about QoL. CONCLUSIONS: This study sheds light on the nature of PRO evidence in rare diseases and associated challenges. Results emphasise the need for improved development and use of PRO/HSUVs. Other forms of evidence and expert input are crucial to support better appraisal of uncertain or missing evidence.

Factors Affecting Citizen Trust and Public Engagement Relating to the Generation and Use of Real-World Evidence in Healthcare.

The potential for the use of real-world data (RWD) to generate real-world evidence (RWE) that can inform clinical decision-making and health policy is increasingly recognized, albeit with hesitancy in some circles. If used appropriately, the rapidly expanding wealth of health data could improve healthcare research, delivery of care, and patient outcomes. However, this depends on two key factors: (1) building structures that increase the confidence and willingness of European Union (EU) citizens to permit the collection and use of their data, and (2) development of EU health policy to support and shape data collection infrastructures, methodologies, transmission, and use. The great potential for use of RWE in healthcare improvement merits careful exploration of the drivers of, and challenges preventing, efficient RWD curation. Literature-based research was performed to identify relevant themes and discussion topics for two sets of expert panels, organized by the European Alliance for Personalised Medicine. These expert panels discussed steps that would enable a gradual but steady growth in the quantity, quality, and beneficial deployment of RWE. Participants were selected to provide insight based on their professional medical, economic, patient, industry, or governmental experience. Here, we propose a framework that addresses public trust and access to data, cross-border governance, alignment of evidence frameworks, and demonstrable improvements in healthcare decisions. We also discuss key case studies that support these recommendations, in accordance with the discussions at the expert panels.

What principles should govern the use of managed entry agreements?

BACKGROUND: To ensure rapid access to new potentially beneficial health technologies, obtain best value for money, and ensure affordability, healthcare payers are adopting a range of innovative reimbursement approaches that may be called Managed Entry Agreements (MEAs). METHODS: The Health Technology Assessment International (HTAi) Policy Forum sought to identify why MEAs might be used, issues associated with implementation and develop principles for their use. A 2-day deliberative workshop discussed key papers, members' experiences, and collectively addressed four policy questions that resulted in this study. RESULTS: MEAs are used to give access to new technologies where traditional reimbursement is deemed inappropriate. Three different forms of MEAs have been identified: management of budget impact, management of uncertainty relating to clinical and/or cost-effectiveness, and management of utilization to optimize performance. The rationale for using these approaches and their advantages and disadvantages differ. However, all forms of MEA should take the form of a formal written agreement among stakeholders, clearly identifying the rationale for the agreement, aspects to be assessed, methods of data collection and review, and the criteria for ending the agreement. CONCLUSIONS: MEAs should only be used when HTA identifies issues or concerns about key outcomes and/or costs and/or organizational/budget impacts that are material to a reimbursement decision. They provide patient access and can be useful to manage technology diffusion and optimize use. However, they are administratively complex and may be difficult to negotiate and their effectiveness has yet to be evaluated.

The Evolution of AIFA Registries to Support Managed Entry Agreements for Orphan Medicinal Products in Italy.

Italy has a well-established prominent system of national registries to support managed entry agreements (MEAs), monitoring innovative medicinal products (MPs) with clinical as well as economic uncertainties to ensure appropriate use and best value for money. The technological architecture of the registries is funded by pharmaceutical companies, but fully governed by the national medicines agency (AIFA). A desktop analysis was undertaken of data over a 15-year timeframe of all AIFA indication-based registries and associated EMA information. The characteristics of registries were evaluated, comparing orphan MPs vs. all MPs exploring cancer and non-cancer indications. OMP (orphan medicinal product) registries' type vs. AIFA innovation status and EMA approval was reviewed. Of the 283 registries, 182 are appropriateness registries (35.2% relate to OMPs, with an almost equal split of cancer vs. non-cancer for OMPs and MPs), 35 include financial-based agreements [20% OMPs (2 non-cancer, 5 cancer)], and 60 registries are payment by result agreements [23.3% OMPs (4 non-cancer, 10 cancer)]. Most OMPs (53/88) came through the normal regulatory route. With the strengthening of the system for evaluation of innovation, fewer outcomes-based registries have been instigated. AIFA has overcome many of the challenges experienced with MEA through developing an integrated national web-based data collection system: the challenge that remains for AIFA is to move from using the system for individual patient decisions about treatment to reviewing the wealth of data it now holds to optimize healthcare.

Implementing Outcomes-Based Managed Entry Agreements for Rare Disease Treatments: Nusinersen and Tisagenlecleucel.

BACKGROUND AND OBJECTIVE: Enthusiasm for the use of outcomes-based managed entry agreements (OBMEAs) to manage uncertainties apparent at the time of appraisal/pricing and reimbursement of new medicines has waned over the past decade, as challenges in establishment, implementation and re-appraisal have been identified. With the recent advent of innovative treatments for rare diseases that have uncertainties in the clinical evidence base, but which could meet a high unmet need, there has been renewed interest in the potential of OBMEAs. The objective of this research was to review the implementation of OBMEAs for two case studies across countries in the European Union, Australia and Canada, to identify good practices that could inform development of tools to support implementation of OBMEAs. METHODS: To investigate how OBMEAs are being implemented with rare disease treatments, we collected information from health technology assessment/payer experts in countries that had implemented OBMEAs for either nusinersen in spinal muscular atrophy or tisagenlecleucel in two cancer indications. Operational characteristics of the OBMEAs that were publicly available were documented. Then, the experts discussed issues in implementing these OBMEAs and specific approaches taken to overcome challenges. RESULTS: The OBMEAs identified were based on individual outcomes to ensure appropriate use, manage continuation of treatment and in two cases linked to payment schedules, or they were population based, coverage with evidence development. For nusinersen, population-based OBMEAs are documented in Belgium, England and the Netherlands and individual-based schemes in Bulgaria, Ireland, Italy and Lithuania. For tisagenlecleucel, there were population-based schemes in Australia, Belgium, England and France and individual-based schemes in Italy and Spain. Comparison of the OBMEA constructs showed some clear published frameworks and clarity of the uncertainties to be addressed that were similar across countries. Agreements were generally made between the marketing authorisation holder and the payer with involvement of expert physicians. Only England and the Netherlands involved patients. Italy used its long-established, national, web-based, treatment-specific data collection system linked to reimbursement and Spain has just developed such a national treatment registry system. Other countries relied on a variety of data collection systems (including clinical registries) and administrative data. Durations of agreements varied for these treatments as did processes for interim reporting. The processes to ensure data quality, completeness and sufficiency for re-analysis after coverage with evidence development were not always clear, neither were analysis plans. CONCLUSIONS: These case studies have shown that important information about the constructs of OBMEAs for rare disease treatments are publicly available, and for some jurisdictions, interim reports of progress. Outcomes-based managed entry agreements can play an important role not only in reimbursement, but also in treatment optimisation. However, they are complex to implement and should be the exception and not the rule. More recent OBMEAs have developed document covenants among stakeholders or electronic systems to provide assurances about data sufficiency. For coverage with evidence development, there is an opportunity for greater collaboration among jurisdictions to share processes, develop common data collection agreements, and share interim and final reports. The establishment of an international public portal to host such reports would be particularly valuable for rare disease treatments.

The use of nonrandomized evidence to estimate treatment effects in health technology assessment.

Health technology assessment (HTA) is increasingly informed by nonrandomized studies, but there is limited guidance from HTA bodies on expectations around evidence quality and study conduct. We developed recommendations to support the appropriate use of such evidence based on a pragmatic literature review and a workshop involving 16 experts from eight countries as part of the EU's Horizon-2020 IMPACT-HTA program (work package six). To ensure HTA processes remain rigorous and robust, HTA bodies should demand clear, extensive and structured reporting of nonrandomized studies, including an in-depth assessment of the risk of bias. In recognition of the additional uncertainty imparted by nonrandomized designs in estimates of treatment effects, HTA bodies should strengthen early scientific advice and engage in collaborative efforts to improve use of real-world data.

Health technology assessment to optimize health technology utilization: using implementation initiatives and monitoring processes.

BACKGROUND: The way in which a health technology is used in any particular health system depends on the decisions and actions of a variety of stakeholders, the local culture, and context. In 2009, the HTAi Policy Forum considered how health technology assessment (HTA) could be improved to optimize the use of technologies (in terms of uptake, change in use, or disinvestment) in such complex systems. METHODS: In scoping, it was agreed to focus on initiatives to implement evidence-based guidance and monitoring activities. A review identified systematic reviews of implementation initiatives and monitoring activities. A two-day deliberative workshop was held to discuss key papers, members' experiences, and collectively address key questions. This consensus paper was developed by email and finalized at a postworkshop meeting. RESULTS: Evidence suggests that the impact and use of HTA could be increased by ensuring timely delivery of relevant reports to clearly determined policy receptor (decision-making) points. To achieve this, the breadth of assessment, implementation initiatives such as incentives and targeted, intelligent dissemination of HTA result, needs to be considered. HTA stakeholders undertake a variety of monitoring activities, which could inform optimal use of a technology. However, the quality of these data varies and is often not submitted to an HTA. CONCLUSIONS: Monitoring data should be sufficiently robust so that they can be used in HTA to inform optimal use of technology. Evidence-based implementation initiatives should be developed for HTA, to better inform decision makers at all levels in a health system about the optimal use of technology.

Patients' perspectives in health technology assessment: a route to robust evidence and fair deliberation.

There is increasing emphasis on providing patient-focused health care and ensuring patient involvement in the design of health services. As health technology assessment (HTA) is meant to be a multidisciplinary, wide-ranging policy analysis that informs decision making, it would be expected that patients' views should be incorporated into the assessment. However, HTA is still driven by collection of quantitative evidence to determine the clinical and cost effectiveness of a health technology. Patients' perspectives about their illness and the technology are rarely included, perhaps because they are seen as anecdotal, biased views. There are two distinct but complementary ways in which HTAs can be strengthened by: (i) gathering robust evidence about the patients' perspectives, and (ii) ensuring effective engagement of patients in the HTA process from scoping, through evidence gathering, assessment of value, development of recommendations and dissemination of findings. Robust evidence eliciting patients' perspectives can be obtained through social science research that is well conducted, critically appraised and carefully reported, either through meta-synthesis of existing studies or new primary research. Engagement with patients can occur at several levels and we propose that HTA should seek to support effective patient participation to create a fair deliberative process. This should allow two-way flow of information, so that the views of patients are obtained in a supportive way and fed into decision-making processes in a transparent manner.