Uncertainty management in regulatory and health technology assessment decision-making on drugs: guidance of the HTAi-DIA Working Group.

OBJECTIVES: Uncertainty is a fundamental component of decision making regarding access to and pricing and reimbursement of drugs. The context-specific interpretation and mitigation of uncertainty remain major challenges for decision makers. Following the 2021 HTAi Global Policy Forum, a cross-sectoral, interdisciplinary HTAi-DIA Working Group (WG) was initiated to develop guidance to support stakeholder deliberation on the systematic identification and mitigation of uncertainties in the regulatory-HTA interface. METHODS: Six online discussions among WG members (Dec 2021-Sep 2022) who examined the output of a scoping review, two literature-based case studies and a survey; application of the initial guidance to a real-world case study; and two international conference panel discussions. RESULTS: The WG identified key concepts, clustered into twelve building blocks that were collectively perceived to define uncertainty: "unavailable," "inaccurate," "conflicting," "not understandable," "random variation," "information," "prediction," "impact," "risk," "relevance," "context," and "judgment." These were converted into a checklist to explain and define whether any issue constitutes a decision-relevant uncertainty. A taxonomy of domains in which uncertainty may exist within the regulatory-HTA interface was developed to facilitate categorization. The real-world case study was used to demonstrate how the guidance may facilitate deliberation between stakeholders and where additional guidance development may be needed. CONCLUSIONS: The systematic approach taken for the identification of uncertainties in this guidance has the potential to facilitate understanding of uncertainty and its management across different stakeholders involved in drug development and evaluation. This can improve consistency and transparency throughout decision processes. To further support uncertainty management, linkage to suitable mitigation strategies is necessary.

Exploring the opportunities for alignment of regulatory postauthorization requirements and data required for performance-based managed entry agreements.

Performance-based managed entry agreements (PB-MEAs) might allow patient access to new medicines, but practical hurdles make competent authorities for pricing and reimbursement (CAPR) reluctant to implement PB-MEAs. We explored if the feasibility of PB-MEAs might improve by better aligning regulatory postauthorization requirements with the data generation of PB-MEAs and by active collaboration and data sharing. Reviewers from seven CAPRs provided structured assessments of the information available at the European Medicines Agency (EMA) Web site on regulatory postauthorization requirements for fifteen recently authorized products. The reviewers judged to what extent regulatory postauthorization studies could help implement PB-MEAs by addressing uncertainty gaps. Study domains assessed were: patient population, intervention, comparators, outcomes, time horizon, anticipated data quality, and anticipated robustness of analysis. Reviewers shared general comments about PB-MEAs for each product and on cooperation with other CAPRs. Reviewers rated regulatory postauthorization requirements at least partly helpful for most products and across domains except the comparator domain. One quarter of responses indicated that public information provided by the EMA was insufficient to support the implementation of PB-MEAs. Few PB-MEAs were in place for these products, but the potential for implementation of PB-MEAs or collaboration across CAPRs was seen as more favorable. Responses helped delineate a set of conditions where PB-MEAs may help reduce uncertainty. In conclusion, PB-MEAs are not a preferred option for CAPRs, but we identified conditions where PB-MEAs might be worth considering. The complexities of implementing PB-MEAs remain a hurdle, but collaboration across silos and more transparency on postauthorization studies could help overcome some barriers.

Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Improving transparency to build trust in real-world secondary data studies for hypothesis testing-Why, what, and how: recommendations and a road map from the real-world evidence transparency initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

The evolution of clinical trials: Can we address the challenges of the future?

The authors describe key challenges facing the clinical trials community and propose solutions to these issues, including the role the Clinical Trials Transformation Initiative can play in addressing these issues. Specifically, the authors reflect on clinical trial globalization and the harmonization of frameworks and requirements across regions; the challenges associated with balancing the desire for external validity, pragmatic trials, and precision medicine; clinical trial transparency; and operational complexity and the expense of clinical trials. By addressing these challenges, future clinical trials will be more feasible, relevant, and credible, and support both the continuing altruistic contributions of patients and the collection of more meaningful data.

Emerging technologies for food and drug safety.

Emerging technologies are playing a major role in the generation of new approaches to assess the safety of both foods and drugs. However, the integration of emerging technologies in the regulatory decision-making process requires rigorous assessment and consensus amongst international partners and research communities. To that end, the Global Coalition for Regulatory Science Research (GCRSR) in partnership with the Brazilian Health Surveillance Agency (ANVISA) hosted the seventh Global Summit on Regulatory Science (GSRS17) in Brasilia, Brazil on September 18-20, 2017 to discuss the role of new approaches in regulatory science with a specific emphasis on applications in food and medical product safety. The global regulatory landscape concerning the application of new technologies was assessed in several countries worldwide. Challenges and issues were discussed in the context of developing an international consensus for objective criteria in the development, application and review of emerging technologies. The need for advanced approaches to allow for faster, less expensive and more predictive methodologies was elaborated. In addition, the strengths and weaknesses of each new approach was discussed. And finally, the need for standards and reproducible approaches was reviewed to enhance the application of the emerging technologies to improve food and drug safety. The overarching goal of GSRS17 was to provide a venue where regulators and researchers meet to develop collaborations addressing the most pressing scientific challenges and facilitate the adoption of novel technical innovations to advance the field of regulatory science.

Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making.

PURPOSE: Real-world evidence (RWE) includes data from retrospective or prospective observational studies and observational registries and provides insights beyond those addressed by randomized controlled trials. RWE studies aim to improve health care decision making. METHODS: The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) created a task force to make recommendations regarding good procedural practices that would enhance decision makers' confidence in evidence derived from RWD studies. Peer review by ISPOR/ISPE members and task force participants provided a consensus-building iterative process for the topics and framing of recommendations. RESULTS: The ISPOR/ISPE Task Force recommendations cover seven topics such as study registration, replicability, and stakeholder involvement in RWE studies. These recommendations, in concert with earlier recommendations about study methodology, provide a trustworthy foundation for the expanded use of RWE in health care decision making. CONCLUSION: The focus of these recommendations is good procedural practices for studies that test a specific hypothesis in a specific population. We recognize that some of the recommendations in this report may not be widely adopted without appropriate incentives from decision makers, journal editors, and other key stakeholders.

Definition of variables required for comprehensive description of drug dosage and clinical pharmacokinetics.

PURPOSE: Electronic clinical decision support systems (CDSS) require drug information that can be processed by computers. The goal of this project was to determine and evaluate a compilation of variables that comprehensively capture the information contained in the summary of product characteristic (SmPC) and unequivocally describe the drug, its dosage options, and clinical pharmacokinetics. METHODS: An expert panel defined and structured a set of variables and drafted a guideline to extract and enter information on dosage and clinical pharmacokinetics from textual SmPCs as published by the European Medicines Agency (EMA). The set of variables was iteratively revised and evaluated by data extraction and variable allocation of roughly 7% of all centrally approved drugs. RESULTS: The information contained in the SmPC was allocated to three information clusters consisting of 260 variables. The cluster "drug characterization" specifies the nature of the drug. The cluster "dosage" provides information on approved drug dosages and defines corresponding specific conditions. The cluster "clinical pharmacokinetics" includes pharmacokinetic parameters of relevance for dosing in clinical practice. A first evaluation demonstrated that, despite the complexity of the current free text SmPCs, dosage and pharmacokinetic information can be reliably extracted from the SmPCs and comprehensively described by a limited set of variables. CONCLUSION: By proposing a compilation of variables well describing drug dosage and clinical pharmacokinetics, the project represents a step forward towards the development of a comprehensive database system serving as information source for sophisticated CDSS.

Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making.

PURPOSE: Real-world evidence (RWE) includes data from retrospective or prospective observational studies and observational registries and provides insights beyond those addressed by randomized controlled trials. RWE studies aim to improve health care decision making. METHODS: The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) created a task force to make recommendations regarding good procedural practices that would enhance decision makers' confidence in evidence derived from RWD studies. Peer review by ISPOR/ISPE members and task force participants provided a consensus-building iterative process for the topics and framing of recommendations. RESULTS: The ISPOR/ISPE Task Force recommendations cover seven topics such as study registration, replicability, and stakeholder involvement in RWE studies. These recommendations, in concert with earlier recommendations about study methodology, provide a trustworthy foundation for the expanded use of RWE in health care decision making. CONCLUSION: The focus of these recommendations is good procedural practices for studies that test a specific hypothesis in a specific population. We recognize that some of the recommendations in this report may not be widely adopted without appropriate incentives from decision makers, journal editors, and other key stakeholders.

Facilitating Antibacterial Drug Development in a Time of Great Need.

The continued development of new antibacterial drugs is critical to meet patient and public health needs. In this editorial, authors from the US Food and Drug Administration and European Medicines Agency reflect on the role of public-private partnerships and the development of clinical trials networks as agents to guide and perform quality studies of antibacterial drugs.

Regulatory bioinformatics for food and drug safety.

"Regulatory Bioinformatics" strives to develop and implement a standardized and transparent bioinformatic framework to support the implementation of existing and emerging technologies in regulatory decision-making. It has great potential to improve public health through the development and use of clinically important medical products and tools to manage the safety of the food supply. However, the application of regulatory bioinformatics also poses new challenges and requires new knowledge and skill sets. In the latest Global Coalition on Regulatory Science Research (GCRSR) governed conference, Global Summit on Regulatory Science (GSRS2015), regulatory bioinformatics principles were presented with respect to global trends, initiatives and case studies. The discussion revealed that datasets, analytical tools, skills and expertise are rapidly developing, in many cases via large international collaborative consortia. It also revealed that significant research is still required to realize the potential applications of regulatory bioinformatics. While there is significant excitement in the possibilities offered by precision medicine to enhance treatments of serious and/or complex diseases, there is a clear need for further development of mechanisms to securely store, curate and share data, integrate databases, and standardized quality control and data analysis procedures. A greater understanding of the biological significance of the data is also required to fully exploit vast datasets that are becoming available. The application of bioinformatics in the microbiological risk analysis paradigm is delivering clear benefits both for the investigation of food borne pathogens and for decision making on clinically important treatments. It is recognized that regulatory bioinformatics will have many beneficial applications by ensuring high quality data, validated tools and standardized processes, which will help inform the regulatory science community of the requirements necessary to ensure the safe introduction and effective use of these applications.

Improving the contribution of regulatory assessment reports to health technology assessments--a collaboration between the European Medicines Agency and the European network for Health Technology Assessment.

In response to a recommendation from the Pharmaceutical Forum, the European Medicines Agency and the European network for Health Technology Assessment initiated a collaboration with the aim to improve the contribution regulatory assessment reports can make to the assessment of relative effectiveness of medicinal products by health technology assessment bodies. This collaboration on improving European Public Assessment Reports (EPARs) started in February 2010 and was performed over 2 years. As a result, the templates for preparing EPARs were revised to better address the needs of heath technology organizations. The better understanding of information needs was a key outcome of the collaboration. To ascertain whether these template changes led to the inclusion of relevant information, a review of a small set of EPARs for recently approved medicinal products was carried out in parallel by both the European network for Health Technology Assessment and the European Medicines Agency. This report provides an account of this project on improving EPARs, which is part of the ongoing dialogue between regulators and health technology assessment bodies on a European level to support policymaker decisions in the future.

Modelling the risk-benefit impact of H1N1 influenza vaccines.

BACKGROUND: Shortly after the H1N1 influenza virus reached pandemic status in June 2009, the benefit-risk project team at the European Medicines Agency recognized this presented a research opportunity for testing the usefulness of a decision analysis model in deliberations about approving vaccines soon based on limited data or waiting for more data. Undertaken purely as a research exercise, the model was not connected to the ongoing assessment by the European Medicines Agency, which approved the H1N1 vaccines on 25 September 2009. METHODS: A decision tree model constructed initially on 1 September 2009, and slightly revised subsequently as new data were obtained, represented an end-of-September or end-of-October approval of vaccines. The model showed combinations of uncertain events, the severity of the disease and the vaccines' efficacy and safety, leading to estimates of numbers of deaths and serious disabilities. The group based their probability assessments on available information and background knowledge about vaccines and similar pandemics in the past. RESULTS: Weighting the numbers by their joint probabilities for all paths through the decision tree gave a weighted average for a September decision of 216 500 deaths and serious disabilities, and for a decision delayed to October of 291 547, showing that an early decision was preferable. CONCLUSIONS: The process of constructing the model facilitated communications among the group's members and led to new insights for several participants, while its robustness built confidence in the decision. These findings suggest that models might be helpful to regulators, as they form their preferences during the process of deliberation and debate, and more generally, for public health issues when decision makers face considerable uncertainty.

Orphan drugs' clinical uncertainty and prices: Addressing allocative and technical inefficiencies in orphan drug reimbursement.

Legislations incentivising orphan drug development and scientific advances have made orphan drugs pharma's high-end favourite for the past two decades. Currently, around 50% of new marketing authorizations are for orphan drugs. For third-party healthcare payers ("payers") the rise of orphan drugs presents new challenges, including a high degree of uncertainty around clinical benefits and harms, a moderate effect size (for many orphan drugs), and a high price tag. The association of high clinical uncertainty and moderate effect sizes is not surprising in small target populations but in combination with high prices creates the risk of allocative and technical inefficiencies for payers. We here discuss and illustrate these risks. A combination of policies is needed for mitigation of allocative inefficiency: while there may be a rationale for higher prices for orphan than non-orphan drugs, a focus of pricing and reimbursement negotiations should include considerations of product profitability and of the consequences of orphan drug costs on the distribution inequality of medication costs for individual insured persons, coupled to knowledge generation from reimbursement contracts covering high-price orphan drugs that would benefit the wider patient community. Performance-based managed entry agreements could help to de-risk the economic consequences of clinical uncertainty and to mitigate technical inefficiency.

Patients, payers and developers of Orphan Medicinal Products: lessons learned from 10 years' multi-stakeholder dialogue on improving access in Europe via MoCA.

BACKGROUND: The Mechanism of Coordinated Access to Orphan Medicinal Products (MoCA) was established in 2013 with the intention of developing a coordinated mechanism between volunteering EU stakeholders and developers of Orphan Medicinal Products (OMPs) to support the exchange of information aimed at enabling informed decisions on pricing and reimbursement at Member State level and to evaluate the value of an OMP based on a Transparent Value Framework. The objective of the collaborative approach was to support more equitable access to authorised therapies for people living with rare diseases, rational prices for payers and more predictable market conditions for OMP developers. Over the past 10 years, the MoCA has conducted a series of pilot projects, examining a variety of different products and technologies at different stages of development; and with contributions from a variety of patient representatives, participation from EU payers from a range of Member States and, recently, with EUnetHTA members and the European Medicines Agency participating in the meetings as observers. RESULTS: 10 years on from the establishment of the MoCA, the European landscape has significantly evolved, not only in the field of drug development with increasingly transformative therapies based on novel technologies, but also in terms of larger numbers of approved treatments, increased budget impact and the resulting associated uncertainties; as well as in terms of stakeholder collaboration and interactions. The value of early dialogue with OMP developers, including the EU payer community via their national decision-making authorities, is a key element within this early interaction and contributes to identifying, managing and reducing uncertainties allowing a prospectively planned approach earlier in development and, consequently, to support more timely, sustainable and equitable access to new OMPs, particularly where there is a high unmet medical need. CONCLUSIONS: The voluntary, informal nature of the MoCA interactions creates a flexible framework for non-binding dialogue. A forum for such interactions is needed to achieve the aims of the MoCA and both to support healthcare systems in planning as well as to underpin timely, equitable and sustainable access to new therapies for patients with rare diseases within the EU.

Precision Reimbursement for Precision Medicine: Using Real-World Evidence to Evolve From Trial-and-Project to Track-and-Pay to Learn-and-Predict.

Basic scientists and drug developers are accelerating innovations toward the goal of precision medicine. Regulators create pathways for timely patient access to precision medicines, including individualized therapies. Healthcare payors acknowledge the need for change but downstream innovation for coverage and reimbursement is only haltingly occurring. Performance uncertainty, high price-tags, payment timing, and actuarial risk issues associated with precision medicines present novel financial challenges for payors. With traditional drug reimbursement frameworks, payment is based on an assumed randomized controlled trial (RCT) projection of real-world effectiveness, a "trial-and-project" strategy; the clinical benefit realized for patients is not usually ascertained ex post by collection of real-world data (RWD). To mitigate financial risks resulting from clinical performance uncertainty, manufacturers and payors devised "track-and-pay" frameworks (i.e., the tracking of a pre-agreed treatment outcome which is linked to financial consequences). Whereas some track-and-pay arrangements have been successful, inherent weaknesses include the potential for misalignment of incentives, the risk of channeling of patients, and a failure to use the RWD generated to enable continuous learning about treatments. "Precision reimbursement" (PR) intends to overcome inherent weaknesses of simple track-and-pay schemes. In combining the collection of RWD with advanced analytics (e.g., artificial intelligence and machine learning) to generate actionable real-world evidence, with prospective alignment of incentives across all stakeholders (including providers and patients), and with pre-agreed use and dissemination of information generated, PR becomes a "learn-and-predict" model of payment for performance. We here describe in detail the concept of PR and lay out the next steps to make it a reality.

A Useful and Sustainable Role for N-of-1 Trials in the Healthcare Ecosystem.

Clinicians and patients often try a treatment for an initial period to inform longer-term therapeutic decisions. A more rigorous approach involves N-of-1 trials. In these single-patient crossover trials, typically conducted in patients with chronic conditions, individual patients are given candidate treatments in a double-blinded, random sequence of alternating periods to determine the most effective treatment for that patient. However, to date, these trials are rarely done outside of research settings and have not been integrated into general care where they could offer substantial benefit. Designating this classical, N-of-1 trial design as type 1, there also are new and evolving uses of N-of-1 trials that we designate as type 2. In these, rather than focusing on optimizing treatment for chronic diseases when multiple approved choices are available, as is typical of type 1, a type 2 N-of-1 trial tests treatments designed specifically for a patient with a rare disease, to facilitate personalized medicine. While the aims differ, both types face the challenge of collecting individual-patient evidence using standard, trusted, widely accepted methods. To fulfill their potential for producing both clinical and research benefits, and to be available for wide use, N-of-1 trials will have to fit into the current healthcare ecosystem. This will require generalizable and accepted processes, platforms, methods, and standards. This also will require sustainable value-based arrangements among key stakeholders. In this article, we review opportunities, stakeholders, issues, and possible approaches that could support general use of N-of-1 trials and deliver benefit to patients and the healthcare enterprise. To assess and expand the benefits of N-of-1 trials, we propose multistakeholder meetings, workshops, and the generation of methods, standards, and platforms that would support wider availability and the value of N-of-1 trials.