Perspectives on Patient-Reported Outcome Data After Treatment Discontinuation in Cancer Clinical Trials.

OBJECTIVES: Patient-reported outcome (PRO) data are critical in understanding treatments from the patient perspective in cancer clinical trials. The potential benefits and methodological approaches to the collection of PRO data after treatment discontinuation (eg, because of progressive disease or unacceptable drug toxicity) are less clear. The purpose of this article is to describe the Food and Drug Administration's Oncology Center of Excellence and the Critical Path Institute cosponsored 2-hour virtual roundtable, held in 2020, to discuss this specific issue. METHODS: We summarize key points from this discussion with 16 stakeholders representing academia, clinical practice, patients, international regulatory agencies, health technology assessment bodies/payers, industry, and PRO instrument development. RESULTS: Stakeholders recognized that any PRO data collection after treatment discontinuation should have clearly defined objectives to ensure that data can be analyzed and reported. CONCLUSIONS: Data collection after discontinuation without a justification for its use wastes patients' time and effort and is unethical.

A systematic comparison of different composite measures (DAS 28, CDAI, SDAI, and Boolean approach) for determining treatment effects on low disease activity and remission in rheumatoid arthritis.

BACKGROUND: Some composite measures for determining the treatment effects of disease-modifying antirheumatic drugs on remission and low disease activity (LDA) in rheumatoid arthritis (RA) may produce misleading results if they include an acute phase reactant (APR). To inform the choice of appropriate measure, we performed a systematic comparison of treatment effects using different composite measures. METHODS: We used data generated for a systematic review of biologics in RA conducted by the Institute for Quality and Efficiency in Health Care and data from systematic reviews of newer biologics and Janus kinase (JAK) inhibitors provided by sponsors. The studies included had been conducted up to 2020 and investigated comparisons of biologics with placebo and head-to-head comparisons of biologics. Treatment effects on LDA and remission in studies investigating biologics or JAK inhibitors in RA were compared among 4 composite measures: the disease activity score 28 (DAS 28), the simplified disease activity index (SDAI), the Boolean approach (remission only), and the clinical disease activity index (CDAI)-only the latter does not include an APR. RESULTS: 49 placebo-controlled studies included 9 different biologics; 48 studies (16,233 patients) investigated LDA and 49 (16,338 patients) investigated remission. 11 active-controlled studies (5996 patients) investigated both LDA and remission and included 5 different head-to-head comparisons of biologics and 5 different comparisons (6 studies) of biologics with JAK inhibitors. Statistically significantly larger treatment effects were found for biologics or JAK inhibitors versus placebo or active control in 16% of pairwise comparisons of composite measures (27 of 168). Most of these larger effects were observed for composite measures with an APR, i.e. the DAS 28 (19 comparisons) followed by the SDAI (n = 7). Larger effects were most frequently detected in favour of interleukin (IL)-6 inhibitors and to a lesser extent for JAK inhibitors versus treatments with different modes of action. CONCLUSIONS: The use of the DAS 28 and SDAI in clinical studies may generate results favouring certain treatments based on their mode of action (e.g. IL-6 inhibitors versus other biologics). To enable unbiased comparative effectiveness research, a composite measure without an APR (i.e. the CDAI) should thus be the measure of choice.

Comparative effectiveness of biological medicines in rheumatoid arthritis: systematic review and network meta-analysis including aggregate results from reanalysed individual patient data.

OBJECTIVE: To assess the comparative effectiveness of biological medicines in rheumatoid arthritis in sufficiently similar patient populations, based on the current definitions of key outcomes. DESIGN: Systematic review and network meta-analysis including aggregate results from reanalysed individual patient data. DATA SOURCES: Clinical study reports and aggregate results from reanalyses of individual patient data on key outcomes for rheumatoid arthritis provided by study sponsors for studies conducted up to 2017, and several databases and registries from inception up to February 2017. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Randomised controlled trials investigating patient relevant outcomes in adults with rheumatoid arthritis treated with biological medicines in combination with methotrexate after methotrexate failure for at least 24 weeks. RESULTS: 45 eligible trials were identified. Combining data from clinical study reports and aggregate results from reanalyses of individual patient data allowed extensive analyses yielding sufficiently similar populations and homogeneous study results for network meta-analyses, including up to 35 studies on eight biological medicines combined with methotrexate. These analyses showed few statistically significant differences between the combination treatments. For example, anakinra showed less benefit than almost all the other seven biological medicines regarding clinical remission or low disease activity (clinical disease activity index ≤2.8 or ≤10, respectively) and certolizumab pegol showed more harm than the other seven biological medicines regarding serious adverse events or infections. Some outcomes had very wide 95% confidence intervals, potentially implying unidentified differences between the eight biological medicines, but wide 95% confidence intervals were less prominent for low disease activity, serious adverse events, and infections. Owing to a lack of head-to-head trials, results were mainly based on indirect comparisons with a limited number of studies, and recently approved Janus kinase inhibitors could not be included. CONCLUSIONS: For patients with rheumatoid arthritis after methotrexate failure, only minor differences in benefits and harms were seen between biological medicines in combination with methotrexate. However, the analysis was hampered by a lack of long term direct comparisons. The substantial information gain achieved by the reanalysis of individual patient data calls for the routine availability of individual patient data.

Generating comparative evidence on new drugs and devices before approval.

Fewer than half of new drugs have data on their comparative benefits and harms against existing treatment options at the time of regulatory approval in Europe and the USA. Even when active-comparator trials exist, they might not produce meaningful data to inform decisions in clinical practice and health policy. The uncertainty associated with the paucity of well designed active-comparator trials has been compounded by legal and regulatory changes in Europe and the USA that have created a complex mix of expedited programmes aimed at facilitating faster access to new drugs. Comparative evidence generation is even sparser for medical devices. Some have argued that the current process for regulatory approval needs to generate more evidence that is useful for patients, clinicians, and payers in health-care systems. We propose a set of five key principles relevant to the European Medicines Agency, European medical device regulatory agencies, US Food and Drug Administration, as well as payers, that we believe will provide the necessary incentives for pharmaceutical and device companies to generate comparative data on drugs and devices and assure timely availability of evidence that is useful for decision making. First, labelling should routinely inform patients and clinicians whether comparative data exist on new products. Second, regulators should be more selective in their use of programmes that facilitate drug and device approvals on the basis of incomplete benefit and harm data. Third, regulators should encourage the conduct of randomised trials with active comparators. Fourth, regulators should use prospectively designed network meta-analyses based on existing and future randomised trials. Last, payers should use their policy levers and negotiating power to incentivise the generation of comparative evidence on new and existing drugs and devices, for example, by explicitly considering proven added benefit in pricing and payment decisions.

Beyond journal publications - a new format for the publication of clinical trials.

Journal publications are the major route to communicate methods and results of clinical trials. However, the shortcomings of this format are well known, including insufficient quality of the information provided as well as publication and outcome reporting bias. Attempts to improve the situation via peer review, reporting guidelines or study registration did not solve the problem. Currently, new ways of data presentation in electronic databases, increased access to previously confidential documents, and the potential use of anonymized individual patient data from clinical trials beyond the individual trial, have led to discussions about new publication formats for clinical trials. The current paper describes the components required for full information on a clinical trial and discusses a new format to provide this information.

Assumptions of Mixed Treatment Comparisons in Health Technology Assessments - Challenges and Possible Steps for Practical Application.

The validity of mixed treatment comparisons (MTCs), also called network meta-analysis, relies on whether it is reasonable to accept the underlying assumptions on similarity, homogeneity, and consistency. The aim of this paper is to propose a practicable approach to addressing the underlying assumptions of MTCs. Using data from clinical studies of antidepressants included in a health technology assessment (HTA), we present a stepwise approach to dealing with challenges related to checking the above assumptions and to judging the robustness of the results of an MTC. At each step, studies that were dissimilar or contributed to substantial heterogeneity or inconsistency were excluded from the primary analysis. In a comparison of the MTC estimates from the consistent network with the MTC estimates from the homogeneous network including inconsistencies, few were affected by notable changes; that is, a change in effect size (factor 2), direction of effect or statistical significance. Considering the small proportion of studies excluded from the network due to inconsistency, as well as the number of notable changes, the MTC results were deemed sufficiently robust. In the absence of standard methods, our approach to checking assumptions in MTCs may inform other researchers in need of practical options, particularly in HTA.

Methodological approach to determine minor, considerable, and major treatment effects in the early benefit assessment of new drugs.

At the beginning of 2011, the early benefit assessment of new drugs was introduced in Germany with the Act on the Reform of the Market for Medicinal Products (AMNOG). The Federal Joint Committee (G-BA) generally commissions the Institute for Quality and Efficiency in Health Care (IQWiG) with this type of assessment, which examines whether a new drug shows an added benefit (a positive patient-relevant treatment effect) over the current standard therapy. IQWiG is required to assess the extent of added benefit on the basis of a dossier submitted by the pharmaceutical company responsible. In this context, IQWiG was faced with the task of developing a transparent and plausible approach for operationalizing how to determine the extent of added benefit. In the case of an added benefit, the law specifies three main extent categories (minor, considerable, major). To restrict value judgements to a minimum in the first stage of the assessment process, an explicit and abstract operationalization was needed. The present paper is limited to the situation of binary data (analysis of 2 × 2 tables), using the relative risk as an effect measure. For the treatment effect to be classified as a minor, considerable, or major added benefit, the methodological approach stipulates that the (two-sided) 95% confidence interval of the effect must exceed a specified distance to the zero effect. In summary, we assume that our approach provides a robust, transparent, and thus predictable foundation to determine minor, considerable, and major treatment effects on binary outcomes in the early benefit assessment of new drugs in Germany. After a decision on the added benefit of a new drug by G-BA, the classification of added benefit is used to inform pricing negotiations between the umbrella organization of statutory health insurance and the pharmaceutical companies.

Information on new drugs at market entry: retrospective analysis of health technology assessment reports versus regulatory reports, journal publications, and registry reports.

BACKGROUND: When a new drug becomes available, patients and doctors require information on its benefits and harms. In 2011, Germany introduced the early benefit assessment of new drugs through the act on the reform of the market for medicinal products (AMNOG). At market entry, the pharmaceutical company responsible must submit a standardised dossier containing all available evidence of the drug's added benefit over an appropriate comparator treatment. The added benefit is mainly determined using patient relevant outcomes. The "dossier assessment" is generally performed by the Institute for Quality and Efficiency in Health Care (IQWiG) and then published online. It contains all relevant study information, including data from unpublished clinical study reports contained in the dossiers. The dossier assessment refers to the patient population for which the new drug is approved according to the summary of product characteristics. This patient population may comprise either the total populations investigated in the studies submitted to regulatory authorities in the drug approval process, or the specific subpopulations defined in the summary of product characteristics ("approved subpopulations"). OBJECTIVE: To determine the information gain from AMNOG documents compared with non-AMNOG documents for methods and results of studies available at market entry of new drugs. AMNOG documents comprise dossier assessments done by IQWiG and publicly available modules of company dossiers; non-AMNOG documents comprise conventional, publicly available sources-that is, European public assessment reports, journal publications, and registry reports. The analysis focused on the approved patient populations. DESIGN: Retrospective analysis. DATA SOURCES: All dossier assessments conducted by IQWiG between 1 January 2011 and 28 February 2013 in which the dossiers contained suitable studies allowing for a full early benefit assessment. We also considered all European public assessment reports, journal publications, and registry reports referring to these studies and included in the dossiers. DATA ANALYSIS: We assessed reporting quality for each study and each available document for eight methods and 11 results items (three baseline characteristics and eight patient relevant outcomes), and dichotomised them as "completely reported" or "incompletely reported (including items not reported at all)." For each document type we calculated the proportion of items with complete reporting for methods and results, for each item and overall, and compared the findings.Results 15 out of 27 dossiers were eligible for inclusion and contained 22 studies. The 15 dossier assessments contained 28 individual assessments of 15 total study populations and 13 approved subpopulations. European public assessment reports were available for all drugs. Journal publications were available for 14 out of 15 drugs and 21 out of 22 studies. A registry report in ClinicalTrials.gov was available for all drugs and studies; however, only 11 contained results. In the analysis of total study populations, the AMNOG documents reached the highest grade of completeness, with about 90% of methods and results items completely reported. In non-AMNOG documents, the rate was 75% for methods and 52% for results items; journal publications achieved the best rates, followed by European public assessment reports and registry reports. The analysis of approved subpopulations showed poorer complete reporting of results items, particularly in non-AMNOG documents (non-AMNOG versus AMNOG: 11% v 71% for overall results items and 5% v 70% for patient relevant outcomes). The main limitation of our analysis is the small sample size. CONCLUSION: Conventional, publicly available sources provide insufficient information on new drugs, especially on patient relevant outcomes in approved subpopulations. This type of information is largely available in AMNOG documents, albeit only partly in English. The AMNOG approach could be used internationally to develop a comprehensive publication model for clinical studies and thus represents a key open access measure.

European collaboration on relative effectiveness assessments: What is needed to be successful?

OBJECTIVE: The objective of this study is to identify the possible barriers and critical success factors for the implementation of European collaboration in the field of relative effectiveness assessment (REA) of drugs. METHODS: Data were gathered through semi-structured interviews with representatives from eight European health technology assessment (HTA) organisations involved in assessment of drugs for coverage decision-making (AAZ, AIFA, AHTAPol, HAS, HVB, IQWIG, NICE and ZiN). RESULTS: Potential barriers identified mainly relate to methodology, resources and challenges with implementation in the respective national processes (e.g. legal restrictions). The most critical success factors for production of cross-border assessments were the continuous cooperation of competent partners, and the quality and timely availability of the assessment. CONCLUSION: Further adaptation of the process and methods is required for optimal collaboration. In the near future it can be expected that cross-border assessments will meet in particular the needs of smaller/middle-sized European countries and also European countries with less developed HTA systems as the potential efficiency/quality gains are the highest for these countries. Therefore, national implementation of cross-border assessments is especially likely in these countries in the coming years. Once more experience is gained with cross-border assessments, and successes become more evident, efficiency/quality gains may also be likely for some larger countries with well established processes.

[Patient-relevant outcomes and surrogates in the early benefit assessment of drugs: first experiences]. / Patientenrelevante Endpunkte und Surrogate in der frühen Nutzenbewertung von Arzneimitteln: erste Erfahrungen.

The Act on the Reform of the Market for Medicinal Products (AMNOG) became effective in Germany on January 1, 2011. Since then, the assessment of the added benefit of new drugs versus a therapeutic standard on the basis of dossiers submitted by pharmaceutical companies has been required by law. The Federal Joint Committee (G-BA) generally commissions the Institute for Quality and Efficiency in Health Care (IQWiG) with this task. The added benefit is primarily to be demonstrated on the basis of patient-relevant outcomes. The aim of this paper is to describe the feasibility of the early benefit assessment on the basis of patient-relevant outcomes by systematically characterising the outcomes available in company dossiers and comparing the companies' and IQWiG's evaluations regarding patient relevance and surrogate validity. Dossier assessments published between October 2011 and June 2012 were used for this purpose. The outcomes available and the respective evaluations were extracted and compared. 12 out of 22 submitted dossiers contained sufficient data to assess outcomes; all 12 assessable dossiers provided data on patient-relevant outcomes. Data on mortality and adverse events were available in all dossiers, except that one dossier did not contain adverse event data on the relevant subpopulation. In contrast, data on morbidity and health-related quality of life were available in 8 and 7 dossiers, respectively. Of a total of 214 outcomes extracted by IQWiG, 124 patient-relevant and 3 surrogate outcomes were included in IQWiG's assessment (companies: a total of 183 outcomes included, of which 172 were patient-relevant and 11 were surrogates). The first experiences with AMNOG have shown that in principle an early benefit assessment of drugs based on patient-relevant outcomes is feasible. The companies' and IQWiG's evaluations regarding patient relevance and surrogate validity of outcomes partly deviated from each other. By increasingly considering patient-relevant outcomes in approval studies, pharmaceutical companies can create the necessary data basis for the early benefit assessment.