Assumptions of Mixed Treatment Comparisons in Health Technology Assessments - Challenges and Possible Steps for Practical Application.

The validity of mixed treatment comparisons (MTCs), also called network meta-analysis, relies on whether it is reasonable to accept the underlying assumptions on similarity, homogeneity, and consistency. The aim of this paper is to propose a practicable approach to addressing the underlying assumptions of MTCs. Using data from clinical studies of antidepressants included in a health technology assessment (HTA), we present a stepwise approach to dealing with challenges related to checking the above assumptions and to judging the robustness of the results of an MTC. At each step, studies that were dissimilar or contributed to substantial heterogeneity or inconsistency were excluded from the primary analysis. In a comparison of the MTC estimates from the consistent network with the MTC estimates from the homogeneous network including inconsistencies, few were affected by notable changes; that is, a change in effect size (factor 2), direction of effect or statistical significance. Considering the small proportion of studies excluded from the network due to inconsistency, as well as the number of notable changes, the MTC results were deemed sufficiently robust. In the absence of standard methods, our approach to checking assumptions in MTCs may inform other researchers in need of practical options, particularly in HTA.

[Patient-relevant outcomes and surrogates in the early benefit assessment of drugs: first experiences]. / Patientenrelevante Endpunkte und Surrogate in der frühen Nutzenbewertung von Arzneimitteln: erste Erfahrungen.

The Act on the Reform of the Market for Medicinal Products (AMNOG) became effective in Germany on January 1, 2011. Since then, the assessment of the added benefit of new drugs versus a therapeutic standard on the basis of dossiers submitted by pharmaceutical companies has been required by law. The Federal Joint Committee (G-BA) generally commissions the Institute for Quality and Efficiency in Health Care (IQWiG) with this task. The added benefit is primarily to be demonstrated on the basis of patient-relevant outcomes. The aim of this paper is to describe the feasibility of the early benefit assessment on the basis of patient-relevant outcomes by systematically characterising the outcomes available in company dossiers and comparing the companies' and IQWiG's evaluations regarding patient relevance and surrogate validity. Dossier assessments published between October 2011 and June 2012 were used for this purpose. The outcomes available and the respective evaluations were extracted and compared. 12 out of 22 submitted dossiers contained sufficient data to assess outcomes; all 12 assessable dossiers provided data on patient-relevant outcomes. Data on mortality and adverse events were available in all dossiers, except that one dossier did not contain adverse event data on the relevant subpopulation. In contrast, data on morbidity and health-related quality of life were available in 8 and 7 dossiers, respectively. Of a total of 214 outcomes extracted by IQWiG, 124 patient-relevant and 3 surrogate outcomes were included in IQWiG's assessment (companies: a total of 183 outcomes included, of which 172 were patient-relevant and 11 were surrogates). The first experiences with AMNOG have shown that in principle an early benefit assessment of drugs based on patient-relevant outcomes is feasible. The companies' and IQWiG's evaluations regarding patient relevance and surrogate validity of outcomes partly deviated from each other. By increasingly considering patient-relevant outcomes in approval studies, pharmaceutical companies can create the necessary data basis for the early benefit assessment.

Evidence synthesis for decision making 7: a reviewer's checklist.

This checklist is for the review of evidence syntheses for treatment efficacy used in decision making based on either efficacy or cost-effectiveness. It is intended to be used for pairwise meta-analysis, indirect comparisons, and network meta-analysis, without distinction. It does not generate a quality rating and is not prescriptive. Instead, it focuses on a series of questions aimed at revealing the assumptions that the authors of the synthesis are expecting readers to accept, the adequacy of the arguments authors advance in support of their position, and the need for further analyses or sensitivity analyses. The checklist is intended primarily for those who review evidence syntheses, including indirect comparisons and network meta-analyses, in the context of decision making but will also be of value to those submitting syntheses for review, whether to decision-making bodies or journals. The checklist has 4 main headings: A) definition of the decision problem, B) methods of analysis and presentation of results, C) issues specific to network synthesis, and D) embedding the synthesis in a probabilistic cost-effectiveness model. The headings and implicit advice follow directly from the other tutorials in this series. A simple table is provided that could serve as a pro forma checklist.

Introducing GRADE across the NICE clinical guideline program.

OBJECTIVES: Grading of Recommendations Assessment, Development and Evaluation (GRADE) is a system for rating the confidence in estimates of effect and grading guideline recommendations. It promotes evaluation of the quality of the evidence for each outcome and an assessment of balance between desirable and undesirable outcomes leading to a judgment about the strength of the recommendation. In 2007, the National Institute for Health and Clinical Excellence began introducing GRADE across its clinical guideline program to enable separation of judgments about the evidence quality from judgments about the strength of the recommendation. STUDY DESIGN AND SETTING: We describe the process of implementing GRADE across guidelines. RESULTS: Use of GRADE has been positively received by both technical staff and guideline development group members. CONCLUSION: A shift in thinking about confidence in the evidence was required leading to a more structured and transparent approach to decision making. Practical problems were also encountered; these have largely been resolved, but some areas require further work, including the application of imprecision and presenting results from analyses considering more than two alternative interventions. The use of GRADE for nonrandomized and diagnostic accuracy studies needs to be refined.