Safety analysis of new medications in clinical trials: a simulation study to assess the differences between cause-specific and subdistribution frameworks in the presence of competing events.

Safety is an essential part of the evaluation of new medications and competing risks that occur in most clinical trials are a well identified challenge in the analysis of adverse events. Two statistical frameworks exist to consider competing risks: the cause-specific and the subdistribution framework. To date, the application of the cause-specific framework is the standard practice in safety analyses. Here we analyze how the safety analysis results of new medications would be affected if instead of the cause-specific the subdistribution framework was chosen. We conducted a simulation study with 600 participants, equally allocated to verum and control groups and a 30 months follow-up period. Simulated trials were analyzed for safety in a competing risk (death) setting using both the cause-specific and subdistribution frameworks. Results show that comparing safety profiles in a subdistribution setting is always more pessimistic than in a cause-specific setting. For the group with the longest survival and a safety advantage in a cause-specific setting, the advantage either disappeared or a disadvantage was found in the subdistribution analysis setting. These observations are not contradictory but show different perspectives. To evaluate the safety of a new medication over its comparator, one needs to understand the origin of both the risks and the benefits associated with each therapy. These requirements are best met with a cause-specific framework. The subdistribution framework seems better suited for clinical prediction, and therefore more relevant for providers or payers, for example.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): Rationale and statistical concept of a meta-analytic study.

The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the incidence proportion, that is the number of patients with a specific adverse event out of all patients in the treatment groups, do not account for both varying follow-up times and competing risks. Alternative approaches such as the Aalen-Johansen estimator of the cumulative incidence function have been suggested. Theoretical arguments and numerical evaluations support the application of these more advanced methodology, but as yet there is to our knowledge only insufficient empirical evidence whether these methods would lead to different conclusions in safety evaluations. The Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) project strives to close this gap in evidence by conducting a meta-analytical study to assess the impact of the methodology on the conclusion of the safety assessment empirically. Here we present the rationale and statistical concept of the empirical study conducted as part of the SAVVY project. The statistical methods are presented in unified notation, and examples of their implementation in R and SAS are provided.

Does big data require a methodological change in medical research?

BACKGROUND: Use of big data is becoming increasingly popular in medical research. Since big data-based projects differ notably from classical research studies, both in terms of scope and quality, a debate is apt as to whether big data require new approaches to scientific reasoning different from those established in statistics and philosophy of science. MAIN TEXT: The progressing digitalization of our societies generates vast amounts of data that also become available for medical research. Here, the big promise of big data is to facilitate major improvements in the treatment, diagnosis and prevention of diseases. An ongoing examination of the idiosyncrasies of big data is therefore essential to ensure that the field stays congruent with the principles of evidence-based medicine. We discuss the inherent challenges and opportunities of big data in medicine from a methodological point of view, particularly highlighting the relative importance of causality and correlation in commercial and medical research settings. We make a strong case for upholding the distinction between exploratory data analysis facilitating hypothesis generation and confirmatory approaches involving hypothesis validation. An independent verification of research results will be ever more important in the context of big data, where data quality is often hampered by a lack of standardization and structuring. CONCLUSIONS: We argue that it would be both unnecessary and dangerous to discard long-established principles of data generation, analysis and interpretation in the age of big data. While many medical research areas may reasonably benefit from big data analyses, they should nevertheless be complemented by carefully designed (prospective) studies.

On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies.

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit-risk ratio. The statistical analysis of AEs is complicated by the fact that the follow-up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow-up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta-analyses of AE data and sketch possible solutions.

Safety data from randomized controlled trials: applying models for recurrent events.

Simple descriptive listings and inference statistics based on 2×2 tables are still the most common way of summarizing and reporting adverse events data from randomized controlled trials, although these methods do not account for differences in observation times between treatment groups. Using standard methods from survival analysis such as the Cox model or Kaplan-Meier estimates would overcome this problem but limit the analysis to the first safety-related event of each subject. As an alternative, we discuss two models for recurrent events data-the Andersen-Gill and Prentice-Williams-Peterson model-regarding their applicability to safety data from randomized controlled trials. We argue that these models can be used to estimate two different quantities: a direct treatment effect on the risk of an event (Prentice-Williams-Peterson) and a total treatment effect as sum of the direct effect and the treatment's indirect effect via the event history (Anderson-Gill). Using simulated data, we illustrate the difference between these treatment effects and analyze the performance of both models in different scenarios. Because both models are limited to the analysis of cause-specific hazards if competing risks are present, we suggest to incorporate estimates of the mean frequency of events in the analysis to additionally allow the comparison of treatment effects on absolute event probabilities. We demonstrate the application of both models and the mean frequency function to safety endpoints with an illustrative analysis of data from a randomized phase-III study. Copyright © 2016 John Wiley & Sons, Ltd.

Implementation of AMNOG: An industry perspective.

In 2010, the Federal Parliament (Bundestag) of Germany passed a new law (Arzneimittelmarktneuordnungsgesetz, AMNOG) on the regulation of medicinal products that applies to all pharmaceutical products with active ingredients that are launched beginning January 1, 2011. The law describes the process to determine the price at which an approved new product will be reimbursed by the statutory health insurance system. The process consists of two phases. The first phase assesses the additional benefit of the new product versus an appropriate comparator (zweckmäßige Vergleichstherapie, zVT). The second phase involves price negotiation. Focusing on the first phase, this paper investigates requirements of benefit assessment of a new product under this law with special attention on the methods applied by the German authorities on issues such as the choice of the comparator, patient relevant endpoints, subgroup analyses, extent of benefit, determination of net benefit, primary and secondary endpoints, and uncertainty of the additional benefit. We propose alternative approaches to address the requirements in some cases and invite other researchers to help develop solutions in other cases.

Clinical Evidence Informing Treatment Guidelines on Repurposed Drugs for Hospitalized Patients During the Early COVID-19 Pandemic: Corticosteroids, Anticoagulants, (Hydroxy)chloroquine.

INTRODUCTION: In early 2020, the coronavirus disease 2019 (COVID-19) pandemic spread worldwide, overwhelming hospitals with severely ill patients and posing the urgent need for clinical evidence to guide patient care. First treatment options available were repurposed drugs to fight inflammation, coagulopathy, and viral replication. A vast number of clinical studies were launched globally to test their efficacy and safety. Our analysis describes the development of global evidence on repurposed drugs, in particular corticosteroids, anticoagulants, and (hydroxy)chloroquine in hospitalized COVID-19 patients based on different study types. We track the incorporation of clinical data in international and national treatment guidelines and identify factors that characterize studies and analyses with the greatest impact on treatment recommendations. METHODS: A literature search in MEDLINE was conducted to assess the clinical evidence on treatment with corticosteroids, anticoagulants, and (hydroxy)chloroquine in hospitalized COVID-19 patients during the first year of the pandemic. Adoption of the evidence from this clinical data in treatment guidelines of the World Health Organization (WHO), Germany, and United States (US) was evaluated over time. RESULTS: We identified 106 studies on corticosteroids, 141 studies on anticoagulants, and 115 studies on (hydroxy)chloroquine. Most studies were retrospective cohort studies; some were randomized clinical trials (RCTs), and a few were platform trials. These studies were compared to studies directly and indirectly referred to in WHO (7 versions), German (5 versions), and US (21 versions) guidelines. We found that initially large, well-adjusted, mainly retrospective cohort studies and ultimately large platform trials or coordinated meta-analyses of RCTs provided best available clinical evidence supporting treatment recommendations. DISCUSSION: Particularly early in the pandemic, evidence for the efficacy and safety of repurposed drugs was of low quality, since time and scientific rigor seemed to be competing factors. Pandemic preparedness, coordinated efforts, and combined analyses were crucial to generating timely and robust clinical evidence that informed national and international treatment guidelines on corticosteroids, anticoagulants, and (hydroxy)chloroquine. Multi-arm platform trials with master protocols and coordinated meta-analyses proved particularly successful, with researchers joining forces to answer the most pressing questions as quickly as possible.

Population-based incidence and mortality of community-acquired pneumonia in Germany.

BACKGROUND: Little information on the current burden of community-acquired pneumonia (CAP) in adults in Germany is available. METHODS: We conducted a retrospective cohort study using a representative healthcare claims database of approx. 4 million adults to estimate the incidence rates (IR) and associated mortality of CAP in 2015. IR and mortality were stratified by treatment setting, age group, and risk group status. A pneumonia coded in the primary diagnosis position or in the second diagnosis position with another pneumonia-related condition coded in the primary position was used as the base cases definition for the study. Sensitivity analyses using broader and more restrictive case definitions were also performed. RESULTS: The overall IR of CAP in adults ≥18 years was 1,054 cases per 100,000 person-years of observation. In adults aged 16 to 59 years, IR for overall CAP, hospitalized CAP and outpatient CAP was 551, 96 and 466 (with a hospitalization rate of 17%). In adults aged ≥60 years, the respective IR were 2,032, 1,061 and 1,053 (with a hospitalization rate of 52%). If any pneumonia coded in the primary or secondary diagnosis position was considered for hospitalized patients, the IR increased 1.5-fold to 1,560 in the elderly ≥60 years. The incidence of CAP hospitalizations was substantially higher in adults ≥18 years with at-risk conditions and high-risk conditions (IR of 608 and 1,552, respectively), compared to adults without underlying risk conditions (IR 108). High mortality of hospitalized CAP in adults ≥18 was observed in-hospital (18.5%), at 30 days (22.9%) and at one-year (44.5%) after CAP onset. Mortality was more than double in older adults in comparison to younger patients. CONCLUSION: CAP burden in older adults and individuals with underlying risk conditions was high. Maximizing uptake of existing vaccines for respiratory diseases may help to mitigate the disease burden, especially in times of strained healthcare resources.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY)-estimation of adverse event risks.

BACKGROUND: The SAVVY project aims to improve the analyses of adverse events (AEs), whether prespecified or emerging, in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses, often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator are used, which ignore either censoring or CEs. In an empirical study including randomized clinical trials from several sponsor organizations, these potential sources of bias are investigated. The main purpose is to compare the estimators that are typically used to quantify AE risk within trial arms to the non-parametric Aalen-Johansen estimator as the gold-standard for estimating cumulative AE probabilities. A follow-up paper will consider consequences when comparing safety between treatment groups. METHODS: Estimators are compared with descriptive statistics, graphical displays, and a more formal assessment using a random effects meta-analysis. The influence of different factors on the size of deviations from the gold-standard is investigated in a meta-regression. Comparisons are conducted at the maximum follow-up time and at earlier evaluation times. CEs definition does not only include death before AE but also end of follow-up for AEs due to events related to the disease course or safety of the treatment. RESULTS: Ten sponsor organizations provided 17 clinical trials including 186 types of investigated AEs. The one minus Kaplan-Meier estimator was on average about 1.2-fold larger than the Aalen-Johansen estimator and the probability transform of the incidence density ignoring CEs was even 2-fold larger. The average bias using the incidence proportion was less than 5%. Assuming constant hazards using incidence densities was hardly an issue provided that CEs were accounted for. The meta-regression showed that the bias depended mainly on the amount of censoring and on the amount of CEs. CONCLUSIONS: The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. We recommend using the Aalen-Johansen estimator with an appropriate definition of CEs whenever the risk for AEs is to be quantified and to change the guidelines accordingly.

Comparing the EQ-5D-5L utility index based on value sets of different countries: impact on the interpretation of clinical study results.

OBJECTIVE: To compare the country-specific value sets of the EQ-5D-5L utility index and to evaluate the impact on the interpretation of clinical study results. Six country value sets from Canada, England, Japan, Korea, Netherlands and Uruguay were obtained from literature. In addition, ten crosswalk value sets were downloaded from the EuroQol.org website. RESULTS: For each of the 3125 possible health states the difference between the country with the highest index and the country with the lowest index was calculated. The median difference was 0.417 across the health states. When analyzing multinational clinical studies, country-specific value sets should be used to evaluate treatment effects. Additional country-specific analyses are needed.

[Estimating the incidence of venous thromboembolism (VTE) using various types of routine data of the German healthcare system]. / Schätzung der Inzidenz venöser Thromboembolien (VTE) anhand verschiedener Routinedaten des Gesundheitswesens in Deutschland.

BACKGROUND: Venous thromboembolism (VTE) mainly manifests as deep vein thrombosis (TVT) or pulmonary embolism (LE), and is the third most common cardiovascular disease worldwide. However, robust evidence on the incidence of VTE in Germany is lacking. OBJECTIVE: Estimation and comparison of the incidence of VTE based on different routine data sources of the German healthcare system. METHODS: Estimates and comparisons of the incidence of VTE, TVT and LE were made using two databases that both covered the inpatient and the outpatient setting; the DaTraV database comprising information of all persons subject to compulsory health insurance, and the Health Risk Institute (HRI) database derived from approximately 70 statutory health insurance funds. In addition, IMS Disease Analyzer, a medical record database comprising information from the outpatient setting, was used as a data source. RESULTS: Patterns of age- and sex-specific VTE incidence estimates were comparable between all databases used. However, estimates based on the medical record database were comparatively high. Analyses of DaTraV data led to a VTE incidence of 0.14%. Use of HRI data yielded comparable results (0.17-0.20%). VTE incidence based on data of the IMS Disease Analyzer was comparatively high (0.32%). DISCUSSION: Results on the VTE incidence based on DaTraV or HRI date are comparable to international evidence, whereas the use of the IMS Disease Analyzer data presumably led to an overestimation due to double-counting of VTE cases. Different types of routine healthcare data sources can therefore lead to very heterogeneous results. Thus, the selection of adequate data sources strongly depends on the study question and the quality of the dataset.

Evaluation of Adjusted and Unadjusted Indirect Comparison Methods in Benefit Assessment. A Simulation Study for Time-to-event Endpoints.

BACKGROUND: With the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany, pharmaceutical manufacturers are obliged to submit a dossier demonstrating added benefit of a new drug compared to an appropriate comparator. Underlying evidence was planned for registration purposes and therefore often does not meet the appropriate comparator as defined by the Federal Joint Committee (G-BA). For this reason AMNOG allows indirect comparisons to assess the extent of added benefit. OBJECTIVES: The aim of this study is to evaluate the characteristics and applicability of adjusted indirect comparison described by Bucher and Matching-Adjusted Indirect Comparison (MAIC) in various situations within the early benefit assessment according to §35a Social Code Book 5. In particular, we consider time-to-event endpoints. METHODS: We conduct a simulation study where we consider three different scenarios: I) similar study populations, II) dissimilar study populations without interactions and III) dissimilar study populations with interactions between treatment effect and effect modifiers. We simulate data from a Cox model with Weibull distributed survival times. Desired are unbiased effect estimates. We compare the power and the proportion of type 1 errors of the methods. RESULTS: I) Bucher and MAIC perform equivalently well and yield unbiased effect estimates as well as proportions of type 1 errors below the significance level of 5 %. II) Both Bucher and MAIC yield unbiased effect estimates, but Bucher shows a higher power for detection of true added benefit than MAIC. III) Only MAIC, but not Bucher yields unbiased effect estimates. When using robust variance estimation MAIC yields a proportion of type 1 error close to 5 %. In general, power of all methods for indirect comparisons is low. An increasing loss of power for the indirect comparisons can be observed as the true treatment effects decrease. CONCLUSION: Due to the great loss of power and the potential bias for indirect comparisons, head-to-head trials using the appropriate comparator as defined by the Federal Joint Committee should be conducted whenever possible. However, indirect comparisons are needed if no such direct evidence is available. To conduct indirect comparisons in case of a present common comparator and similar study populations in the trials to be compared, both Bucher and MAIC can be recommended. In case of using adjusted effect measures (such as Hazard Ratio), the violation of the similarity assumption has no relevant effect on the Bucher approach as long as interactions between treatment effect and effect modifiers are absent. Therefore Bucher can still be considered appropriate in this specific situation. In the authors' opinion, MAIC can be considered as an option (at least as sensitivity analysis to Bucher) if such interactions are present or cannot be ruled out. Nevertheless, in practice MAIC is potentially biased and should always be considered with utmost care.

Chronic pain patients' treatment preferences: a discrete-choice experiment.

OBJECTIVE: The objective of this study was to identify, document, and weight attributes of a pain medication that are relevant from the perspective of patients with chronic pain. Within the sub-population of patients suffering from "chronic neuropathic pain", three groups were analyzed in depth: patients with neuropathic back pain, patients with painful diabetic polyneuropathy, and patients suffering from pain due to post-herpetic neuralgia. The central question was: "On which features do patients base their assessment of pain medications and which features are most useful in the process of evaluating and selecting possible therapies?" METHODS: A detailed literature review, focus groups with patients, and face-to-face interviews with widely recognized experts for pain treatment were conducted to identify relevant treatment attributes of a pain medication. A pre-test was conducted to verify the structure of relevant and dominant attributes using factor analyses by evaluating the most frequently mentioned representatives of each factor. The Discrete-Choice Experiment (DCE) used a survey based on self-reported patient data including socio-demographics and specific parameters concerning pain treatment. Furthermore, the neuropathic pain component was determined in all patients based on their scoring in the painDETECT(®) questionnaire. For statistical data analysis of the DCE, a random effect logit model was used and coefficients were presented. RESULTS: A total of 1,324 German patients participated in the survey, of whom 44 % suffered from neuropathic back pain (including mixed pain syndrome), 10 % complained about diabetic polyneuropathy, and 4 % reported pain due to post-herpetic neuralgia. A total of 36 single quality aspects of pain treatment, detected in the qualitative survey, were grouped in 7 dimensions by factor analysis. These 7 dimensions were used as attributes for the DCE. The DCE model resulted in the following ranking of relevant attributes for treatment decision: "no character change", "less nausea and vomiting", "pain reduction" (coefficient: >0.9 for all attributes, "high impact"), "rapid effect", "low risk of addiction" (coefficient ~0.5, "middle impact"), "applicability with comorbidity" (coefficient ~0.3), and "improvement of quality of sleep" (coefficient ~0.25). All attributes were highly significant (p < 0.001). CONCLUSIONS: The results were intended to enable early selection of an individualized pain medication. The results of the study showed that DCE is an appropriate means for the identification of patient preferences when being treated with specific pain medications. Due to the fact that pain perception is subjective in nature, the identification of patients´ preferences will enable therapists to better develop and implement patient-oriented treatment of chronic pain. It is therefore essential to improve the therapists´ understanding of patient preferences in order to make decisions concerning pain treatment. DCE and direct assessment should become valid instruments to elicit treatment preferences in chronic pain.

[Methodology for analysing quality-of-life data in the benefit assessment of pharmaceuticals]. / Methodik für die Auswertung von Lebensqualitätsdaten im Rahmen der Nutzenbewertung von Arzneimitteln.

Early benefit assessment aims to prove a benefit of a new pharmaceutical over the appropriate comparator based on patient-relevant endpoints. In addition to mortality and morbidity, quality of life is a patient-relevant endpoint. Thus, phase III clinical trials are the basis of evidence. But HTA and health authorities attach different importance to quality of life. Using the example of oncology, the challenges with study design and analysis will be discussed. A particular challenge to the analysis of quality-of-life data is varying observation times in treatment arms with different effectiveness. Based on the example of Crizotinib possible solutions will be presented.

Development and validation of a new measure to evaluate psychological resistance to insulin treatment.

OBJECTIVE: To develop a psychometric questionnaire to measure psychological barriers to insulin treatment in patients with type 2 diabetes. RESEARCH DESIGN AND METHODS: Scale development was based on principal component analyses in two cross-sectional studies of insulin-naïve patients with type 2 diabetes. The structure of the questionnaire was developed in the first sample of 448 patients and subsequently cross-validated in an independent sample of 449 patients. RESULTS: Analyses in the first sample yielded five components that accounted for 74.5% of the variance based on 14 items and led to the following subscales: fear of injection and self-testing, expectations regarding positive insulin-related outcomes, expected hardship from insulin treatment, stigmatization by insulin injections, and fear of hypoglycemia. In addition, an overall sum score of all values was calculated. The structure of the questionnaire was cross-validated in the second sample, with almost identical component loadings and an explained variance of 69.4%. An additional confirmatory factor analysis also indicated an acceptable to good model fit with root mean square error of approximation equal to 0.04 and comparative fit index equal to 0.97. Coefficients of reliability (Cronbach's alpha 0.62-0.85 and 0.78 for overall sum score) were acceptable, considering the very small number of items for each scale. CONCLUSIONS: The Barriers to Insulin Treatment Questionnaire appears to be a reliable and valid measure of psychological insulin resistance in patients with type 2 diabetes. This short instrument is easy to administer and may be used by both clinicians and researchers to assess the psychological barriers to insulin treatment.