Challenges on Multiple Endpoints in Clinical Trials: An Industry Survey in Japan.

BACKGROUND: Two issues on clinical trials with multiple endpoints were surveyed: (1) the terminology of multiple endpoints, relationship between rare events and endpoints, and differences in multiplicity adjustment between regions; and (2) the current practice on multiplicity adjustment and sample size calculation. This article provides a summary of the results of a survey on the first issue. METHODS: The survey was conducted among 63 members of the Japan Pharmaceutical Manufacturers Association from October to November 2017. RESULTS: Thirty-five companies based in Japan and 12 companies based in other countries, 47 companies in total, responded to the survey. The terms co-primary endpoints, secondary endpoint, and composite endpoint were used in a variety of ways. An endpoint for a clinically most important event that is expected to occur rarely differed between regions. Although the Pharmaceuticals and Medical Devices Agency did not demand multiplicity adjustment, it was considered in clinical trials with multiple endpoints for approval in Japan. CONCLUSIONS: The use of terminology differed from the definition in the Food and Drug Administration guidance and the European Medicines Agency guideline. There remain challenges on a clinically most important event that is expected to occur rarely and multiplicity adjustment in clinical trials with multiple endpoints.

Multiplicity Adjustment and Sample Size Calculation in Clinical Trials with Multiple Endpoints: An Industry Survey of Current Practices in Japan.

BACKGROUND: Two issues in clinical trials with multiple endpoints were surveyed: (1) the terminology of multiple endpoints, the relationship between rare events and endpoints, and the differences in multiplicity adjustment between regions, and (2) the current practice on multiplicity adjustment and sample size calculation. This article summarizes the results of the survey on the second issue. METHODS: Eligible trials for this survey fulfilled the following conditions: (1) confirmatory phase 3 trial; (2) use of multiple primary endpoints, co-primary endpoints, key secondary endpoint(s) or composite endpoint(s); (3) inclusion of Japanese participants; and (4) protocols created in 2010 or later. The survey was conducted at member companies of the Japan Pharmaceutical Manufacturers Association from October 2017 to November 2017. RESULTS: Useable responses were obtained from 78 trials in 13 companies based in Japan and 9 companies based in other countries. The Bonferroni procedure was mostly used in clinical trials with multiple primary endpoints, while multiple testing procedures that consider a hierarchy of endpoints or a structure of hypotheses were used in clinical trials with key secondary endpoint(s). In sample size calculation, we can consider the probability of study success, such as the probability of statistical significance in at least one comparison of primary endpoints; however, other probabilities were also considered. This survey reveals that multiplicity adjustment and the correlation of endpoints were not always considered in sample size calculation. CONCLUSIONS: In clinical trials with multiple endpoints, clinical importance was considered when determining multiple testing procedures. Challenges remain with the definition of power, the consideration of multiple testing procedures and the correlation between endpoints in sample size calculation.

Challenges on Multiple Endpoints in Clinical Trials: An Industry Survey in Japan.

BACKGROUND: Two issues on clinical trials with multiple endpoints were surveyed: (1) the terminology of multiple endpoints, relationship between rare events and endpoints, and differences in multiplicity adjustment between regions; and (2) the current practice on multiplicity adjustment and sample size calculation. This article provides a summary of the results of a survey on the first issue. METHODS: The survey was conducted among 63 members of the Japan Pharmaceutical Manufacturers Association from October to November 2017. RESULTS: Thirty-five companies based in Japan and 12 companies based in other countries, 47 companies in total, responded to the survey. The terms co-primary endpoints, secondary endpoint, and composite endpoint were used in a variety of ways. An endpoint for a clinically most important event that is expected to occur rarely differed between regions. Although the Pharmaceuticals and Medical Devices Agency did not demand multiplicity adjustment, it was considered in clinical trials with multiple endpoints for approval in Japan. CONCLUSIONS: The use of terminology differed from the definition in the Food and Drug Administration guidance and the European Medicines Agency guideline. There remain challenges on a clinically most important event that is expected to occur rarely and multiplicity adjustment in clinical trials with multiple endpoints.

A Bayesian meta-analytic approach for safety signal detection in randomized clinical trials.

BACKGROUND/AIM: Meta-analyses are frequently performed on adverse event data and are primarily used for improving statistical power to detect safety signals. However, in the evaluation of drug safety for New Drug Applications, simple pooling of adverse event data from multiple clinical trials is still commonly used. We sought to propose a new Bayesian hierarchical meta-analytic approach based on consideration of a hierarchical structure of reported individual adverse event data from multiple randomized clinical trials. METHODS: To develop our meta-analysis model, we extended an existing three-stage Bayesian hierarchical model by including an additional stage of the clinical trial level in the hierarchical model; this generated a four-stage Bayesian hierarchical model. We applied the proposed Bayesian meta-analysis models to published adverse event data from three premarketing randomized clinical trials of tadalafil and to a simulation study motivated by the case example to evaluate the characteristics of three alternative models. RESULTS: Comparison of the results from the Bayesian meta-analysis model with those from Fisher's exact test after simple pooling showed that 6 out of 10 adverse events were the same within a top 10 ranking of individual adverse events with regard to association with treatment. However, more individual adverse events were detected in the Bayesian meta-analysis model than in Fisher's exact test under the body system "Musculoskeletal and connective tissue disorders." Moreover, comparison of the overall trend of estimates between the Bayesian model and the standard approach (odds ratios after simple pooling methods) revealed that the posterior median odds ratios for the Bayesian model for most adverse events shrank toward values for no association. Based on the simulation results, the Bayesian meta-analysis model could balance the false detection rate and power to a better extent than Fisher's exact test. For example, when the threshold value of the posterior probability for signal detection was set to 0.8, the false detection rate was 41% and power was 88% in the Bayesian meta-analysis model, whereas the false detection rate was 56% and power was 86% in Fisher's exact test. LIMITATIONS: Adverse events under the same body system were not necessarily positively related when we used "system organ class" and "preferred term" in the Medical Dictionary for Regulatory Activities as a hierarchical structure of adverse events. For the Bayesian meta-analysis models to be effective, the validity of the hierarchical structure of adverse events and the grouping of adverse events are critical. CONCLUSION: Our proposed meta-analysis models considered trial effects to avoid confounding by trial and borrowed strength from both within and across body systems to obtain reasonable and stable estimates of an effect measure by considering a hierarchical structure of adverse events.

Adjustment for Propensity Score in Nonrandomized Clinical Studies: Comparison of Sivelestat Versus Conventional Therapy for Acute Lung Injury in Acute Respiratory Distress Syndrome.

BACKGROUND: To confirm the effectiveness of sivelestat, a clinical trial was conducted comparing sivelestat with conventional treatment in an open, nonrandomized, multicenter study of patients with systemic inflammatory response syndrome (SIRS)-associated acute lung injury. The primary endpoint was ventilator-free days (VFD). METHODS: This study adopted a "cluster entry" method to control for patient selection bias arising from the unblinded and nonrandomized clinical trial. Thus, all patients in the same hospital during the same entry period entered the same treatment arm, and entry periods did not overlap. In the primary analysis of VFD, adjusted mean VFD values were compared between groups using the inverse probability of treatment weighted (IPTW) method, based on propensity score, for control of confounding factors. RESULTS: There were 374 patients in the sivelestat group and 168 in the conventional therapy group. The primary analysis confirmed that sivelestat was effective (between-group difference of adjusted mean was 3.5 [2-sided 95% confidence interval, 1.3-5.8]; P = .0022). CONCLUSIONS: In general, a study where all patients in the same cluster enter the same treatment arm has within-cluster correlations, which need to be considered in the study analysis. However, in analysis using the IPTW method, it is usual to use a robust variance estimator, the sandwich variance estimator, which is consistent regardless of whether the specification of within-cluster correlation structure is correct. Thus, in the analysis using the IPTW method, it was found that it was not necessary to adopt any other adjustment method for within-cluster correlation.

Current Practice on Multiplicity Adjustment and Sample Size Calculation in Multi-arm Clinical Trials: An Industry Survey in Japan.

BACKGROUND: This study provides the results of a survey on the current practice of multiplicity adjustment and sample size calculation in multi-arm clinical trials. METHODS: The survey was aimed at members of the Japan Pharmaceutical Manufacturers Association (JPMA) and was conducted in 2015. RESULTS: Of the 66 JPMA member companies, effective responses were obtained on 151 trials from 33 companies based in Japan and 11 companies based in other countries. The results from this survey indicate that multiplicity adjustment in confirmatory multi-arm trials is adequate in terms of controlling the familywise error rate. Multiplicity was adjusted in 38.3% of exploratory multi-arm trials. Various multiple comparison procedures (MCPs) were applied, with the fixed sequence procedure being the one applied most frequently. This survey also reveals that there are issues that need to be addressed within sample size calculation. CONCLUSIONS: To adequately design a multi-arm clinical trial, it is important within sample size calculation to consider whether to perform multiplicity adjustment, select MCPs, and define their power.