A framework for longitudinal latent factor modelling of treatment response in clinical trials with applications to Psoriatic Arthritis and Rheumatoid Arthritis.

OBJECTIVE: Clinical trials involve the collection of a wealth of data, comprising multiple diverse measurements performed at baseline and follow-up visits over the course of a trial. The most common primary analysis is restricted to a single, potentially composite endpoint at one time point. While such an analytical focus promotes simple and replicable conclusions, it does not necessarily fully capture the multi-faceted effects of a drug in a complex disease setting. Therefore, to complement existing approaches, we set out here to design a longitudinal multivariate analytical framework that accepts as input an entire clinical trial database, comprising all measurements, patients, and time points across multiple trials. METHODS: Our framework composes probabilistic principal component analysis with a longitudinal linear mixed effects model, thereby enabling clinical interpretation of multivariate results, while handling data missing at random, and incorporating covariates and covariance structure in a computationally efficient and principled way. RESULTS: We illustrate our approach by applying it to four phase III clinical trials of secukinumab in Psoriatic Arthritis (PsA) and Rheumatoid Arthritis (RA). We identify three clinically plausible latent factors that collectively explain 74.5% of empirical variation in the longitudinal patient database. We estimate longitudinal trajectories of these factors, thereby enabling joint characterisation of disease progression and drug effect. We perform benchmarking experiments demonstrating our method's competitive performance at estimating average treatment effects compared to existing statistical and machine learning methods, and showing that our modular approach leads to relatively computationally efficient model fitting. CONCLUSION: Our multivariate longitudinal framework has the potential to illuminate the properties of existing composite endpoint methods, and to enable the development of novel clinical endpoints that provide enhanced and complementary perspectives on treatment response.

Predicting subgroup treatment effects for a new study: Motivations, results and learnings from running a data challenge in a pharmaceutical corporation.

We present the motivation, experience, and learnings from a data challenge conducted at a large pharmaceutical corporation on the topic of subgroup identification. The data challenge aimed at exploring approaches to subgroup identification for future clinical trials. To mimic a realistic setting, participants had access to 4 Phase III clinical trials to derive a subgroup and predict its treatment effect on a future study not accessible to challenge participants. A total of 30 teams registered for the challenge with around 100 participants, primarily from Biostatistics organization. We outline the motivation for running the challenge, the challenge rules, and logistics. Finally, we present the results of the challenge, the participant feedback as well as the learnings. We also present our view on the implications of the results on exploratory analyses related to treatment effect heterogeneity.

All that Glitters Is not Gold: Type-I Error Controlled Variable Selection from Clinical Trial Data.

Clinical trials are primarily conducted to estimate causal effects, but the data collected can also be invaluable for additional research, such as identifying prognostic measures of disease or biomarkers that predict treatment efficacy. However, these exploratory settings are prone to false discoveries (type-I errors) due to the multiple comparisons they entail. Unfortunately, many methods fail to address this issue, in part because the algorithms used are generally designed to optimize predictions and often only provide the measures used for variable selection, such as machine learning model importance scores, as a byproduct. To address the resulting unclear uncertainty in the selection sets, the knockoff framework offers a model-agnostic, robust approach to variable selection with guaranteed type-I error control. Here, we review the knockoff framework in the setting of clinical data, highlighting main considerations using simulation studies. We also extend the framework by introducing a novel knockoff generation method that addresses two main limitations of previously suggested methods relevant for clinical development settings. With this new method, we empirically obtain tighter bounds on type-I error control and gain an order of magnitude in computational efficiency in mixed data settings. We demonstrate comparable selections to those of the competing method for identifying prognostic biomarkers for C-reactive protein levels in patients with psoriatic arthritis in four clinical trials. Our work increases access to the knockoff framework for variable selection from clinical trial data. Hereby, this paper helps to address the current replicability crisis which can result in unnecessary research efforts, increased patient burden, and avoidable costs.

Methodology for Good Machine Learning with Multi-Omics Data.

In 2020, Novartis Pharmaceuticals Corporation and the U.S. Food and Drug Administration (FDA) started a 4-year scientific collaboration to approach complex new data modalities and advanced analytics. The scientific question was to find novel radio-genomics-based prognostic and predictive factors for HR+/HER- metastatic breast cancer under a Research Collaboration Agreement. This collaboration has been providing valuable insights to help successfully implement future scientific projects, particularly using artificial intelligence and machine learning. This tutorial aims to provide tangible guidelines for a multi-omics project that includes multidisciplinary expert teams, spanning across different institutions. We cover key ideas, such as "maintaining effective communication" and "following good data science practices," followed by the four steps of exploratory projects, namely (1) plan, (2) design, (3) develop, and (4) disseminate. We break each step into smaller concepts with strategies for implementation and provide illustrations from our collaboration to further give the readers actionable guidance.

A deep learning approach to private data sharing of medical images using conditional generative adversarial networks (GANs).

Clinical data sharing can facilitate data-driven scientific research, allowing a broader range of questions to be addressed and thereby leading to greater understanding and innovation. However, sharing biomedical data can put sensitive personal information at risk. This is usually addressed by data anonymization, which is a slow and expensive process. An alternative to anonymization is construction of a synthetic dataset that behaves similar to the real clinical data but preserves patient privacy. As part of a collaboration between Novartis and the Oxford Big Data Institute, a synthetic dataset was generated based on images from COSENTYX® (secukinumab) ankylosing spondylitis (AS) clinical studies. An auxiliary classifier Generative Adversarial Network (ac-GAN) was trained to generate synthetic magnetic resonance images (MRIs) of vertebral units (VUs), conditioned on the VU location (cervical, thoracic and lumbar). Here, we present a method for generating a synthetic dataset and conduct an in-depth analysis on its properties along three key metrics: image fidelity, sample diversity and dataset privacy.

Comparing algorithms for characterizing treatment effect heterogeneity in randomized trials.

The identification and estimation of heterogeneous treatment effects in biomedical clinical trials are challenging, because trials are typically planned to assess the treatment effect in the overall trial population. Nevertheless, the identification of how the treatment effect may vary across subgroups is of major importance for drug development. In this work, we review some existing simulation work and perform a simulation study to evaluate recent methods for identifying and estimating the heterogeneous treatments effects using various metrics and scenarios relevant for drug development. Our focus is not only on a comparison of the methods in general, but on how well these methods perform in simulation scenarios that reflect real clinical trials. We provide the R package benchtm that can be used to simulate synthetic biomarker distributions based on real clinical trial data and to create interpretable scenarios to benchmark methods for identification and estimation of treatment effect heterogeneity.

Using knockoffs for controlled predictive biomarker identification.

One of the key challenges of personalized medicine is to identify which patients will respond positively to a given treatment. The area of subgroup identification focuses on this challenge, that is, identifying groups of patients that experience desirable characteristics, such as an enhanced treatment effect. A crucial first step towards the subgroup identification is to identify the baseline variables (eg, biomarkers) that influence the treatment effect, which are known as predictive variables. Many subgroup discovery algorithms return importance scores that capture the variables' predictive strength. However, a major limitation of these scores is that they do not answer the core question: "Which variables are actually predictive?" With our work we answer this question by using the knockoff framework, which is a general framework for controlling the false discovery rate when performing prognostic variable selection. In contrast, our work is the first that uses knockoffs for predictive variable selection. We introduce two novel knockoff filters: one parametric, building on variable importance scores derived from a penalized linear regression model, and one non-parametric, building on causal forest variable importance scores. We conduct extensive simulations to validate performance of the proposed methodology and we also apply the proposed methods to data from a randomized clinical trial.

Sequential knockoffs for continuous and categorical predictors: With application to a large psoriatic arthritis clinical trial pool.

Knockoffs provide a general framework for controlling the false discovery rate when performing variable selection. Much of the Knockoffs literature focuses on theoretical challenges and we recognize a need for bringing some of the current ideas into practice. In this paper we propose a sequential algorithm for generating knockoffs when underlying data consists of both continuous and categorical (factor) variables. Further, we present a heuristic multiple knockoffs approach that offers a practical assessment of how robust the knockoff selection process is for a given dataset. We conduct extensive simulations to validate performance of the proposed methodology. Finally, we demonstrate the utility of the methods on a large clinical data pool of more than 2000 patients with psoriatic arthritis evaluated in four clinical trials with an IL-17A inhibitor, secukinumab (Cosentyx), where we determine prognostic factors of a well established clinical outcome. The analyses presented in this paper could provide a wide range of applications to commonly encountered datasets in medical practice and other fields where variable selection is of particular interest.

Model averaging for treatment effect estimation in subgroups.

In many clinical trials, biological, pharmacological, or clinical information is used to define candidate subgroups of patients that might have a differential treatment effect. Once the trial results are available, interest will focus on subgroups with an increased treatment effect. Estimating a treatment effect for these groups, together with an adequate uncertainty statement is challenging, owing to the resulting "random high" / selection bias. In this paper, we will investigate Bayesian model averaging to address this problem. The general motivation for the use of model averaging is to realize that subgroup selection can be viewed as model selection, so that methods to deal with model selection uncertainty, such as model averaging, can be used also in this setting. Simulations are used to evaluate the performance of the proposed approach. We illustrate it on an example early-phase clinical trial.

A Bayesian hierarchical surrogate outcome model for multiple sclerosis.

The development of novel therapies in multiple sclerosis (MS) is one area where a range of surrogate outcomes are used in various stages of clinical research. While the aim of treatments in MS is to prevent disability, a clinical trial for evaluating a drugs effect on disability progression would require a large sample of patients with many years of follow-up. The early stage of MS is characterized by relapses. To reduce study size and duration, clinical relapses are accepted as primary endpoints in phase III trials. For phase II studies, the primary outcomes are typically lesion counts based on magnetic resonance imaging (MRI), as these are considerably more sensitive than clinical measures for detecting MS activity. Recently, Sormani and colleagues in 'Surrogate endpoints for EDSS worsening in multiple sclerosis' provided a systematic review and used weighted regression analyses to examine the role of either MRI lesions or relapses as trial level surrogate outcomes for disability. We build on this work by developing a Bayesian three-level model, accommodating the two surrogates and the disability endpoint, and properly taking into account that treatment effects are estimated with errors. Specifically, a combination of treatment effects based on MRI lesion count outcomes and clinical relapse was used to develop a study-level surrogate outcome model for the corresponding treatment effects based on disability progression. While the primary aim for developing this model was to support decision-making in drug development, the proposed model may also be considered for future validation. Copyright © 2016 John Wiley & Sons, Ltd.

Characterization of dose-response for count data using a generalized MCP-Mod approach in an adaptive dose-ranging trial.

Understanding the dose-response relationship is a key objective in Phase II clinical development. Yet, designing a dose-ranging trial is a challenging task, as it requires identifying the therapeutic window and the shape of the dose-response curve for a new drug on the basis of a limited number of doses. Adaptive designs have been proposed as a solution to improve both quality and efficiency of Phase II trials as they give the possibility to select the dose to be tested as the trial goes. In this article, we present a 'shapebased' two-stage adaptive trial design where the doses to be tested in the second stage are determined based on the correlation observed between efficacy of the doses tested in the first stage and a set of pre-specified candidate dose-response profiles. At the end of the trial, the data are analyzed using the generalized MCP-Mod approach in order to account for model uncertainty. A simulation study shows that this approach gives more precise estimates of a desired target dose (e.g. ED70) than a single-stage (fixed-dose) design and performs as well as a two-stage D-optimal design. We present the results of an adaptive model-based dose-ranging trial in multiple sclerosis that motivated this research and was conducted using the presented methodology.

A flexible Bayesian approach for modeling monotonic dose-response relationships in drug development trials.

Clinical trials often involve comparing 2-4 doses or regimens of an experimental therapy with a control treatment. These studies might occur early in a drug development process, where the aim might be to demonstrate a basic level of proof (the so-called proof of concept (PoC) studies), at a later stage, to help establish a dose or doses that should be used in phase III trials (dose-finding), or even in confirmatory studies, where the registration of several doses might be considered. When a small number of doses are examined, the ability to implement parametric modeling is somewhat limited. As an alternative, in this paper, a flexible Bayesian model is suggested. In particular, we draw on the idea of using Bayesian model averaging (BMA) to exploit an assumed monotonic dose-response relationship, without using strong parametric assumptions. The approach is exemplified by assessing operating characteristics in the design of a PoC study examining a new treatment for psoriatic arthritis and a post hoc data analysis involving three confirmatory clinical trials, which examined an adjunctive treatment for partial epilepsy. Key difficulties, such as prior specification and computation, are discussed. A further extension, based on combining the flexible modeling with a classical multiple comparisons procedure, known as MCP-MOD, is examined. The benefit of this extension is a potential reduction in the number of simulations that might be needed to investigate operating characteristics of the statistical analysis.

Guidance on the implementation and reporting of a drug safety Bayesian network meta-analysis.

The Drug Information Association Bayesian Scientific Working Group (BSWG) was formed in 2011 with a vision to ensure that Bayesian methods are well understood and broadly utilized for design and analysis and throughout the medical product development process, and to improve industrial, regulatory, and economic decision making. The group, composed of individuals from academia, industry, and regulatory, has as its mission to facilitate the appropriate use and contribute to the progress of Bayesian methodology. In this paper, the safety sub-team of the BSWG explores the use of Bayesian methods when applied to drug safety meta-analysis and network meta-analysis. Guidance is presented on the conduct and reporting of such analyses. We also discuss different structural model assumptions and provide discussion on prior specification. The work is illustrated through a case study involving a network meta-analysis related to the cardiovascular safety of non-steroidal anti-inflammatory drugs.

A practical guide to Bayesian group sequential designs.

Bayesian approaches to the monitoring of group sequential designs have two main advantages compared with classical group sequential designs: first, they facilitate implementation of interim success and futility criteria that are tailored to the subsequent decision making, and second, they allow inclusion of prior information on the treatment difference and on the control group. A general class of Bayesian group sequential designs is presented, where multiple criteria based on the posterior distribution can be defined to reflect clinically meaningful decision criteria on whether to stop or continue the trial at the interim analyses. To evaluate the frequentist operating characteristics of these designs, both simulation methods and numerical integration methods are proposed, as implemented in the corresponding R package gsbDesign. Normal approximations are used to allow fast calculation of these characteristics for various endpoints. The practical implementation of the approach is illustrated with several clinical trial examples from different phases of drug development, with various endpoints, and informative priors.

The current state of Bayesian methods in medical product development: survey results and recommendations from the DIA Bayesian Scientific Working Group.

Bayesian applications in medical product development have recently gained popularity. Despite many advances in Bayesian methodology and computations, increase in application across the various areas of medical product development has been modest. The DIA Bayesian Scientific Working Group (BSWG), which includes representatives from industry, regulatory agencies, and academia, has adopted the vision to ensure Bayesian methods are well understood, accepted more broadly, and appropriately utilized to improve decision making and enhance patient outcomes. As Bayesian applications in medical product development are wide ranging, several sub-teams were formed to focus on various topics such as patient safety, non-inferiority, prior specification, comparative effectiveness, joint modeling, program-wide decision making, analytical tools, and education. The focus of this paper is on the recent effort of the BSWG Education sub-team to administer a Bayesian survey to statisticians across 17 organizations involved in medical product development. We summarize results of this survey, from which we provide recommendations on how to accelerate progress in Bayesian applications throughout medical product development. The survey results support findings from the literature and provide additional insight on regulatory acceptance of Bayesian methods and information on the need for a Bayesian infrastructure within an organization. The survey findings support the claim that only modest progress in areas of education and implementation has been made recently, despite substantial progress in Bayesian statistical research and software availability.

Bayesian modeling and inference for clinical trials with partial retrieved data following dropout.

In randomized clinical trials, it is common that patients may stop taking their assigned treatments and then switch to a standard treatment (standard of care available to the patient) but not the treatments under investigation. Although the availability of limited retrieved data on patients who switch to standard treatment, called off-protocol data, could be highly valuable in assessing the associated treatment effect with the experimental therapy, it leads to a complex data structure requiring the development of models that link the information of per-protocol data with the off-protocol data. In this paper, we develop a novel Bayesian method to jointly model longitudinal treatment measurements under various dropout scenarios. Specifically, we propose a multivariate normal mixed-effects model for repeated measurements from the assigned treatments and the standard treatment, a multivariate logistic regression model for those stopping the assigned treatments, logistic regression models for those starting a standard treatment off protocol, and a conditional multivariate logistic regression model for completely withdrawing from the study. We assume that withdrawing from the study is non-ignorable, but intermittent missingness is assumed to be at random. We examine various properties of the proposed model. We develop an efficient Markov chain Monte Carlo sampling algorithm. We analyze in detail via the proposed method a real dataset from a clinical trial.

Safety of formoterol in adults and children with asthma: a meta-analysis.

BACKGROUND: The safety of long-acting ß2 agonists (LABA) for the treatment of persistent asthma remains a topic of ongoing debate. OBJECTIVE: To evaluate the risk of serious asthma-related events among patients treated with formoterol, a meta-analysis of all Novartis-sponsored controlled clinical trials was conducted. METHODS: Forty-five randomized, placebo- and active-controlled, parallel-group or crossover studies with formoterol were included. Background inhaled corticosteroid (ICS) use was permitted in all studies; however, in only 2 studies was ICS randomized as study medication. Sub-analyses of the pooled data were performed according to age (5-12; 13-18; >18 years), baseline ICS use, and lung function. Odds ratios (OR) and 95% confidence intervals (CIs) were calculated between formoterol (twice-daily), albuterol (salbutamol) 4 times per day (active control), and placebo. RESULTS: Patients were randomized to formoterol (n = 5,367), placebo (n = 2,026), and albuterol (n = 976). Two deaths were reported, 1 each in the formoterol (asthma exacerbation) and the placebo (hemorrhagic pancreatitis) groups. No statistically significant differences in serious asthma exacerbations were observed compared with placebo in adolescents and adults. In children, a higher frequency of hospitalizations was observed among patients treated with formoterol compared with placebo (OR 8.4; 95% CI: 1.1-65.3). A trend toward fewer exacerbations was observed among subjects reporting concomitant ICS use at baseline. CONCLUSIONS: This analysis supports current guideline recommendations for the use of LABAs only as add-on therapy to ICS.

Bayesian models for subgroup analysis in clinical trials.

BACKGROUND: In a pharmaceutical drug development setting, possible interactions between the treatment and particular baseline clinical or demographic factors are often of interest. However, the subgroup analysis required to investigate such associations remains controversial. Concerns with classical hypothesis testing approaches to the problem include low power, multiple testing, and the possibility of data dredging. PURPOSE: As an alternative to hypothesis testing, the use of shrinkage estimation techniques is investigated in the context of an exploratory post hoc subgroup analysis. A range of models that have been suggested in the literature are reviewed. Building on this, we explore a general modeling strategy, considering various options for shrinkage of effect estimates. This is applied to a case-study, in which evidence was available from seven-phase II-III clinical trials examining a novel therapy, and also to two artificial datasets with the same structure. METHODS: Emphasis is placed on hierarchical modeling techniques, adopted within a Bayesian framework using freely available software. A range of possible subgroup model structures are applied, each incorporating shrinkage estimation techniques. RESULTS: The investigation of the case-study showed little evidence of subgroup effects. Because inferences appeared to be consistent across a range of well-supported models, and model diagnostic checks showed no obvious problems, it seemed this conclusion was robust. It is reassuring that the structured shrinkage techniques appeared to work well in a situation where deeper inspection of the data suggested little evidence of subgroup effects. LIMITATIONS: The post hoc examination of subgroups should be seen as an exploratory analysis, used to help make better informed decisions regarding potential future studies examining specific subgroups. To a certain extent, the degree of understanding provided by such assessments will be limited by the quality and quantity of available data. CONCLUSIONS: In light of recent interest by health authorities into the use of subgroup analysis in the context of drug development, it appears that Bayesian approaches involving shrinkage techniques could play an important role in this area. Hopefully, the developments outlined here provide useful methodology for tackling such a problem, in-turn leading to better informed decisions regarding subgroups.

Use of the false discovery rate when comparing multiple health care providers.

OBJECTIVE: Comparisons of the performance of multiple health care providers are often based on hypothesis tests, those with resulting P-values below some critical threshold being identified as potentially extreme. Because of the multiple testing involved, the classical P-value threshold of, say, 0.05 may not be considered strict enough, as it will tend to lead to too many "false positives." However, we argue that the commonly used Bonferroni-corrected threshold is in general too strict for the problem in hand. The purpose of this article is to demonstrate a suitable alternative thresholding procedure that is already well established in other fields. STUDY DESIGN AND SETTING: The suggested procedure involves control of an error measure called the "false discovery rate" (FDR). We present a worked example involving a comparison of risk-adjusted mortality rates following heart surgery in New York State hospitals during 2000-2002. It is shown that the FDR critical threshold lines can be drawn on a "funnel plot," providing a simple graphical presentation of the results. RESULTS: The FDR procedure identified more providers as potentially extreme than the Bonferroni correction, while maintaining control of an intuitively sensible error measure. CONCLUSION: Control of the FDR offers a simple guideline to determining where to draw critical thresholds when comparing multiple health care providers.

Components of variance in a multicentre functional MRI study and implications for calculation of statistical power.

This article firstly presents a theoretical analysis of the statistical power of a parallel-group, repeated-measures (two-session) and two-centre design suitable for a placebo-controlled pharmacological MRI study. For arbitrary effect size, power is determined by the pooled between-session error, the pooled measurement error, the ratio of centre measurement errors, the total number of subjects and the proportion of subjects studied at the centre with greatest measurement error. Secondly, an experiment is described to obtain empirical estimates of variance components in task-related and resting state functional magnetic resonance imaging. Twelve healthy volunteers were scanned at two centres during performance of blocked and event-related versions of an affect processing task (each repeated twice per session) and rest. In activated regions, variance components were estimated: between-subject (23% of total), between-centre (2%), between-paradigm (4%), within-session occasion (paradigm repeat; 2%) and residual (measurement) error (69%). The between-centre ratio of measurement errors was 0.8. A similar analysis for the Hurst exponent estimated in resting data showed negligible contributions of between-subject and between-centre variability; measurement error accounted for 99% of total variance. Substituting these estimates in the theoretical expression for power, incorporation of two centres in the design necessitates a modest (10%) increase in the total number of subjects compared with a single-centre study. Furthermore, considerable improvements in power can be attained by repetition of the task within each scanning session. Thus, theoretical models of power and empirical data indicate that between-centre variability can be small enough to encourage multicentre designs without major compensatory increases in sample size.