Design and analysis of a clinical trial using previous trials as historical control.

BACKGROUND/AIMS: For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation. METHODS: We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials. RESULTS: We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial. CONCLUSION: This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.

Evidence for generalizability of edaravone efficacy using a novel machine learning risk-based subgroup analysis tool.

Introduction: The edaravone development program for amyotrophic lateral sclerosis (ALS) included trials MCI186-16 (Study 16) and MCI186-19 (Study 19). A cohort enrichment strategy was based on a Study 16 post hoc analysis and applied to Study 19 to elucidate a treatment effect in that study. To determine whether the Study 19 results could be generalized to a broader ALS population, we used a machine learning (ML) model to create a novel risk-based subgroup analysis tool. Methods: A validated ML model was used to rank order all Study 16 participants by predicted time to 50% expected vital capacity. Subjects were stratified into nearest-neighbor risk-based subgroups that were systematically expanded to include the entire Study 16 population. For each subgroup, a statistical analysis generated heat maps that revealed statistically significant effect sizes. Results: A broad region of the Study 16 heat map with significant effect sizes was identified, including up to 70% of the trial population. Incorporating participants identified in the cohort enrichment strategy yielded a broad group comprising 76% of the original participants with a statistically significant treatment effect. This broad group spanned the full range of the functional score progression observed in Study 16. Conclusions: This analysis, applying predictions derived using an ML model to a novel methodology for subgroup identification, ascertained a statistically significant edaravone treatment effect in a cohort of participants with broader disease characteristics than the Study 19 inclusion criteria. This novel methodology may assist clinical interpretation of study results and potentially inform efficient future clinical trial design strategies.

Longitudinal modeling to predict vital capacity in amyotrophic lateral sclerosis.

OBJECTIVES: Death in amyotrophic lateral sclerosis (ALS) patients is related to respiratory failure, which is assessed in clinical settings by measuring vital capacity. We developed ALS-VC, a modeling tool for longitudinal prediction of vital capacity in ALS patients. METHODS: A gradient boosting machine (GBM) model was trained using the PRO-ACT (Pooled Resource Open-access ALS Clinical Trials) database of over 10,000 ALS patient records. We hypothesized that a reliable vital capacity predictive model could be developed using PRO-ACT. RESULTS: The model was used to compare FVC predictions with a 30-day run-in period to predictions made from just baseline. The internal root mean square deviations (RMSD) of the run-in and baseline models were 0.534 and 0.539, respectively, across the 7L FVC range captured in PRO-ACT. The RMSDs of the run-in and baseline models using an unrelated, contemporary external validation dataset (0.553 and 0.538, respectively) were comparable to the internal validation. The model was shown to have similar accuracy for predicting SVC (RMSD = 0.562). The most important features for both run-in and baseline models were "Baseline forced vital capacity" and "Days since baseline." CONCLUSIONS: We developed ALS-VC, a GBM model trained with the PRO-ACT ALS dataset that provides vital capacity predictions generalizable to external datasets. The ALS-VC model could be helpful in advising and counseling patients, and, in clinical trials, it could be used to generate virtual control arms against which observed outcomes could be compared, or used to stratify patients into slowly, average, and rapidly progressing subgroups.

Improved stratification of ALS clinical trials using predicted survival.

INTRODUCTION: In small trials, randomization can fail, leading to differences in patient characteristics across treatment arms, a risk that can be reduced by stratifying using key confounders. In ALS trials, riluzole use (RU) and bulbar onset (BO) have been used for stratification. We hypothesized that randomization could be improved by using a multifactorial prognostic score of predicted survival as a single stratifier. METHODS: We defined a randomization failure as a significant difference between treatment arms on a characteristic. We compared randomization failure rates when stratifying for RU and BO ("traditional stratification") to failure rates when stratifying for predicted survival using a predictive algorithm. We simulated virtual trials using the PRO-ACT database without application of a treatment effect to assess balance between cohorts. We performed 100 randomizations using each stratification method - traditional and algorithmic. We applied these stratification schemes to a randomization simulation with a treatment effect using survival as the endpoint and evaluated sample size and power. RESULTS: Stratification by predicted survival met with fewer failures than traditional stratification. Stratifying predicted survival into tertiles performed best. Stratification by predicted survival was validated with an external dataset, the placebo arm from the BENEFIT-ALS trial. Importantly, we demonstrated a substantial decrease in sample size required to reach statistical power. CONCLUSIONS: Stratifying randomization based on predicted survival using a machine learning algorithm is more likely to maintain balance between trial arms than traditional stratification methods. The methodology described here can translate to smaller, more efficient clinical trials for numerous neurological diseases.

Pilot trial of inosine to elevate urate levels in amyotrophic lateral sclerosis.

OBJECTIVE: To test the safety, tolerability, and urate-elevating capability of the urate precursor inosine taken orally or by feeding tube in people with amyotrophic lateral sclerosis (ALS). METHODS: This was a pilot, open-label trial in 25 participants with ALS. Treatment duration was 12 weeks. The dose of inosine was titrated at pre-specified time points to elevate serum urate levels to 7-8 mg/dL. Primary outcomes were safety (as assessed by the occurrence of adverse events [AEs]) and tolerability (defined as the ability to complete the 12-week study on study drug). Secondary outcomes included biomarkers of oxidative stress and damage. As an exploratory analysis, observed outcomes were compared with a virtual control arm built using prediction algorithms to estimate ALSFRS-R scores. RESULTS: Twenty-four out of 25 participants (96%) completed 12 weeks of study drug treatment. One participant was unable to comply with study visits and was lost to follow-up. Serum urate rose to target levels in 6 weeks. No serious AEs attributed to study drug and no AEs of special concern, such as urolithiasis and gout, occurred. Selected biomarkers of oxidative stress and damage had significant changes during the study period. Observed changes in ALSFRS-R did not differ from baseline predictions. INTERPRETATION: Inosine appeared safe, well tolerated, and effective in raising serum urate levels in people with ALS. These findings, together with epidemiological observations and preclinical data supporting a neuroprotective role of urate in ALS models, provide the rationale for larger clinical trials testing inosine as a potential disease-modifying therapy for ALS.

Predicting disease progression in amyotrophic lateral sclerosis.

OBJECTIVE: It is essential to develop predictive algorithms for Amyotrophic Lateral Sclerosis (ALS) disease progression to allow for efficient clinical trials and patient care. The best existing predictive models rely on several months of baseline data and have only been validated in clinical trial research datasets. We asked whether a model developed using clinical research patient data could be applied to the broader ALS population typically seen at a tertiary care ALS clinic. METHODS: Based on the PRO-ACT ALS database, we developed random forest (RF), pre-slope, and generalized linear (GLM) models to test whether accurate, unbiased models could be created using only baseline data. Secondly, we tested whether a model could be validated with a clinical patient dataset to demonstrate broader applicability. RESULTS: We found that a random forest model using only baseline data could accurately predict disease progression for a clinical trial research dataset as well as a population of patients being treated at a tertiary care clinic. The RF Model outperformed a pre-slope model and was similar to a GLM model in terms of root mean square deviation at early time points. At later time points, the RF Model was far superior to either model. Finally, we found that only the RF Model was unbiased and was less subject to overfitting than either of the other two models when applied to a clinic population. INTERPRETATION: We conclude that the RF Model delivers superior predictions of ALS disease progression.

Being PRO-ACTive: What can a Clinical Trial Database Reveal About ALS?

Advancing research and clinical care, and conducting successful and cost-effective clinical trials requires characterizing a given patient population. To gather a sufficiently large cohort of patients in rare diseases such as amyotrophic lateral sclerosis (ALS), we developed the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) platform. The PRO-ACT database currently consists of >8600 ALS patient records from 17 completed clinical trials, and more trials are being incorporated. The database was launched in an open-access mode in December 2012; since then, >400 researchers from >40 countries have requested the data. This review gives an overview on the research enabled by this resource, through several examples of research already carried out with the goal of improving patient care and understanding the disease. These examples include predicting ALS progression, the simulation of future ALS clinical trials, the verification of previously proposed predictive features, the discovery of novel predictors of ALS progression and survival, the newly identified stratification of patients based on their disease progression profiles, and the development of tools for better clinical trial recruitment and monitoring. Results from these approaches clearly demonstrate the value of large datasets for developing a better understanding of ALS natural history, prognostic factors, patient stratification, and more. The increasing use by the community suggests that further analyses of the PRO-ACT database will continue to reveal more information about this disease that has for so long defied our understanding.