Modeling tumor measurement data to predict overall survival (OS) in cancer clinical trials.

INTRODUCTION: Longitudinal tumor measurements (TM) are commonly recorded in cancer clinical trials of solid tumors. To define patient response to treatment, the Response Evaluation Criteria in Solid Tumors (RECIST) categorizes the otherwise continuous measurements, which results in substantial information loss. We investigated two modeling approaches to incorporate all available cycle-by-cycle (continuous) TM to predict overall survival (OS) and compare the predictive accuracy of these two approaches to RECIST. MATERIAL AND METHODS: Joint modeling (JM) for longitudinal TM and OS and two-stage modeling with potential time-varying coefficients were utilized to predict OS using data from three trials with cycle-by-cycle TM. The JM approach incorporates TM data collected throughout the course of the clinical trial. The two-stage modeling approach incorporates information from early assessments (before 12 weeks) to predict subsequent OS outcome. The predictive accuracy was quantified by c-indices. RESULTS: Data from 577, 337, and 126 patients were included for the analysis (from two stage IV colorectal cancer trials (N9741, N9841) and an advanced non-small cell lung cancer trial (N0026), respectively). Both the JM and two-stage modeling reached a similar conclusion, i.e. the baseline covariates (age, gender, and race) were mostly not predictive of OS (p-value > 0.05). Quantities derived from TM were strong predictors of OS in the two colorectal cancer trials (p < 0.001 for both association in JM and two-stage modeling parameters); but less so in the lung cancer trial (p = 0.053 for association in JM and p = 0.024 and 0.160 for two-stage modeling parameters). The c-indices from the two-stage modeling were higher than those from a model using RECIST (range: 0.611-0.633 versus 0.586-0.590). The dynamic c-indices from the JM were in the range of 0.627-0.683 indicating good predictive accuracy. CONCLUSION: Both modeling approaches provide highly interpretable and clinical meaningful results; the improved predictive performance compared with RECIST indicates the possibility of deriving better trial endpoints from these approaches.

Principles of Good Clinical Trial Design.

Clinical trials are a fundamental component of medical research and serve as the main route to obtain evidence of the safety and efficacy of treatment before its approval. A trial's ability to provide the intended evidence hinges on appropriate design, background knowledge, trial rationale to sample size, and interim monitoring rules. In this article, we present some general design principles for investigators and their research teams to consider when planning to conduct a trial.

Missing tumor measurement (TM) data in the search for alternative TM-based endpoints in cancer clinical trials.

PURPOSE: Missing data commonly occur in cancer clinical trials (CCT) and may hinder the search for alternative trial endpoints. We consider reasons for missing tumor measurement (TM) data in CCT and how missing TM data are typically handled. We explore the potential impact of missing TM data on predictive ability of a set of TM-based endpoints. METHODS: Literature review identifies reasons for and approaches to handling missing TM data. Data from 3 actual clinical trials were used for illustration. A sensitivity analysis of the potential impact of missing TM data was performed by comparing overall survival (OS) predictive ability of alternative endpoints using observed and imputed data. RESULTS: Reasons for missing TM data in CCT are presented, based on the literature review and the three trials. Although missing TM data impacted individual objective status (e.g. 12-week status changed for 53% of patients in one imputation set), it surprisingly only minimally impacted endpoint predictive ability (e.g. median c-indices of 500 imputed datasets ranged from 0.566 to 0.570 for N9741, 0.592-0.616 for N9841, and 0.542-0.624 for N0026). CONCLUSION: By understanding the reasons for missingness, we can better anticipate them and minimize their occurrence. Our preliminary analysis suggests missing TM data may not impact endpoint predictive ability, but could impact objective response status classification; however these findings require further validation. With response status accepted as an important phase II endpoint in the development of new cancer therapies (including immunotherapy), we urge that in CCT complete TM data collection and adherence to protocol-defined disease evaluation as closely as possible be a priority.

Discussion of Trial Designs for Biomarker Identification and Validation Through the Use of Case Studies.

With the launch of the National Cancer Institute's Precision Medicine Initiative in 2015, there has been a shift to trial designs that tailor health care solutions to individual patients by using a screening platform and by moving away from the one-trial/one-biomarker-at-a-time approach. To make precision medicine a reality, it is critical to identify and validate potential biomarkers to help select patients who will truly benefit from a targeted therapy. In this article, we discuss five trial designs: enrichment, umbrella, basket, subgroup, and window of opportunity. For each trial design, we describe the design characteristics, use ongoing or completed trials as case studies, provide any recent advances to the trial design, and discuss advantages and disadvantages of each design.

Precision oncology: A new era of cancer clinical trials.

Traditionally, site of disease and anatomic staging have been used to define patient populations to be studied in individual cancer clinical trials. In the past decade, however, oncology has become increasingly understood on a cellular and molecular level, with many cancer subtypes being described as a function of biomarkers or tumor genetic mutations. With these changes in the science of oncology have come changes to the way we design and perform clinical trials. Increasingly common are trials tailored to detect enhanced efficacy in a patient subpopulation, e.g. patients with a known biomarker value or whose tumors harbor a specific genetic mutation. Here, we provide an overview of traditional and newer biomarker-based trial designs, and highlight lessons learned through implementation of several ongoing and recently completed trials.

Clinical trial designs incorporating predictive biomarkers.

Development of oncologic therapies has traditionally been performed in a sequence of clinical trials intended to assess safety (phase I), preliminary efficacy (phase II), and improvement over the standard of care (phase III) in homogeneous (in terms of tumor type and disease stage) patient populations. As cancer has become increasingly understood on the molecular level, newer "targeted" drugs that inhibit specific cancer cell growth and survival mechanisms have increased the need for new clinical trial designs, wherein pertinent questions on the relationship between patient biomarkers and response to treatment can be answered. Herein, we review the clinical trial design literature from initial to more recently proposed designs for targeted agents or those treatments hypothesized to have enhanced effectiveness within patient subgroups (e.g., those with a certain biomarker value or who harbor a certain genetic tumor mutation). We also describe a number of real clinical trials where biomarker-based designs have been utilized, including a discussion of their respective advantages and challenges. As cancers become further categorized and/or reclassified according to individual patient and tumor features, we anticipate a continued need for novel trial designs to keep pace with the changing frontier of clinical cancer research.

Clinical Utility of Metrics Based on Tumor Measurements in Phase II Trials to Predict Overall Survival Outcomes in Phase III Trials by Using Resampling Methods.

PURPOSE: Phase II clinical trials inform go/no-go decisions for proceeding to phase III trials, and appropriate end points in phase II trials are critical for facilitating this decision. Phase II solid tumor trials have traditionally used end points such as tumor response defined by Response Evaluation Criteria for Solid Tumors (RECIST). We previously reported that absolute and relative changes in tumor measurements demonstrated potential, but not convincing, improvement over RECIST to predict overall survival (OS). We have evaluated the metrics by using additional measures of clinical utility and data from phase III trials. METHODS: Resampling methods were used to assess the clinical utility of metrics to predict phase III outcomes from simulated phase II trials. In all, 2,000 phase II trials were simulated from four actual phase III trials (two positive for OS and two negative for OS). Cox models for three metrics landmarked at 12 weeks and adjusted for baseline tumor burden were fit for each phase II trial: absolute changes, relative changes, and RECIST. Clinical utility was assessed by positive predictive value and negative predictive value, that is, the probability of a positive or negative phase II trial predicting an effective or ineffective phase III conclusion, by prediction error, and by concordance index (c-index). RESULTS: Absolute and relative change metrics had higher positive predictive value and negative predictive value than RECIST in five of six treatment comparisons and lower prediction error curves in all six. However, differences were negligible. No statistically significant difference in c-index across metrics was found. CONCLUSION: The absolute and relative change metrics are not meaningfully better than RECIST in predicting OS.

Evaluating Continuous Tumor Measurement-Based Metrics as Phase II Endpoints for Predicting Overall Survival.

BACKGROUND: We sought to develop and validate clinically relevant, early assessment continuous tumor measurement-based metrics for predicting overall survival (OS) using the Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 data warehouse. METHODS: Data from 13 trials representing 2096 patients with breast cancer, non-small cell lung cancer (NSCLC), or colorectal cancer were used in a complete case analysis. Tumor measurements from weeks 0-6-12 assessments were used to evaluate the ability of slope (absolute change in tumor size from 0-6 and 6-12 weeks) and percent change (relative change in tumor size from 0-6 and 6-12 weeks) metrics to predict OS using Cox models, adjusted for average baseline tumor size. Metrics were evaluated by discrimination (via concordance or c-index), calibration (goodness-of-fit type statistics), association (hazard ratios), and likelihood (Bayesian Information Criteria), with primary focus on the c-index. All statistical tests were two-sided. RESULTS: Comparison of c-indices suggests slight improvement in predictive ability for the continuous tumor measurement-based metrics vs categorical RECIST response metrics, with slope metrics performing better than percent change metrics for breast cancer and NSCLC. However, these differences were not statistically significant. The goodness-of-fit statistics for the RECIST metrics were as good as or better than those for the continuous metrics. In general, all the metrics performed poorly in breast cancer, compared with NSCLC and colorectal cancer. CONCLUSION: Absolute and relative change in tumor measurements do not demonstrate convincingly improved overall survival predictive ability over the RECIST model. Continued work is necessary to address issues of missing tumor measurements and model selection in identifying improved tumor measurement-based metrics.

Test on existence of histology subtype-specific prognostic signatures among early stage lung adenocarcinoma and squamous cell carcinoma patients using a Cox-model based filter.

BACKGROUND: Non-small cell lung cancer (NSCLC) is the predominant histological type of lung cancer, accounting for up to 85% of cases. Disease stage is commonly used to determine adjuvant treatment eligibility of NSCLC patients, however, it is an imprecise predictor of the prognosis of an individual patient. Currently, many researchers resort to microarray technology for identifying relevant genetic prognostic markers, with particular attention on trimming or extending a Cox regression model. Adenocarcinoma (AC) and squamous cell carcinoma (SCC) are two major histology subtypes of NSCLC. It has been demonstrated that fundamental differences exist in their underlying mechanisms, which motivated us to postulate the existence of specific genes related to the prognosis of each histology subtype. RESULTS: In this article, we propose a simple filter feature selection algorithm with a Cox regression model as the base. Applying this method to real-world microarray data identifies a histology-specific prognostic gene signature. Furthermore, the resulting 32-gene (32/12 for AC/SCC) prognostic signature for early-stage AC and SCC samples has superior predictive ability relative to two relevant prognostic signatures, and has comparable performance with signatures obtained by applying two state-of-the art algorithms separately to AC and SCC samples. CONCLUSIONS: Our proposal is conceptually simple, and straightforward to implement. Furthermore, it can be easily adapted and applied to a range of other research settings.

The direct assignment option as a modular design component: an example for the setting of two predefined subgroups.

BACKGROUND: A phase II design with an option for direct assignment (stop randomization and assign all patients to experimental treatment based on interim analysis, IA) for a predefined subgroup was previously proposed. Here, we illustrate the modularity of the direct assignment option by applying it to the setting of two predefined subgroups and testing for separate subgroup main effects. METHODS: We power the 2-subgroup direct assignment option design with 1 IA (DAD-1) to test for separate subgroup main effects, with assessment of power to detect an interaction in a post-hoc test. Simulations assessed the statistical properties of this design compared to the 2-subgroup balanced randomized design with 1 IA, BRD-1. Different response rates for treatment/control in subgroup 1 (0.4/0.2) and in subgroup 2 (0.1/0.2, 0.4/0.2) were considered. RESULTS: The 2-subgroup DAD-1 preserves power and type I error rate compared to the 2-subgroup BRD-1, while exhibiting reasonable power in a post-hoc test for interaction. CONCLUSION: The direct assignment option is a flexible design component that can be incorporated into broader design frameworks, while maintaining desirable statistical properties, clinical appeal, and logistical simplicity.

Exploring the statistical and clinical impact of two interim analyses on the Phase II design with option for direct assignment.

PURPOSE: The primary goal of Phase II clinical trials is to understand better a treatment's safety and efficacy to inform a Phase III go/no-go decision. Many Phase II designs have been proposed, incorporating randomization, interim analyses, adaptation, and patient selection. The Phase II design with an option for direct assignment (i.e. stop randomization and assign all patients to the experimental arm based on a single interim analysis (IA) at 50% accrual) was recently proposed [An et al., 2012]. We discuss this design in the context of existing designs, and extend it from a single-IA to a two-IA design. METHODS: We compared the statistical properties and clinical relevance of the direct assignment design with two IA (DAD-2) versus a balanced randomized design with two IA (BRD-2) and a direct assignment design with one IA (DAD-1), over a range of response rate ratios (2.0-3.0). RESULTS: The DAD-2 has minimal loss in power (<2.2%) and minimal increase in T1ER (<1.6%) compared to a BRD-2. As many as 80% more patients were treated with experimental vs. control in the DAD-2 than with the BRD-2 (experimental vs. control ratio: 1.8 vs. 1.0), and as many as 64% more in the DAD-2 than with the DAD-1 (1.8 vs. 1.1). We illustrate the DAD-2 using a case study in lung cancer. CONCLUSION: In the spectrum of Phase II designs, the direct assignment design, especially with two IA, provides a middle ground with desirable statistical properties and likely appeal to both clinicians and patients.

Evaluation of alternate categorical tumor metrics and cut points for response categorization using the RECIST 1.1 data warehouse.

PURPOSE: We sought to test and validate the predictive utility of trichotomous tumor response (TriTR; complete response [CR] or partial response [PR] v stable disease [SD] v progressive disease [PD]), disease control rate (DCR; CR/PR/SD v PD), and dichotomous tumor response (DiTR; CR/PR v others) metrics using alternate cut points for PR and PD. The data warehouse assembled to guide the Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 was used. METHODS: Data from 13 trials (5,480 patients with metastatic breast cancer, non-small-cell lung cancer, or colorectal cancer) were randomly split (60:40) into training and validation data sets. In all, 27 pairs of cut points for PR and PD were considered: PR (10% to 50% decrease by 5% increments) and PD (10% to 20% increase by 5% increments), for which 30% and 20% correspond to the RECIST categorization. Cox proportional hazards models with landmark analyses at 12 and 24 weeks stratified by study and number of lesions (fewer than three v three or more) and adjusted for average baseline tumor size were used to assess the impact of each metric on overall survival (OS). Model discrimination was assessed by using the concordance index (c-index). RESULTS: Standard RECIST cut points demonstrated predictive ability similar to the alternate PR and PD cut points. Regardless of tumor type, the TriTR, DiTR, and DCR metrics had similar predictive performance. The 24-week metrics (albeit with higher c-index point estimate) were not meaningfully better than the 12-week metrics. None of the metrics did particularly well for breast cancer. CONCLUSION: Alternative cut points to RECIST standards provided no meaningful improvement in OS prediction. Metrics assessed at 12 weeks have good predictive performance.

Choosing profile double-sampling designs for survival estimation with application to President's Emergency Plan for AIDS Relief evaluation.

Most studies that follow subjects over time are challenged by having some subjects who dropout. Double sampling is a design that selects and devotes resources to intensively pursue and find a subset of these dropouts, then uses data obtained from these to adjust naïve estimates, which are potentially biased by the dropout. Existing methods to estimate survival from double sampling assume a random sample. In limited-resource settings, however, generating accurate estimates using a minimum of resources is important. We propose using double-sampling designs that oversample certain profiles of dropouts as more efficient alternatives to random designs. First, we develop a framework to estimate the survival function under these profile double-sampling designs. We then derive the precision of these designs as a function of the rule for selecting different profiles, in order to identify more efficient designs. We illustrate using data from the United States President's Emergency Plan for AIDS Relief-funded HIV care and treatment program in western Kenya. Our results show why and how more efficient designs should oversample patients with shorter dropout times. Further, our work suggests generalizable practice for more efficient double-sampling designs, which can help maximize efficiency in resource-limited settings.

A review of phase II trial designs for initial marker validation.

Phase II clinical trials aim to identify promising experimental regimens for further testing in phase III trials. In this review article, we focus on phase II designs for initial predictive biomarker validation to determine if a drug should be developed for an unselected patient population or for a biomarker-defined patient subset only. Several prospective designs for biomarker-directed therapy have been proposed, differing primarily in the study population, or randomization scheme, or both. The design choice is driven by scientific rationale, marker prevalence, strength of preliminary evidence, assay performance, and turn-around times for marker assessment. The enrichment design is most appropriate when compelling preliminary evidence suggests treatment benefit in only certain marker-defined subgroups, the all-comers design is useful when preliminary evidence regarding treatment effects in marker subgroups is unclear, and adaptive designs have the most potential in the setting of multiple treatment options and multiple marker-defined subgroups. We recently proposed a 2-stage phase II design that has the option for direct assignment (i.e., stop randomization and assign all patients to the experimental arm in stage 2) based on interim analysis (IA) results. This design not only recognizes the need for randomization but also acknowledges the possibility of promising but inconclusive results after pre-planned IA. Simulation studies demonstrated that the direct assignment-option design has minimal power loss, marginal increase in type I error rates, and reasonable robustness to population shift effects. Systematic evaluation and implementation of these design strategies in the phase II setting are essential for accelerating the clinical validation of biomarker guided-therapy.

A 2-stage phase II design with direct assignment option in stage II for initial marker validation.

Biomarkers are critical to targeted therapies, as they may identify patients more likely to benefit from a treatment. Several prospective designs for biomarker-directed therapy have been previously proposed, differing primarily in the study population, randomization scheme, or both. Recognizing the need for randomization, yet acknowledging the possibility of promising but inconclusive results after a stage I cohort of randomized patients, we propose a 2-stage phase II design on marker-positive patients that allows for direct assignment in a stage II cohort. In stage I, marker-positive patients are equally randomized to receive experimental treatment or control. Stage II has the option to adopt "direct assignment" whereby all patients receive experimental treatment. Through simulation, we studied the power and type I error rate of our design compared with a balanced randomized two-stage design, and conducted sensitivity analyses to study the effect of timing of stage I analysis, population shift effects, and unbalanced randomization. Our proposed design has minimal loss in power (<1.8%) and increased type I error rate (<2.1%) compared with a balanced randomized design. The maximum increase in type I error rate in the presence of a population shift was between 3.1% and 5%, and the loss in power across possible timings of stage I analysis was less than 1.2%. Our proposed design has desirable statistical properties with potential appeal in practice. The direct assignment option, if adopted, provides for an "extended confirmation phase" as an alternative to stopping the trial early for evidence of efficacy in stage I.

Comparison of continuous versus categorical tumor measurement-based metrics to predict overall survival in cancer treatment trials.

PURPOSE: The categorical definition of response assessed via the Response Evaluation Criteria in Solid Tumors has documented limitations. We sought to identify alternative metrics for tumor response that improve prediction of overall survival. EXPERIMENTAL DESIGN: Individual patient data from three North Central Cancer Treatment Group trials (N0026, n = 117; N9741, n = 1,109; and N9841, n = 332) were used. Continuous metrics of tumor size based on longitudinal tumor measurements were considered in addition to a trichotomized response [TriTR: response (complete or partial) vs. stable disease vs. progression). Cox proportional hazards models, adjusted for treatment arm and baseline tumor burden, were used to assess the impact of the metrics on subsequent overall survival, using a landmark analysis approach at 12, 16, and 24 weeks postbaseline. Model discrimination was evaluated by the concordance (c) index. RESULTS: The overall best response rates for the three trials were 26%, 45%, and 25%, respectively. Although nearly all metrics were statistically significantly associated with overall survival at the different landmark time points, the concordance indices (c-index) for the traditional response metrics ranged from 0.59 to 0.65; for the continuous metrics from 0.60 to 0.66; and for the TriTR metrics from 0.64 to 0.69. The c-indices for TriTR at 12 weeks were comparable with those at 16 and 24 weeks. CONCLUSIONS: Continuous tumor measurement-based metrics provided no predictive improvement over traditional response-based metrics or TriTR; TriTR had better predictive ability than best TriTR or confirmed response. If confirmed, TriTR represents a promising endpoint for future phase II trials.

A stochastic simulator of a blood product donation environment with demand spikes and supply shocks.

The availability of an adequate blood supply is a critical public health need. An influenza epidemic or another crisis affecting population mobility could create a critical donor shortage, which could profoundly impact blood availability. We developed a simulation model for the blood supply environment in the United States to assess the likely impact on blood availability of factors such as an epidemic. We developed a simulator of a multi-state model with transitions among states. Weekly numbers of blood units donated and needed were generated by negative binomial stochastic processes. The simulator allows exploration of the blood system under certain conditions of supply and demand rates, and can be used for planning purposes to prepare for sudden changes in the public's health. The simulator incorporates three donor groups (first-time, sporadic, and regular), immigration and emigration, deferral period, and adjustment factors for recruitment. We illustrate possible uses of the simulator by specifying input values for an 8-week flu epidemic, resulting in a moderate supply shock and demand spike (for example, from postponed elective surgeries), and different recruitment strategies. The input values are based in part on data from a regional blood center of the American Red Cross during 1996-2005. Our results from these scenarios suggest that the key to alleviating deficit effects of a system shock may be appropriate timing and duration of recruitment efforts, in turn depending critically on anticipating shocks and rapidly implementing recruitment efforts.