Augmenting control arms with real-world data for cancer trials: Hybrid control arm methods and considerations.

Background: Hybrid controlled trials with real-world data (RWD), where the control arm is composed of both trial and real-world patients, could facilitate research when the feasibility of randomized controlled trials (RCTs) is challenging and single-arm trials would provide insufficient information. Methods: We propose a frequentist two-step borrowing method to construct hybrid control arms. We use parameters informed by a completed randomized trial in metastatic triple-negative breast cancer to simulate the operating characteristics of dynamic and static borrowing methods, highlighting key trade-offs and analytic decisions in the design of hybrid studies. Results: Simulated data were generated under varying residual-bias assumptions (no bias: HRRWD = 1) and experimental treatment effects (target trial scenario: HRExp = 0.78). Under the target scenario with no residual bias, all borrowing methods achieved the desired 88% power, an improvement over the reference model (74% power) that does not borrow information externally. The effective number of external events tended to decrease with higher bias between RWD and RCT (i.e. HRRWD away from 1), and with weaker experimental treatment effects (i.e. HRExp closer to 1). All dynamic borrowing methods illustrated (but not the static power prior) cap the maximum Type 1 error over the residual-bias range considered. Our two-step model achieved comparable results for power, type 1 error, and effective number of external events borrowed compared to other borrowing methodologies. Conclusion: By pairing high-quality external data with rigorous simulations, researchers have the potential to design hybrid controlled trials that better meet the needs of patients and drug development.

Concordance of Clinician-Documented and Imaging Response in Patients With Stage IV Non-Small Cell Lung Cancer Treated With First-Line Therapy.

Importance: In observational oncology studies of solid tumors, response to treatment can be evaluated based on electronic health record (EHR) documentation (clinician-assessed response [CAR]), an approach different from standardized radiologist-measured response (Response Evaluation Criteria in Solid Tumours [RECIST] 1.1). Objective: To evaluate the feasibility of an imaging response based on RECIST (IRb-RECIST) and the concordance between CAR and imaging response based on RECIST assessments, and investigate discordance causes. Design, Setting, and Participants: This cohort study used an EHR-derived, deidentified database that included patients with stage IV non-small cell lung cancer (NSCLC) diagnosed between January 1, 2011, to June 30, 2019, selected from 3 study sites. Data analysis was conducted in August, 2020. Exposures: Undergoing first-line therapy and imaging assessments of response to treatment. Main Outcomes and Measures: In this study, CAR assessments (referred to in prior publications as "real-world response" [rwR]) were defined as clinician-documented changes in disease burden at radiologic evaluation time points; they were abstracted manually and assigned to response categories. The RECIST-based assessments accommodated routine practice patterns by using a modified version of RECIST 1.1 (IRb-RECIST), with independent radiology reads. Concordance was calculated as the percent agreement across all response categories and across a dichotomous stratification (response [complete or partial] vs no response), unconfirmed or confirmed. Results: This study found that, in 100 patients evaluated for concordance, agreement between CAR and IRb-RECIST was 71% (95% CI, 61%-80%), and 74% (95% CI, 64%-82%) for confirmed and unconfirmed response, respectively. There were more responders using CAR than IRb-RECIST (40 vs 29 with confirmation; 64 vs 43 without confirmation). The main sources of discordance were the different use of thresholds for tumor size changes by RECIST vs routine care, and unavailable baseline or follow-up scans resulting in inconsistent anatomic coverage over time. Conclusions and Relevance: In this cohort study of patients with stage IV NSCLC, we collected routine-care imaging, showing the feasibility of response evaluation using IRb-RECIST criteria with independent centralized review. Concordance between CAR and centralized IRb-RECIST was moderate. Future work is needed to evaluate the generalizability of these results to broader populations, and investigate concordance in other clinical settings.

Emulating Control Arms for Cancer Clinical Trials Using External Cohorts Created From Electronic Health Record-Derived Real-World Data.

Electronic health record (EHR)-derived real-world data (RWD) can be sourced to create external comparator cohorts to oncology clinical trials. This exploratory study assessed whether EHR-derived patient cohorts could emulate select clinical trial control arms across multiple tumor types. The impact of analytic decisions on emulation results was also evaluated. By digitizing Kaplan-Meier curves, we reconstructed published control arm results from 15 trials that supported drug approvals from January 1, 2016, to April 30, 2018. RWD cohorts were constructed using a nationwide EHR-derived de-identified database by aligning eligibility criteria and weighting to trial baseline characteristics. Trial data and RWD cohorts were compared using Kaplan-Meier and Cox proportional hazards regression models for progression-free survival (PFS) and overall survival (OS; individual cohorts) and multitumor random effects models of hazard ratios (HRs) for median endpoint correlations (across cohorts). Post hoc, the impact of specific analytic decisions on endpoints was assessed using a case study. Comparing trial data and weighted RWD cohorts, PFS results were more similar (HR range = 0.63-1.18, pooled HR = 0.84, correlation of median = 0.91) compared to OS (HR range = 0.36-1.09, pooled HR = 0.76, correlation of median = 0.85). OS HRs were more variable and trended toward worse for RWD cohorts. The post hoc case study had OS HR ranging from 0.67 (95% confidence interval (CI): 0.56-0.79) to 0.92 (95% CI: 0.78-1.09) depending on specific analytic decisions. EHR-derived RWD can emulate oncology clinical trial control arm results, although with variability. Visibility into clinical trial cohort characteristics may shape and refine analytic approaches.

Bayesian additional evidence for decision making under small sample uncertainty.

BACKGROUND: Statistical inference based on small datasets, commonly found in precision oncology, is subject to low power and high uncertainty. In these settings, drawing strong conclusions about future research utility is difficult when using standard inferential measures. It is therefore important to better quantify the uncertainty associated with both significant and non-significant results based on small sample sizes. METHODS: We developed a new method, Bayesian Additional Evidence (BAE), that determines (1) how much additional supportive evidence is needed for a non-significant result to reach Bayesian posterior credibility, or (2) how much additional opposing evidence is needed to render a significant result non-credible. Although based in Bayesian analysis, a prior distribution is not needed; instead, the tipping point output is compared to reasonable effect ranges to draw conclusions. We demonstrate our approach in a comparative effectiveness analysis comparing two treatments in a real world biomarker-defined cohort, and provide guidelines for how to apply BAE in practice. RESULTS: Our initial comparative effectiveness analysis results in a hazard ratio of 0.31 with 95% confidence interval (0.09, 1.1). Applying BAE to this result yields a tipping point of 0.54; thus, an observed hazard ratio of 0.54 or smaller in a replication study would result in posterior credibility for the treatment association. Given that effect sizes in this range are not extreme, and that supportive evidence exists from a similar published study, we conclude that this problem is worthy of further research. CONCLUSIONS: Our proposed method provides a useful framework for interpreting analytic results from small datasets. This can assist researchers in deciding how to interpret and continue their investigations based on an initial analysis that has high uncertainty. Although we illustrated its use in estimating parameters based on time-to-event outcomes, BAE easily applies to any normally-distributed estimator, such as those used for analyzing binary or continuous outcomes.

Stability and Change in Health Behavior Profiles of U.S. Adults.

OBJECTIVES: While understanding of complex within-person clustering of health behaviors into meaningful profiles of risk is growing, we still know little about whether and how U.S. adults transition from one profile to another as they age. This study assesses patterns of stability and change in profiles of tobacco and alcohol use and body mass index (BMI). METHOD: A nationally representative cohort of U.S. adults 25 years and older was interviewed up to 5 times between 1986 and 2011. Latent transition analysis (LTA) models characterized the most common profiles, patterning of transitions across profiles over follow-up, and assessed whether some were associated with higher mortality risk. RESULTS: We identified 5 profiles: "health promoting" with normal BMI and moderate alcohol consumption; "overweight"; "current smokers"; "obese"; and "nondrinkers". Profile membership was largely stable, with the most common transitions to death or weight gain. "Obese" was the most stable profile, while "smokers" were most likely to transition to another profile. Mortality was most frequent in the "obese" and "nondrinker" profiles. DISCUSSION: Stability was more common than transition, suggesting that adults sort into health behavior profiles relatively early. Women and men were differently distributed across profiles at baseline, but showed broad similarity in transitions.

Tests of Matrix Structure for Construct Validation.

Psychologists and other behavioral scientists are frequently interested in whether a questionnaire measures a latent construct. Attempts to address this issue are referred to as construct validation. We describe and extend nonparametric hypothesis testing procedures to assess matrix structures, which can be used for construct validation. These methods are based on a quadratic assignment framework and can be used either by themselves or to check the robustness of other methods. We investigate the performance of these matrix structure tests through simulations and demonstrate their use by analyzing a big five personality traits questionnaire administered as part of the Health and Retirement Study. We also derive rates of convergence for our overall test to better understand its behavior.

Fast approximation of small p-values in permutation tests by partitioning the permutations.

Researchers in genetics and other life sciences commonly use permutation tests to evaluate differences between groups. Permutation tests have desirable properties, including exactness if data are exchangeable, and are applicable even when the distribution of the test statistic is analytically intractable. However, permutation tests can be computationally intensive. We propose both an asymptotic approximation and a resampling algorithm for quickly estimating small permutation p-values (e.g., <10-6) for the difference and ratio of means in two-sample tests. Our methods are based on the distribution of test statistics within and across partitions of the permutations, which we define. In this article, we present our methods and demonstrate their use through simulations and an application to cancer genomic data. Through simulations, we find that our resampling algorithm is more computationally efficient than another leading alternative, particularly for extremely small p-values (e.g., <10-30). Through application to cancer genomic data, we find that our methods can successfully identify up- and down-regulated genes. While we focus on the difference and ratio of means, we speculate that our approaches may work in other settings.