Design considerations for two-stage enrichment clinical trials.

When there is a predictive biomarker, enrichment can focus the clinical trial on a benefiting subpopulation. We describe a two-stage enrichment design, in which the first stage is designed to efficiently estimate a threshold and the second stage is a "phase III-like" trial on the enriched population. The goal of this paper is to explore design issues: sample size in Stages 1 and 2, and re-estimation of the Stage 2 sample size following Stage 1. By treating these as separate trials, we can gain insight into how the predictive nature of the biomarker specifically impacts the sample size. We also show that failure to adequately estimate the threshold can have disastrous consequences in the second stage. While any bivariate model could be used, we assume a continuous outcome and continuous biomarker, described by a bivariate normal model. The correlation coefficient between the outcome and biomarker is the key to understanding the behavior of the design, both for predictive and prognostic biomarkers. Through a series of simulations we illustrate the impact of model misspecification, consequences of poor threshold estimation, and requisite sample sizes that depend on the predictive nature of the biomarker. Such insight should be helpful in understanding and designing enrichment trials.

Systems dynamics and the uncertainties of diagnostics, testing and contact tracing for COVID-19.

The current COVID-19 pandemic contains an unprecedented amount of uncertainty and variability and thus, there is a critical need for understanding of the variation documented in the biological, policy, sociological, and infrastructure responses during an epidemic to support decisions at all levels. With the significant asymptomatic spread of the virus and without an immediate vaccine and pharmaceuticals available, the best feasible strategies for testing and diagnostics, contact tracing, and quarantine need to be optimized. With potentially high false negative test results, infected people would not be enrolled in contact-trace programs and thus, may not be quarantined. Similarly, without broad testing, asymptomatic people are not identified and quarantined. Interconnected system dynamics models can be used to optimize strategies for mitigations for decision support during a pandemic. We use a systems dynamics epidemiology model along with other interconnected system models within public health including hospitals, intensive care units, masks, contact tracing, social distancing, and a newly developed testing and diagnostics model to investigate the uncertainties with testing and to optimize strategies for detecting and diagnosing infected people. Using an orthogonal array Latin Hypercube experimental design, we ran 54 simulations each for two scenarios of 10% and 30% asymptomatic people, varying important inputs for testing and social distancing. Systems dynamics modeling, coupled with computer experimental design and statistical analysis can provide rapid and quantitative results for decision support. Our results show that widespread testing, contacting tracing and quarantine can curtail the pandemic through identifying asymptomatic people in the population.

Randomization-based interval estimation in randomized clinical trials.

Randomization-based interval estimation takes into account the particular randomization procedure in the analysis and preserves the confidence level even in the presence of heterogeneity. It is distinguished from population-based confidence intervals with respect to three aspects: definition, computation, and interpretation. The article contributes to the discussion of how to construct a confidence interval for a treatment difference from randomization tests when analyzing data from randomized clinical trials. The discussion covers (i) the definition of a confidence interval for a treatment difference in randomization-based inference, (ii) computational algorithms for efficiently approximating the endpoints of an interval, and (iii) evaluation of statistical properties (ie, coverage probability and interval length) of randomization-based and population-based confidence intervals under a selected set of randomization procedures when assuming heterogeneity in patient outcomes. The method is illustrated with a case study.

Randomization tests for multiarmed randomized clinical trials.

We examine the use of randomization-based inference for analyzing multiarmed randomized clinical trials, including the application of conditional randomization tests to multiple comparisons. The view is taken that the linkage of the statistical test to the experimental design (randomization procedure) should be recognized. A selected collection of randomization procedures generalized to multiarmed treatment allocation is summarized, and generalizations for two randomization procedures that heretofore were designed for only two treatments are developed. We explain the process of computing the randomization test and conditional randomization test via Monte Carlo simulation, developing an efficient algorithm that makes multiple comparisons possible that would not be possible using a standard algorithm, demonstrate the preservation of type I error rate, and explore the relationship of statistical power to the randomization procedure in the presence of a time trend and outliers. We distinguish between the interpretation of the p-value in the randomization test and in the population test and verify that the randomization test can be approximated by the population test on some occasions. Data from two multiarmed clinical trials from the literature are reanalyzed to illustrate the methodology.

Randomization: The forgotten component of the randomized clinical trial.

"…The customary test for an observed difference…is based on an enumeration of the probabilities, on the initial hypothesis that two treatments do not differ in their effects,…of all the various results which would occur if the trial were repeated indefinitely with different random samples of the same size as those actually used." -Peter Armitage ("Sequential tests in prophylactic and therapeutic trials" in Quarterly Journal of Medicine, 1954;23(91):255-274). Randomization has been the hallmark of the clinical trial since Sir Bradford Hill adopted it in the 1946 streptomycin trial. An exploration of the early literature yields three rationales, ie, (i) the incorporation of randomization provides unpredictability in treatment assignments, thereby mitigating selection bias; (ii) randomization tends to ensure similarity in the treatment groups on known and unknown confounders (at least asymptotically); and (iii) the act of randomization itself provides a basis for inference when random sampling is not conducted from a population model. Of these three, rationale (iii) is often forgotten, ignored, or left untaught. Today, randomization is a rote exercise, scarcely considered in protocols or medical journal articles. Yet, the literature of the last century is rich with statistical articles on randomization methods and their consequences, authored by some of the pioneers of the biostatistics and statistics world. In this paper, we review some of this literature and describe very simple methods to rectify some of the oversight. We describe how randomization-based inference can be used for virtually any outcome of interest in a clinical trial. Special mention is made of nonstandard clinical trials situations.

Covariate-adjusted response-adaptive randomization for multi-arm clinical trials using a modified forward looking Gittins index rule.

We introduce a non-myopic, covariate-adjusted response adaptive (CARA) allocation design for multi-armed clinical trials. The allocation scheme is a computationally tractable procedure based on the Gittins index solution to the classic multi-armed bandit problem and extends the procedure recently proposed in Villar et al. (2015). Our proposed CARA randomization procedure is defined by reformulating the bandit problem with covariates into a classic bandit problem in which there are multiple combination arms, considering every arm per each covariate category as a distinct treatment arm. We then apply a heuristically modified Gittins index rule to solve the problem and define allocation probabilities from the resulting solution. We report the efficiency, balance, and ethical performance of our approach compared to existing CARA methods using a recently published clinical trial as motivation. The net savings in terms of expected number of treatment failures is considerably larger and probably enough to make this design attractive for certain studies where known covariates are expected to be important, stratification is not desired, treatment failures have a high ethical cost, and the disease under study is rare. In a two-armed context, this patient benefit advantage comes at the expense of increased variability in the allocation proportions and a reduction in statistical power. However, in a multi-armed context, simple modifications of the proposed CARA rule can be incorporated so that an ethical advantage can be offered without sacrificing power in comparison with balanced designs.

Differential Associations of Socioeconomic Status With Global Brain Volumes and White Matter Lesions in African American and White Adults: the HANDLS SCAN Study.

OBJECTIVE: The aim of the study was to examine interactive relations of race and socioeconomic status (SES) to magnetic resonance imaging (MRI)-assessed global brain outcomes with previously demonstrated prognostic significance for stroke, dementia, and mortality. METHODS: Participants were 147 African Americans (AAs) and whites (ages 33-71 years; 43% AA; 56% female; 26% below poverty) in the Healthy Aging in Neighborhoods of Diversity across the Life Span SCAN substudy. Cranial MRI was conducted using a 3.0 T unit. White matter (WM) lesion volumes and total brain, gray matter, and WM volumes were computed. An SES composite was derived from education and poverty status. RESULTS: Significant interactions of race and SES were observed for WM lesion volume (b = 1.38; η = 0.036; p = .028), total brain (b = 86.72; η = 0.042; p < .001), gray matter (b = 40.16; η = 0.032; p = .003), and WM (b = 46.56; η = 0.050; p < .001). AA participants with low SES exhibited significantly greater WM lesion volumes than white participants with low SES. White participants with higher SES had greater brain volumes than all other groups (albeit within normal range). CONCLUSIONS: Low SES was associated with greater WM pathology-a marker for increased stroke risk-in AAs. Higher SES was associated with greater total brain volume-a putative global indicator of brain health and predictor of mortality-in whites. Findings may reflect environmental and interpersonal stressors encountered by AAs and those of lower SES and could relate to disproportionate rates of stroke, dementia, and mortality.

ERDO - a framework to select an appropriate randomization procedure for clinical trials.

BACKGROUND: Randomization is considered to be a key feature to protect against bias in randomized clinical trials. Randomization induces comparability with respect to known and unknown covariates, mitigates selection bias, and provides a basis for inference. Although various randomization procedures have been proposed, no single procedure performs uniformly best. In the design phase of a clinical trial, the scientist has to decide which randomization procedure to use, taking into account the practical setting of the trial with respect to the potential of bias. Less emphasis has been placed on this important design decision than on analysis, and less support has been available to guide the scientist in making this decision. METHODS: We propose a framework that weights the properties of the randomization procedure with respect to practical needs of the research question to be answered by the clinical trial. In particular, the framework assesses the impact of chronological and selection bias on the probability of a type I error. The framework is applied to a case study with a 2-arm parallel group, single center randomized clinical trial with continuous endpoint, with no-interim analysis, 1:1 allocation and no adaptation in the randomization process. RESULTS: In so doing, we derive scientific arguments for the selection of an appropriate randomization procedure and develop a template which is illustrated in parallel by a case study. Possible extensions are discussed. CONCLUSION: The proposed ERDO framework guides the investigator through a template for the choice of a randomization procedure, and provides easy to use tools for the assessment. The barriers for the thorough reporting and assessment of randomization procedures could be further reduced in the future when regulators and pharmaceutical companies employ similar, standardized frameworks for the choice of a randomization procedure.

On the use of randomization tests following adaptive designs.

Randomization tests (sometimes referred to as "re-randomization" tests) are used in clinical trials, either as an assumption-free confirmation of parametric analyses, or as an independent analysis based on the principle of randomization-based inference. In the context of adaptive randomization, either restricted or response-adaptive procedures, it is unclear how accurate such Monte Carlo approximations are, or how many Monte Carlo sequences to generate. In this paper, we describe several randomization procedures for which there is a known exact or asymptotic distribution of the randomization test. For a special class of procedures, called [Formula: see text], and binary responses, the exact test statistic has a simple closed form. For the limited subset of existing procedures with known exact and asymptotic distributions, we can use these as a benchmark for the accuracy of Monte Carlo randomization techniques. We conclude that Monte Carlo tests are very accurate, and require minimal computation time. For simple tests with binary response in the class of [Formula: see text] procedures, the exact distribution provides the best test, but Monte Carlo approximations can be used when the exact distribution is difficult to compute.

Inference for blocked randomization under a selection bias model.

We provide an asymptotic test to analyze randomized clinical trials that may be subject to selection bias. For normally distributed responses, and under permuted block randomization, we derive a likelihood ratio test of the treatment effect under a selection bias model. A likelihood ratio test of the presence of selection bias arises from the same formulation. We prove that the test is asymptotically chi-square on one degree of freedom. These results correlate well with the likelihood ratio test of Ivanova et al. (2005, Statistics in Medicine 24, 1537-1546) for binary responses, for which they established by simulation that the asymptotic distribution is chi-square. Simulations also show that the test is robust to departures from normality and under another randomization procedure. We illustrate the test by reanalyzing a clinical trial on retinal detachment.

Exact optimum coin bias in Efron's randomization procedure.

Efron's biased coin design is a restricted randomization procedure that has very favorable balancing properties, yet it is fully randomized, in that subjects are always randomized to one of two treatments with a probability less than 1. The parameter of interest is the bias p of the coin, which can range from 0.5 to 1. In this note, we propose a compound optimization strategy that selects p based on a subjected weighting of the relative importance of the two fundamental criteria of interest for restricted randomization mechanisms, namely balance between the treatment assignments and allocation randomness. We use exact and asymptotic distributional properties of Efron's coin to find the optimal p under compound criteria involving imbalance variability, expected imbalance, selection bias, and accidental bias, for both small/moderate trials and large samples.

Novel Use of Ultrasound Elastography to Quantify Muscle Tissue Changes After Dry Needling of Myofascial Trigger Points in Patients With Chronic Myofascial Pain.

OBJECTIVES: To compare a mechanical heterogeneity index derived from ultrasound vibration elastography with physical findings before and after dry-needling treatment of spontaneously painful active myofascial trigger points in the upper trapezius muscle. METHODS: Forty-eight patients with chronic myofascial pain enrolled in a prospective interventional trial of 3 weekly dry-needling treatments for active myofascial trigger points. Trigger points were evaluated at baseline and at treatment completion using palpation, the pressure-pain threshold, and the mechanical heterogeneity index. Thirty patients were reevaluated at 8 weeks. Trigger points that "responded" changed to tissue that was no longer spontaneously painful, with or without the presence of a palpable nodule. Trigger points that "resolved" changed to tissue without a palpable nodule. The mechanical heterogeneity index was defined as the proportion of the upper trapezius muscle that appeared mechanically stiffer on elastography. Statistical significance for comparisons was determined at P < .05. RESULTS: Following 3 dry needle treatments, the mechanical heterogeneity index decreased significantly for the 38 myofascial trigger points (79% of 48) that responded to treatment. Among these, the baseline mechanical heterogeneity index was significantly lower for the 13 trigger points (27% of 38) that resolved, but the decrease after 3 dry needle treatments did not reach significance. The pressure-pain threshold improved significantly for both groups. At 8 weeks, the mechanical heterogeneity index decreased significantly for the 22 trigger points (73% of 30) that responded and for the 10 (45% of 22) that resolved. The pressure-pain threshold improvement was significant for trigger points that responded but did not reach significance for resolved trigger points. CONCLUSIONS: The mechanical heterogeneity index identifies changes in muscle tissue properties that correlate with changes in the myofascial trigger point status after dry needling.

Conditional Monte Carlo randomization tests for regression models.

We discuss the computation of randomization tests for clinical trials of two treatments when the primary outcome is based on a regression model. We begin by revisiting the seminal paper of Gail, Tan, and Piantadosi (1988), and then describe a method based on Monte Carlo generation of randomization sequences. The tests based on this Monte Carlo procedure are design based, in that they incorporate the particular randomization procedure used. We discuss permuted block designs, complete randomization, and biased coin designs. We also use a new technique by Plamadeala and Rosenberger (2012) for simple computation of conditional randomization tests. Like Gail, Tan, and Piantadosi, we focus on residuals from generalized linear models and martingale residuals from survival models. Such techniques do not apply to longitudinal data analysis, and we introduce a method for computation of randomization tests based on the predicted rate of change from a generalized linear mixed model when outcomes are longitudinal. We show, by simulation, that these randomization tests preserve the size and power well under model misspecification.

Optimal design for inference on the threshold of a biomarker.

Enrichment designs with a continuous biomarker require the estimation of a threshold to determine the subpopulation benefitting from the treatment. This article provides the optimal allocation for inference in a two-stage enrichment design for treatment comparisons when a continuous biomarker is suspected to affect patient response. Several design criteria, associated with different trial objectives, are optimized under balanced or Neyman allocation and under equality of the first two empirical biomarker's moments. Moreover, we propose a new covariate-adaptive randomization procedure that converges to the optimum with the fastest available rate. Theoretical and simulation results show that this strategy improves the efficiency of a two-stage enrichment clinical trial, especially with smaller sample sizes and under heterogeneous responses.

Adaptive randomization for clinical trials.

In February 2010, the U.S. Food and Drug Administration (FDA, 2010 ) drafted guidance that discusses the statistical, clinical, and regulatory aspects of various adaptive designs for clinical trials. An important class of adaptive designs is adaptive randomization, which is considered very briefly in subsection VI.B of the guidance. The objective of this paper is to review several important new classes of adaptive randomization procedures and convey information on the recent developments in the literature on this topic. Much of this literature has been focused on the development of methodology to address past criticisms and concerns that have hindered the broader use of adaptive randomization. We conclude that adaptive randomization is a very broad area of experimental design that has important application in modern clinical trials.

Improving Detection of Disease Re-emergence Using a Web-Based Tool (RED Alert): Design and Case Analysis Study.

BACKGROUND: Currently, the identification of infectious disease re-emergence is performed without describing specific quantitative criteria that can be used to identify re-emergence events consistently. This practice may lead to ineffective mitigation. In addition, identification of factors contributing to local disease re-emergence and assessment of global disease re-emergence require access to data about disease incidence and a large number of factors at the local level for the entire world. This paper presents Re-emerging Disease Alert (RED Alert), a web-based tool designed to help public health officials detect and understand infectious disease re-emergence. OBJECTIVE: Our objective is to bring together a variety of disease-related data and analytics needed to help public health analysts answer the following 3 primary questions for detecting and understanding disease re-emergence: Is there a potential disease re-emergence at the local (country) level? What are the potential contributing factors for this re-emergence? Is there a potential for global re-emergence? METHODS: We collected and cleaned disease-related data (eg, case counts, vaccination rates, and indicators related to disease transmission) from several data sources including the World Health Organization (WHO), Pan American Health Organization (PAHO), World Bank, and Gideon. We combined these data with machine learning and visual analytics into a tool called RED Alert to detect re-emergence for the following 4 diseases: measles, cholera, dengue, and yellow fever. We evaluated the performance of the machine learning models for re-emergence detection and reviewed the output of the tool through a number of case studies. RESULTS: Our supervised learning models were able to identify 82%-90% of the local re-emergence events, although with 18%-31% (except 46% for dengue) false positives. This is consistent with our goal of identifying all possible re-emergences while allowing some false positives. The review of the web-based tool through case studies showed that local re-emergence detection was possible and that the tool provided actionable information about potential factors contributing to the local disease re-emergence and trends in global disease re-emergence. CONCLUSIONS: To the best of our knowledge, this is the first tool that focuses specifically on disease re-emergence and addresses the important challenges mentioned above.

Depressive symptoms are associated with subclinical cerebrovascular disease among healthy older women, not men.

BACKGROUND: Associations among diagnosed unipolar depression, depressive symptoms, and cerebrovascular disease are well known. However, minimal research has investigated whether sex may modify such associations, despite known sex differences in depression and depressive symptoms. This study examined whether depressive symptoms were disproportionately related to subclinical cerebrovascular disease (SCD) in women versus men. METHODS: One hundred one older adults (58% men; mean age = 67 years), free of major comorbidities, completed the Beck Depression Inventory and underwent magnetic resonance imaging (MRI). MRI scans were neuroradiologist rated for markers of SCD (periventricular and deep white matter hyperintensities, and number of silent infarcts) and brain atrophy (ventricular enlargement and sulcal widening). Two rank-sum outcome variables (SCD and brain atrophy) were then created. RESULTS: On average, depressive symptoms were relatively low in magnitude (mean = 3.8, standard deviation = 3.6, range = 0-17). Multiple regression analyses, adjusted for age, sex, education, systolic blood pressure, fasting glucose, maximal oxygen consumption, body mass index, average weekly alcohol consumption, and Mini-Mental State Examination performance revealed sex to be a significant effect modifier of depressive symptoms in the prediction of SCD. Sex-stratified regression analyses indicated depressive symptoms, and SCD was strongly related among women but not men. Depressive symptoms were not related to brain atrophy, regardless of inclusion of sex as an effect modifier. CONCLUSIONS: Depressive symptoms, even in a subclinical range, are significantly associated with an MRI-derived index of SCD among women, but not men, in the present sample of relatively healthy older adults.

Case-mix adjustment and enabled reporting of the health care experiences of adults with disabilities.

OBJECTIVES: To develop activity limitation clusters for case-mix adjustment of health care ratings and as a population profiler, and to develop a cognitively accessible report of statistically reliable quality and access measures comparing the health care experiences of adults with and without disabilities, within and across health delivery organizations. DESIGN: Observational study. SETTING: Three California Medicaid health care organizations. PARTICIPANTS: Adults (N = 1086) of working age enrolled for at least 1 year in Medicaid because of disability. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Principal components analysis created 4 clusters of activity limitations that we used to characterize case mix. We identified and calculated 28 quality measures using responses from a proposed enabled version of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey. We calculated scores for overall care as the weighted mean of the case-mix adjusted ratings. RESULTS: Disability caused a greater bias on health plan ratings and specialist ratings than did demographic factors. Proxy respondents rated care the same as self-respondents. Telephone and mail administration were equivalent for service reports, but telephone respondents tended to offer more positive global ratings. Plan-level reliability estimates for new composites on shared decision making and advice on healthy living are .79 and .87, respectively. Plan-level reliability estimates for a new composite measure on family planning did not discriminate between health plans because respondents rated all health plans poorly. Approximately 125 respondents per site are necessary to detect group differences. CONCLUSIONS: Self-reported activity limitations incorporating standard questions from the American Community Survey can be used to create a disability case-mix index and to construct profiles of a population's activity limitations. The enabled comparative report, which we call the Assessment of Health Plans and Providers by People with Activity Limitations, is more cognitively accessible than typical CAHPS report templates for state Medicaid plans. The CAHPS Medicaid reporting tools may provide misleading ratings of health plan and physician quality by people with disabilities because the mean ratings do not account for systematic biases associated with disability. More testing on larger populations would help to quantify the strength of various reporting biases.