Missing Data Analysis.

Methods for handling missing data in clinical psychology studies are reviewed. Missing data are defined, and a taxonomy of main approaches to analysis is presented, including complete-case and available-case analysis, weighting, maximum likelihood, Bayes, single and multiple imputation, and augmented inverse probability weighting. Missingness mechanisms, which play a key role in the performance of alternative methods, are defined. Approaches to robust inference, and to inference when the mechanism is potentially missing not at random, are discussed.

Multiple imputation of more than one environmental exposure with nondifferential measurement error.

Measurement error is common in environmental epidemiologic studies, but methods for correcting measurement error in regression models with multiple environmental exposures as covariates have not been well investigated. We consider a multiple imputation approach, combining external or internal calibration samples that contain information on both true and error-prone exposures with the main study data of multiple exposures measured with error. We propose a constrained chained equations multiple imputation (CEMI) algorithm that places constraints on the imputation model parameters in the chained equations imputation based on the assumptions of strong nondifferential measurement error. We also extend the constrained CEMI method to accommodate nondetects in the error-prone exposures in the main study data. We estimate the variance of the regression coefficients using the bootstrap with two imputations of each bootstrapped sample. The constrained CEMI method is shown by simulations to outperform existing methods, namely the method that ignores measurement error, classical calibration, and regression prediction, yielding estimated regression coefficients with smaller bias and confidence intervals with coverage close to the nominal level. We apply the proposed method to the Neighborhood Asthma and Allergy Study to investigate the associations between the concentrations of multiple indoor allergens and the fractional exhaled nitric oxide level among asthmatic children in New York City. The constrained CEMI method can be implemented by imposing constraints on the imputation matrix using the mice and bootImpute packages in R.

Multiple Imputation with Massive Data: An Application to the Panel Study of Income Dynamics.

Multiple imputation (MI) is a popular and well-established method for handling missing data in multivariate data sets, but its practicality for use in massive and complex data sets has been questioned. One such data set is the Panel Study of Income Dynamics (PSID), a longstanding and extensive survey of household income and wealth in the United States. Missing data for this survey are currently handled using traditional hot deck methods because of the simple implementation; however, the univariate hot deck results in large random wealth fluctuations. MI is effective but faced with operational challenges. We use a sequential regression/chained-equation approach, using the software IVEware, to multiply impute cross-sectional wealth data in the 2013 PSID, and compare analyses of the resulting imputed data with those from the current hot deck approach. Practical difficulties, such as non-normally distributed variables, skip patterns, categorical variables with many levels, and multicollinearity, are described together with our approaches to overcoming them. We evaluate the imputation quality and validity with internal diagnostics and external benchmarking data. MI produces improvements over the existing hot deck approach by helping preserve correlation structures, such as the associations between PSID wealth components and the relationships between the household net worth and sociodemographic factors, and facilitates completed data analyses with general purposes. MI incorporates highly predictive covariates into imputation models and increases efficiency. We recommend the practical implementation of MI and expect greater gains when the fraction of missing information is large.

A simulation study of diagnostics for selection bias.

A non-probability sampling mechanism arising from non-response or non-selection is likely to bias estimates of parameters with respect to a target population of interest. This bias poses a unique challenge when selection is 'non-ignorable', i.e. dependent upon the unobserved outcome of interest, since it is then undetectable and thus cannot be ameliorated. We extend a simulation study by Nishimura et al. [International Statistical Review, 84, 43-62 (2016)], adding two recently published statistics: the so-called 'standardized measure of unadjusted bias (SMUB)' and 'standardized measure of adjusted bias (SMAB)', which explicitly quantify the extent of bias (in the case of SMUB) or non-ignorable bias (in the case of SMAB) under the assumption that a specified amount of non-ignorable selection exists. Our findings suggest that this new sensitivity diagnostic is more correlated with, and more predictive of, the true, unknown extent of selection bias than other diagnostics, even when the underlying assumed level of non-ignorability is incorrect.

Oliceridine Exhibits Improved Tolerability Compared to Morphine at Equianalgesic Conditions: Exploratory Analysis from Two Phase 3 Randomized Placebo and Active Controlled Trials.

INTRODUCTION: In the management of postoperative acute moderate-to-severe pain, opioids remain an important component. However, conventional opioids have a narrow therapeutic index and are associated with dose-limiting opioid-related adverse events (ORAEs) that can result in worse patient outcomes. Oliceridine, a new intravenous µ-opioid receptor agonist, is shown in nonclinical studies to be biased for G protein signaling (achieving analgesia) with limited recruitment of ß-arrestin (associated with ORAEs). In two phase 3 randomized controlled studies of patients with moderate-to-severe acute pain following hard or soft tissue surgery, in which analgesia was measured using Sum of Pain Intensity Differences (SPID) from baseline over 48 and 24 h (SPID-48 and -24 respectively, oliceridine at demand doses of 0.1, 0.35, or 0.5 mg was highly effective compared to placebo, with a favorable safety profile compared to morphine. This exploratory analysis was conducted to determine whether the safety benefits seen with oliceridine persisted when adjusted for equal levels of analgesia compared to morphine. METHODS: Presence of at least one treatment-emergent ORAE (based on Medical Dictionary for Regulatory Activities [MedDRA]-coded events: hypoxemia, nausea, vomiting, sedation, pruritus, or dizziness) was used as the composite safety endpoint. A logistic regression model was utilized to compare oliceridine (pooled regimens) versus morphine, after controlling for analgesia (using SPID-48 or SPID-24 with pre-rescue scores carried forward 6 h). This analysis excluded patients receiving placebo and was repeated for each study and for pooled data. RESULTS: At a given level of SPID-48 or SPID-24, patients receiving oliceridine were less likely to experience the composite safety endpoint. Although not statistically significant at the 0.05 level in the soft tissue model, the odds ratio (OR) showed a consistent numerical trend for oliceridine, being approximately half that observed with morphine in both the hard (OR 0.499; 95% confidence interval [CI] 0.255, 0.976; p = 0.042) and soft (OR 0.542; 95% CI 0.250, 1.175; p = 0.121) tissue studies. Results from the pooled data were consistent with those observed in the individual studies (OR 0.507; 95% CI 0.304, 0.844; p = 0.009). CONCLUSION: Findings from this exploratory analysis suggest that at comparable levels of analgesia, patients receiving oliceridine were less likely to experience the composite safety endpoint consisting of ORAEs compared to patients treated with morphine. Oliceridine Exhibits Improved Tolerability Compared to Morphine at Equianalgesic Conditions: Exploratory Analysis from Two Phase 3 Randomized Placebo and Active Controlled Trials- A Video (MP4 99188 kb).

Bayesian sensitivity analyses for longitudinal data with dropouts that are potentially missing not at random: A high dimensional pattern-mixture model.

Randomized clinical trials with outcome measured longitudinally are frequently analyzed using either random effect models or generalized estimating equations. Both approaches assume that the dropout mechanism is missing at random (MAR) or missing completely at random (MCAR). We propose a Bayesian pattern-mixture model to incorporate missingness mechanisms that might be missing not at random (MNAR), where the distribution of the outcome measure at the follow-up time tk , conditional on the prior history, differs across the patterns of missing data. We then perform sensitivity analysis on estimates of the parameters of interest. The sensitivity parameters relate the distribution of the outcome of interest between subjects from a missing-data pattern at time tk with that of the observed subjects at time tk . The large number of the sensitivity parameters is reduced by treating them as random with a prior distribution having some pre-specified mean and variance, which are varied to explore the sensitivity of inferences. The missing at random (MAR) mechanism is a special case of the proposed model, allowing a sensitivity analysis of deviations from MAR. The proposed approach is applied to data from the Trial of Preventing Hypertension.

Framework for the treatment and reporting of missing data in observational studies: The Treatment And Reporting of Missing data in Observational Studies framework.

Missing data are ubiquitous in medical research. Although there is increasing guidance on how to handle missing data, practice is changing slowly and misapprehensions abound, particularly in observational research. Importantly, the lack of transparency around methodological decisions is threatening the validity and reproducibility of modern research. We present a practical framework for handling and reporting the analysis of incomplete data in observational studies, which we illustrate using a case study from the Avon Longitudinal Study of Parents and Children. The framework consists of three steps: 1) Develop an analysis plan specifying the analysis model and how missing data are going to be addressed. An important consideration is whether a complete records' analysis is likely to be valid, whether multiple imputation or an alternative approach is likely to offer benefits and whether a sensitivity analysis regarding the missingness mechanism is required; 2) Examine the data, checking the methods outlined in the analysis plan are appropriate, and conduct the preplanned analysis; and 3) Report the results, including a description of the missing data, details on how the missing data were addressed, and the results from all analyses, interpreted in light of the missing data and the clinical relevance. This framework seeks to support researchers in thinking systematically about missing data and transparently reporting the potential effect on the study results, therefore increasing the confidence in and reproducibility of research findings.

A comparison of parametric propensity score-based methods for causal inference with multiple treatments and a binary outcome.

We consider comparative effectiveness research (CER) from observational data with two or more treatments. In observational studies, the estimation of causal effects is prone to bias due to confounders related to both treatment and outcome. Methods based on propensity scores are routinely used to correct for such confounding biases. A large fraction of propensity score methods in the current literature consider the case of either two treatments or continuous outcome. There has been extensive literature with multiple treatment and binary outcome, but interest often lies in the intersection, for which the literature is still evolving. The contribution of this article is to focus on this intersection and compare across methods, some of which are fairly recent. We describe propensity-based methods when more than two treatments are being compared, and the outcome is binary. We assess the relative performance of these methods through a set of simulation studies. The methods are applied to assess the effect of four common therapies for castration-resistant advanced-stage prostate cancer. The data consist of medical and pharmacy claims from a large national private health insurance network, with the adverse outcome being admission to the emergency room within a short time window of treatment initiation.

ASSESSING SELECTION BIAS IN REGRESSION COEFFICIENTS ESTIMATED FROM NONPROBABILITY SAMPLES WITH APPLICATIONS TO GENETICS AND DEMOGRAPHIC SURVEYS.

Selection bias is a serious potential problem for inference about relationships of scientific interest based on samples without well-defined probability sampling mechanisms. Motivated by the potential for selection bias in: (a) estimated relationships of polygenic scores (PGSs) with phenotypes in genetic studies of volunteers and (b) estimated differences in subgroup means in surveys of smartphone users, we derive novel measures of selection bias for estimates of the coefficients in linear and probit regression models fitted to nonprobability samples, when aggregate-level auxiliary data are available for the selected sample and the target population. The measures arise from normal pattern-mixture models that allow analysts to examine the sensitivity of their inferences to assumptions about nonignorable selection in these samples. We examine the effectiveness of the proposed measures in a simulation study and then use them to quantify the selection bias in: (a) estimated PGS-phenotype relationships in a large study of volunteers recruited via Facebook and (b) estimated subgroup differences in mean past-year employment duration in a nonprobability sample of low-educated smartphone users. We evaluate the performance of the measures in these applications using benchmark estimates from large probability samples.

Measures of the Degree of Departure from Ignorable Sample Selection.

With the current focus of survey researchers on "big data" that are not selected by probability sampling, measures of the degree of potential sampling bias arising from this nonrandom selection are sorely needed. Existing indices of this degree of departure from probability sampling, like the R-indicator, are based on functions of the propensity of inclusion in the sample, estimated by modeling the inclusion probability as a function of auxiliary variables. These methods are agnostic about the relationship between the inclusion probability and survey outcomes, which is a crucial feature of the problem. We propose a simple index of degree of departure from ignorable sample selection that corrects this deficiency, which we call the standardized measure of unadjusted bias (SMUB). The index is based on normal pattern-mixture models for nonresponse applied to this sample selection problem and is grounded in the model-based framework of nonignorable selection first proposed in the context of nonresponse by Don Rubin in 1976. The index depends on an inestimable parameter that measures the deviation from selection at random, which ranges between the values zero and one. We propose the use of a central value of this parameter, 0.5, for computing a point index, and computing the values of SMUB at zero and one to provide a range of the index in a sensitivity analysis. We also provide a fully Bayesian approach for computing credible intervals for the SMUB, reflecting uncertainty in the values of all of the input parameters. The proposed methods have been implemented in R and are illustrated using real data from the National Survey of Family Growth.

Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys.

The most widespread method of computing confidence intervals (CIs) in complex surveys is to add and subtract the margin of error (MOE) from the point estimate, where the MOE is the estimated standard error multiplied by the suitable Gaussian quantile. This Wald-type interval is used by the American Community Survey (ACS), the largest US household sample survey. For inferences on small proportions with moderate sample sizes, this method often results in marked under-coverage and lower CI endpoint less than 0. We assess via simulation the coverage and width, in complex sample surveys, of seven alternatives to the Wald interval for a binomial proportion with sample size replaced by the 'effective sample size,' that is, the sample size divided by the design effect. Building on previous work by the present authors, our simulations address the impact of clustering, stratification, different stratum sampling fractions, and stratum-specific proportions. We show that all intervals undercover when there is clustering and design effects are computed from a simple design-based estimator of sampling variance. Coverage can be better calibrated for the alternatives to Wald by improving estimation of the effective sample size through superpopulation modeling. This approach is more effective in our simulations than previously proposed modifications of effective sample size. We recommend intervals of the Wilson or Bayes uniform prior form, with the Jeffreys prior interval not far behind.

A randomized pragmatic trial of telephone-delivered cognitive behavioral-therapy, modafinil, and combination therapy of both for fatigue in multiple sclerosis: The design of the "COMBO-MS" trial.

BACKGROUND: Fatigue is one of the most common and disabling chronic symptoms in multiple sclerosis (MS). Optimization of available treatments for MS-related fatigue has been stymied by lack of comparative effectiveness research that focuses on real-world treatment delivery methods and potential modification of treatment effect by other chronic MS symptoms or disability level. This report describes the design of a patient centered, comparative effectiveness trial of cognitive behavioral-therapy (CBT), modafinil, and combination therapy of both for fatigue in MS ("COMBO-MS"). METHODS: We describe the methods of this pragmatic comparative effectiveness trial that is guided by a team of patient, family, provider, community, and payer stakeholders. Eligible participants with MS and significant fatigue severity are randomly assigned (1:1:1) to received either CBT, modafinil, or a combination of CBT and modafinil for 12 weeks. The primary outcome is change in fatigue impact as measured by the Modified Fatigue Impact Scale (MFIS) at 12 weeks. Secondary outcome measures include ecological momentary assessment (EMA) measures of fatigue intensity, fatigue interference, and fatigability (measured over 7 days' time at baseline and at 12 weeks), and change in MFIS score at 24 weeks. PROJECTED OUTCOMES: We hypothesize that combination therapy will more effectively ameliorate fatigue severity than either monotherapy, and that heterogeneity of treatment effects will be found based on depression status, presence of known or suspected sleep disorder, and disease severity. Study findings will assist patients, providers, payers, and policy makers to provide more effective care for managing fatigue in MS.

Indices of non-ignorable selection bias for proportions estimated from non-probability samples.

Rising costs of survey data collection and declining response rates have caused researchers to turn to non-probability samples to make descriptive statements about populations. However, unlike probability samples, non-probability samples may produce severely biased descriptive estimates due to selection bias. The paper develops and evaluates a simple model-based index of the potential selection bias in estimates of population proportions due to non-ignorable selection mechanisms. The index depends on an inestimable parameter ranging from 0 to 1 that captures the amount of deviation from selection at random and is thus well suited to a sensitivity analysis. We describe modified maximum likelihood and Bayesian estimation approaches and provide new and easy-to-use R functions for their implementation. We use simulation studies to evaluate the ability of the proposed index to reflect selection bias in non-probability samples and show how the index outperforms a previously proposed index that relies on an underlying normality assumption. We demonstrate the use of the index in practice with real data from the National Survey of Family Growth.

Medical-encounter mental health diagnoses, non-fatal injury and polypharmacy indicators of risk for accident death in the US Army enlisted soldiers, 2004-2009.

Accidents are a leading cause of deaths in U.S. active duty personnel. Understanding accident deaths during wartime could facilitate future operational planning and inform risk prevention efforts. This study expands prior research, identifying health risk factors associated with U.S. Army accident deaths during the Afghanistan and Iraq war. Military records for 2004-2009 enlisted, active duty, Regular Army soldiers were analyzed using logistic regression modeling to identify mental health, injury, and polypharmacy (multiple narcotic and/or psychotropic medications) predictors of accident deaths for current, previously, and never deployed groups. Deployed soldiers with anxiety diagnoses showed higher risk for accident deaths. Over half had anxiety diagnoses prior to being deployed, suggesting anticipatory anxiety or symptom recurrence may contribute to high risk. For previously deployed soldiers, traumatic brain injury (TBI) indicated higher risk. Two-thirds of these soldiers had first TBI medical-encounter while non-deployed, but mild, combat-related TBIs may have been undetected during deployments. Post-Traumatic Stress Disorder (PTSD) predicted higher risk for never deployed soldiers, as did polypharmacy which may relate to reasons for deployment ineligibility. Health risk predictors for Army accident deaths are identified and potential practice and policy implications discussed. Further research could test for replicability and expand models to include unobserved factors or modifiable mechanisms related to high risk. PTSD predicted high risk among those never deployed, suggesting importance of identification, treatment, and prevention of non-combat traumatic events. Finally, risk predictors overlapped with those identified for suicides, suggesting effective intervention might reduce both types of deaths.

Correlation of Peripheral Immunity With Rapid Amyotrophic Lateral Sclerosis Progression.

Importance: Amyotrophic lateral sclerosis (ALS) has an immune component, but previous human studies have not examined immune changes over time. Objectives: To assess peripheral inflammatory markers in participants with ALS and healthy control individuals and to track immune changes in ALS and determine whether these changes correlate with disease progression. Design, Setting, and Participants: In this longitudinal cohort study, leukocytes were isolated from peripheral blood samples from 35 controls and 119 participants with ALS at the ALS Clinic of the University of Michigan, Ann Arbor, from June 18, 2014, through May 26, 2016. Follow-up visits occurred every 6 to 12 months. Fifty-one participants with ALS provided samples at multiple points. Immune cell populations were measured and compared between control and ALS groups. Surface marker expression of CD11b+ myeloid cells was also assessed. Changes over time were correlated with disease progression using multivariate regression. Main Outcomes and Measures: The number of immune cells per milliliter of blood and the fold expression of cell surface markers. Multivariate regression models were used to correlate changes in immune metrics with changes on the Revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R). Results: Thirty-five controls (17 women [48.6%] and 18 men [51.4%]; mean [SD] age, 63.5 [9.9] years) and 119 participants with ALS (50 women [42.0%] and 69 men [68.0%]; mean [SD] age, 61.4 [11.5] years) were enrolled. Compared with controls, participants with ALS had increased mean (SEM) counts ( × 106/mL) of total leukocytes (4.57 [0.29; 95% CI, 3.94-5.11] vs 5.53 [0.16; 95% CI, 5.21-5.84]), neutrophils (2.87 [0.23; 95% CI, 2.40-3.35] vs 3.80 [0.12; 95% CI, 3.56-4.04]), CD16+ monocytes (0.03 [0.003; 95% CI, 0.02-0.04] vs 0.04 [0.002; 95% CI, 0.03-0.04]), CD16- monocytes (0.25 [0.02; 95% CI, 0.21-0.30] vs 0.29 [0.01; 95% CI, 0.27-0.31]), and natural killer cells (0.13 [0.02; 95% CI, 0.10-0.17] vs 0.18 [0.01; 95% CI, 0.16-0.21]). We also observed an acute, transient increase in a population of CD11b+ myeloid cells expressing HLA-DR, CD11c, and CX3CR1. Finally, early changes in immune cell numbers had a significant correlation with disease progression measured by change in ALSFRS-R score, particularly neutrophils (-4.37 [95% CI, -6.60 to -2.14] per 11.47 × 104/mL [SD, 58.04 × 104/mL] per year) and CD4 T cells (-30.47 [95% CI, -46.02 to -14.94] per -3.72 × 104/mL [SD, 26.21 × 104/mL] per year). Conclusions and Relevance: Changes in the immune system occur during ALS and may contribute to the pathologic features of ALS.

Launching Effectiveness Research to Guide Practice in Neurosurgery: A National Institute Neurological Disorders and Stroke Workshop Report.

This workshop addressed challenges of clinical research in neurosurgery. Randomized controlled clinical trials (RCTs) have high internal validity, but often insufficiently generalize to real-world practice. Observational studies are inclusive but often lack sufficient rigor. The workshop considered possible solutions, such as (1) statistical methods for demonstrating causality using observational data; (2) characteristics required of a registry supporting effectiveness research; (3) trial designs combining advantages of observational studies and RCTs; and (4) equipoise, an identified challenge for RCTs. In the future, advances in information technology potentially could lead to creation of a massive database where clinical data from all neurosurgeons are integrated and analyzed, ending the separation of clinical research and practice and leading to a new "science of practice."

The treatment of missing data in a large cardiovascular clinical outcomes study.

BACKGROUND: The potential impact of missing data on the results of clinical trials has received heightened attention recently. A National Research Council study provides recommendations for limiting missing data in clinical trial design and conduct, and principles for analysis, including the need for sensitivity analyses to assess robustness of findings to alternative assumptions about the missing data. A Food and Drug Administration advisory committee raised missing data as a serious concern in their review of results from the ATLAS ACS 2 TIMI 51 study, a large clinical trial that assessed rivaroxaban for its ability to reduce the risk of cardiovascular death, myocardial infarction or stroke in patients with acute coronary syndrome. This case study describes a variety of measures that were taken to address concerns about the missing data. METHODS: A range of analyses are described to assess the potential impact of missing data on conclusions. In particular, measures of the amount of missing data are discussed, and the fraction of missing information from multiple imputation is proposed as an alternative measure. The sensitivity analysis in the National Research Council study is modified in the context of survival analysis where some individuals are lost to follow-up. The impact of deviations from ignorable censoring is assessed by differentially increasing the hazard of the primary outcome in the treatment groups and multiply imputing events between dropout and the end of the study. Tipping-point analyses are described, where the deviation from ignorable censoring that results in a reversal of significance of the treatment effect is determined. A study to determine the vital status of participants lost to follow-up was also conducted, and the results of including this additional information are assessed. RESULTS: Sensitivity analyses suggest that findings of the ATLAS ACS 2 TIMI 51 study are robust to missing data; this robustness is reinforced by the follow-up study, since inclusion of data from this study had little impact on the study conclusions. CONCLUSION: Missing data are a serious problem in clinical trials. The methods presented here, namely, the sensitivity analyses, the follow-up study to determine survival of missing cases, and the proposed measurement of missing data via the fraction of missing information, have potential application in other studies involving survival analysis where missing data are a concern.