Bespoke instrumental variables with non-ideal reference populations.

Recently, a bespoke instrumental variable method was proposed, which, under certain assumptions, can eliminate bias due to unmeasured confounding when estimating the causal exposure effect among the exposed. This method uses data from both the study population of interest, and a reference population in which the exposure is completely absent. In this paper, we extend the bespoke instrumental variable method to allow for a non-ideal reference population that may include exposed subjects. Such an extension is particularly important in randomized trials with nonadherence, where even subjects in the control arm may have access to the treatment under investigation. We further scrutinize the assumptions underlying the bespoke instrumental method, and caution the reader about the potential non-robustness of the method to these assumptions.

Flexible evaluation of surrogacy in platform studies.

Trial-level surrogates are useful tools for improving the speed and cost effectiveness of trials but surrogates that have not been properly evaluated can cause misleading results. The evaluation procedure is often contextual and depends on the type of trial setting. There have been many proposed methods for trial-level surrogate evaluation, but none, to our knowledge, for the specific setting of platform studies. As platform studies are becoming more popular, methods for surrogate evaluation using them are needed. These studies also offer a rich data resource for surrogate evaluation that would not normally be possible. However, they also offer a set of statistical issues including heterogeneity of the study population, treatments, implementation, and even potentially the quality of the surrogate. We propose the use of a hierarchical Bayesian semiparametric model for the evaluation of potential surrogates using nonparametric priors for the distribution of true effects based on Dirichlet process mixtures. The motivation for this approach is to flexibly model relationships between the treatment effect on the surrogate and the treatment effect on the outcome and also to identify potential clusters with differential surrogate value in a data-driven manner so that treatment effects on the surrogate can be used to reliably predict treatment effects on the clinical outcome. In simulations, we find that our proposed method is superior to a simple, but fairly standard, hierarchical Bayesian method. We demonstrate how our method can be used in a simulated illustrative example (based on the ProBio trial), in which we are able to identify clusters where the surrogate is, and is not useful. We plan to apply our method to the ProBio trial, once it is completed.

Cross-direct effects in settings with two mediators.

When multiple mediators are present, there are additional effects that may be of interest beyond the well-known natural (NDE) and controlled direct effects (CDE). These effects cross the type of control on the mediators, setting one to a constant level and one to its natural level, which differs across subjects. We introduce five such estimands for the cross-CDE and -NDE when two mediators are measured. We consider both the scenario where one mediator is influenced by the other, referred to as sequential mediators, and the scenario where the mediators do not influence each other. Such estimands may be of interest in immunology, as we discuss in relation to measured immunological responses to SARS-CoV-2 vaccination. We provide identifying expressions for the estimands in observational settings where there is no residual confounding, and where intervention, outcome, and mediators are of arbitrary type. We further provide tight symbolic bounds for the estimands in randomized settings where there may be residual confounding of the outcome and mediator relationship and all measured variables are binary.

Propensity weighting plus adjustment in proportional hazards model is not doubly robust.

Recently, it has become common for applied works to combine commonly used survival analysis modeling methods, such as the multivariable Cox model and propensity score weighting, with the intention of forming a doubly robust estimator of an exposure effect hazard ratio that is unbiased in large samples when either the Cox model or the propensity score model is correctly specified. This combination does not, in general, produce a doubly robust estimator, even after regression standardization, when there is truly a causal effect. We demonstrate via simulation this lack of double robustness for the semiparametric Cox model, the Weibull proportional hazards model, and a simple proportional hazards flexible parametric model, with both the latter models fit via maximum likelihood. We provide a novel proof that the combination of propensity score weighting and a proportional hazards survival model, fit either via full or partial likelihood, is consistent under the null of no causal effect of the exposure on the outcome under particular censoring mechanisms if either the propensity score or the outcome model is correctly specified and contains all confounders. Given our results suggesting that double robustness only exists under the null, we outline 2 simple alternative estimators that are doubly robust for the survival difference at a given time point (in the above sense), provided the censoring mechanism can be correctly modeled, and one doubly robust method of estimation for the full survival curve. We provide R code to use these estimators for estimation and inference in the supporting information.

Inverse probability of treatment weighting with generalized linear outcome models for doubly robust estimation.

There are now many options for doubly robust estimation; however, there is a concerning trend in the applied literature to believe that the combination of a propensity score and an adjusted outcome model automatically results in a doubly robust estimator and/or to misuse more complex established doubly robust estimators. A simple alternative, canonical link generalized linear models (GLM) fit via inverse probability of treatment (propensity score) weighted maximum likelihood estimation followed by standardization (the g $$ g $$ -formula) for the average causal effect, is a doubly robust estimation method. Our aim is for the reader not just to be able to use this method, which we refer to as IPTW GLM, for doubly robust estimation, but to fully understand why it has the doubly robust property. For this reason, we define clearly, and in multiple ways, all concepts needed to understand the method and why it is doubly robust. In addition, we want to make very clear that the mere combination of propensity score weighting and an adjusted outcome model does not generally result in a doubly robust estimator. Finally, we hope to dispel the misconception that one can adjust for residual confounding remaining after propensity score weighting by adjusting in the outcome model for what remains 'unbalanced' even when using doubly robust estimators. We provide R code for our simulations and real open-source data examples that can be followed step-by-step to use and hopefully understand the IPTW GLM method. We also compare to a much better-known but still simple doubly robust estimator.

The use, adherence, and evaluation of interactive text-messaging among women admitted to prevention of mother-to-child transmission of HIV care in Kenya (WelTel PMTCT).

BACKGROUND: To improve future mobile health (mHealth) interventions in resource-limited settings, knowledge of participants' adherence to interactive interventions is needed, but previous studies are limited. We aimed to investigate how women in prevention of mother-to-child transmission of HIV (PMTCT) care in Kenya used, adhered to, and evaluated an interactive text-messaging intervention. METHODS: We conducted a cohort study nested within the WelTel PMTCT trial among 299 pregnant women living with HIV aged ≥ 18 years. They received weekly text messages from their first antenatal care visit until 24 months postpartum asking "How are you?". They were instructed to text within 48 h stating that they were "okay" or had a "problem". Healthcare workers phoned non-responders and problem-responders to manage any issue. We used multivariable-adjusted logistic and negative binomial regression to estimate adjusted odds ratios (aORs), rate ratios (aRRs) and 95% confidence intervals (CIs) to assess associations between baseline characteristics and text responses. Perceptions of the intervention were evaluated through interviewer-administered follow-up questionnaires at 24 months postpartum. RESULTS: The 299 participants sent 15,183 (48%) okay-responses and 438 (1%) problem-responses. There were 16,017 (51%) instances of non-response. The proportion of non-responses increased with time and exceeded 50% around 14 months from enrolment. Most reported problems were health related (84%). Having secondary education was associated with reporting a problem (aOR:1.88; 95%CI: 1.08-3.27) compared to having primary education or less. Younger age (18-24 years) was associated with responding to < 50% of messages (aOR:2.20; 95%CI: 1.03-4.72), compared to being 35-44 years. Women with higher than secondary education were less likely (aOR:0.28; 95%CI: 0.13-0.64), to respond to < 50% of messages compared to women with primary education or less. Women who had disclosed their HIV status had a lower rate of non-response (aRR:0.77; 95%CI: 0.60-0.97). In interviews with 176 women, 167 (95%) agreed or strongly agreed that the intervention had been helpful, mainly by improving access to and communication with their healthcare providers (43%). CONCLUSION: In this observational study, women of younger age, lower education, and who had not disclosed their HIV status were less likely to adhere to interactive text-messaging. The majority of those still enrolled at the end of the intervention reported that text-messaging had been helpful, mainly by improving access to healthcare providers. Future mHealth interventions aiming to improve PMTCT care need to be targeted to attract the attention of women with lower education and younger age.

Confirmatory prediction-driven RCTs in comparative effectiveness settings for cancer treatment.

BACKGROUND: Medical advances in the treatment of cancer have allowed the development of multiple approved treatments and prognostic and predictive biomarkers for many types of cancer. Identifying improved treatment strategies among approved treatment options, the study of which is termed comparative effectiveness, using predictive biomarkers is becoming more common. RCTs that incorporate predictive biomarkers into the study design, called prediction-driven RCTs, are needed to rigorously evaluate these treatment strategies. Although researched extensively in the experimental treatment setting, literature is lacking in providing guidance about prediction-driven RCTs in the comparative effectiveness setting. METHODS: Realistic simulations with time-to-event endpoints are used to compare contrasts of clinical utility and provide examples of simulated prediction-driven RCTs in the comparative effectiveness setting. RESULTS: Our proposed contrast for clinical utility accurately estimates the true clinical utility in the comparative effectiveness setting while in some scenarios, the contrast used in current literature does not. DISCUSSION: It is important to properly define contrasts of interest according to the treatment setting. Realistic simulations should be used to choose and evaluate the RCT design(s) able to directly estimate that contrast. In the comparative effectiveness setting, our proposed contrast for clinical utility should be used.

Causal inference in survival analysis under deterministic missingness of confounders in register data.

Long-term register data offer unique opportunities to explore causal effects of treatments on time-to-event outcomes, in well-characterized populations with minimum loss of follow-up. However, the structure of the data may pose methodological challenges. Motivated by the Swedish Renal Registry and estimation of survival differences for renal replacement therapies, we focus on the particular case when an important confounder is not recorded in the early period of the register, so that the entry date to the register deterministically predicts confounder missingness. In addition, an evolving composition of the treatment arms populations, and suspected improved survival outcomes in later periods lead to informative administrative censoring, unless the entry date is appropriately accounted for. We investigate different consequences of these issues on causal effect estimation following multiple imputation of the missing covariate data. We analyse the performance of different combinations of imputation models and estimation methods for the population average survival. We further evaluate the sensitivity of our results to the nature of censoring and misspecification of fitted models. We find that an imputation model including the cumulative baseline hazard, event indicator, covariates and interactions between the cumulative baseline hazard and covariates, followed by regression standardization, leads to the best estimation results overall, in simulations. Standardization has two advantages over inverse probability of treatment weighting here: it can directly account for the informative censoring by including the entry date as a covariate in the outcome model, and allows for straightforward variance computation using readily available software.

Target parameters and bias in non-causal change-score analyses with measurement errors.

In studies where the outcome is a change-score, it is often debated whether or not the analysis should adjust for the baseline score. When the aim is to make causal inference, it has been argued that the two analyses (adjusted vs. unadjusted) target different causal parameters, which may both be relevant. However, these arguments are not applicable when the aim is to make predictions rather than to estimate causal effects. When the scores are measured with error, there have been attempts to quantify the bias resulting from adjustment for the (mis-)measured baseline score or lack thereof. However, these bias results have been derived under an unrealistically simple model, and assuming that the target parameter is the unadjusted (for the true baseline score) association, thus dismissing the adjusted association as a possibly relevant target parameter. In this paper we address these limitations. We argue that, even if the aim is to make predictions, there are two possibly relevant target parameters; one adjusted for the baseline score and one unadjusted. We consider both the simple case when there are no measurement errors, and the more complex case when the scores are measured with error. For the latter case, we consider a more realistic model than previous authors. Under this model we derive analytic expressions for the biases that arise when adjusting or not adjusting for the (mis-)measured baseline score, with respect to the two possible target parameters. Finally, we use these expressions to discuss when adjustment is warranted in change-score analyses.

Confidence bands in survival analysis.

BACKGROUND: Providing estimates of uncertainty for statistical quantities is important for statistical inference. When the statistical quantity of interest is a survival curve, which is a function over time, the appropriate type of uncertainty estimate is a confidence band constructed to account for the correlation between points on the curve, we will call this a simultaneous confidence band. This, however, is not the type of confidence band provided in standard software, which is constructed by joining the confidence intervals at given time points. METHODS: We show that this type of band does not have desirable joint/simultaneous coverage properties in comparison to simultaneous bands. RESULTS: There are different ways of constructing simultaneous confidence bands, and we find that bands based on the likelihood ratio appear to have the most desirable properties. Although there is no standard software available in the three major statistical packages to compute likelihood-based simultaneous bands, we summarise and give code to use available statistical software to construct other simultaneous forms of bands, which we illustrate using a study of colon cancer. CONCLUSIONS: There is a need for more user-friendly statistical software to compute simultaneous confidence bands using the available methods.

Survival stacking with multiple data types using pseudo-observation-based-AUC loss.

There have been many strategies to adapt machine learning algorithms to account for right censored observations in survival data in order to build more accurate risk prediction models. These adaptions have included pre-processing steps such as pseudo-observation transformation of the survival outcome or inverse probability of censoring weighted (IPCW) bootstrapping of the observed binary indicator of an event prior to a time point of interest. These pre-processing steps allow existing or newly developed machine learning methods, which were not specifically developed with time-to-event data in mind, to be applied to right censored survival data for predicting the risk of experiencing an event. Stacking or ensemble methods can improve on risk predictions, but in general, the combination of pseudo-observation-based algorithms, IPCW bootstrapping, IPC weighting of the methods directly, and methods developed specifically for survival has not been considered in the same ensemble. In this paper, we propose an ensemble procedure based on the area under the pseudo-observation-based-time-dependent ROC curve to optimally stack predictions from any survival or survival adapted algorithm. The real application results show that our proposed method can improve on single survival based methods such as survival random forest or on other strategies that use a pre-processing step such as inverse probability of censoring weighted bagging or pseudo-observations alone.

Adjustment for Disease Severity in the Test-Negative Study Design.

The test-negative study design is often used to estimate vaccine effectiveness in influenza studies, but it has also been proposed in the context of other infectious diseases, such as cholera, dengue, or Ebola. It was introduced as a variation of the case-control design, in an attempt to reduce confounding bias due to health-care-seeking behavior, and has quickly gained popularity because of its logistic advantages. However, examination of the directed acyclic graphs that describe the test-negative design reveals that without strong assumptions, the estimated odds ratio derived under this sampling mechanism is not collapsible over the selection variable, such that the results obtained for the sampled individuals cannot be generalized to the whole population. In this paper, we show that adjustment for severity of disease can reduce this bias and, under certain assumptions, makes it possible to unbiasedly estimate a causal odds ratio. We support our findings with extensive simulations and discuss them in the context of recently published cholera test-negative studies of the effectiveness of cholera vaccines.

Predictive cluster level surrogacy in the presence of interference.

Surrogate evaluation is a difficult problem that is made more so by the presence of interference. Our proposed procedure can allow for relatively easy evaluation of surrogates for indirect or spill-over clinical effects at the cluster level. Our definition of surrogacy is based on the causal-association paradigm (Joffe and Greene, 2009. Related causal frameworks for surrogate outcomes. Biometrics65, 530-538), under which surrogates are evaluated by the strength of the association between a causal treatment effect on the clinical outcome and a causal treatment effect on the candidate surrogate. Hudgens and Halloran (2008, Toward causal inference with interference. Journal of the American Statistical Association103, 832-842) introduced estimators that can be used for many of the marginal causal estimands of interest in the presence of interference. We extend these to consider surrogates for not just direct effects, but indirect and total effects at the cluster level. We suggest existing estimators that can be used to evaluate biomarkers under our proposed definition of surrogacy. In our motivating setting of a transmission blocking malaria vaccine, there is expected to be no direct protection to those vaccinated and predictive surrogates are urgently needed. We use a set of simulated data examples based on the proposed Phase IIb/III trial design of transmission blocking malaria vaccine to demonstrate how our definition, proposed criteria and procedure can be used to identify biomarkers as predictive cluster level surrogates in the presence of interference on the clinical outcome.

Prediction-driven pooled testing methods: Application to HIV treatment monitoring in Rakai, Uganda.

Chronic medical conditions often necessitate regular testing for proper treatment. Regular testing of all afflicted individuals may not be feasible due to limited resources, as is true with HIV monitoring in resource-limited settings. Pooled testing methods have been developed in order to allow regular testing for all while reducing resource burden. However, the most commonly used methods do not make use of covariate information predictive of treatment failure, which could improve performance. We propose and evaluate four prediction-driven pooled testing methods that incorporate covariate information to improve pooled testing performance. We then compare these methods in the HIV treatment management setting to current methods with respect to testing efficiency, sensitivity, and number of testing rounds using simulated data and data collected in Rakai, Uganda. Results show that the prediction-driven methods increase efficiency by up to 20% compared with current methods while maintaining equivalent sensitivity and reducing number of testing rounds by up to 70%. When predictions were incorrect, the performance of prediction-based matrix methods remained robust. The best performing method using our motivating data from Rakai was a prediction-driven hybrid method, maintaining sensitivity over 96% and efficiency over 75% in likely scenarios. If these methods perform similarly in the field, they may contribute to improving mortality and reducing transmission in resource-limited settings. Although we evaluate our proposed pooling methods in the HIV treatment setting, they can be applied to any setting that necessitates testing of a quantitative biomarker for a threshold-based decision.

Aim for Clinical Utility, Not Just Predictive Accuracy.

The predictions from an accurate prognostic model can be of great interest to patients and clinicians. When predictions are reported to individuals, they may decide to take action to improve their health or they may simply be comforted by the knowledge. However, if there is a clearly defined space of actions in the clinical context, a formal decision rule based on the prediction has the potential to have a much broader impact. The use of a prediction-based decision rule should be formalized and preferably compared with the standard of care in a randomized trial to assess its clinical utility; however, evidence is needed to motivate such a trial. We outline how observational data can be used to propose a decision rule based on a prognostic prediction model. We then propose a framework for emulating a prediction driven trial to evaluate the clinical utility of a prediction-based decision rule in observational data. A split-sample structure is often feasible and useful to develop the prognostic model, define the decision rule, and evaluate its clinical utility. See video abstract at, http://links.lww.com/EDE/B656.

A unified evaluation of differential vaccine efficacy.

Many infectious diseases are well prevented by proper vaccination. However, when a vaccine is not completely efficacious, there is great interest in determining how the vaccine effect differs in subgroups conditional on measured immune responses postvaccination and also according to the type of infecting agent (eg, strain of a virus). The former is often called correlate of protection (CoP) analysis, while the latter has been called sieve analysis. We propose a unified framework for simultaneously assessing CoP and sieve effects of a vaccine in a large Phase III randomized trial. We use flexible parametric models treating times to infection from different agents as competing risks and estimated maximum likelihood to fit the models. The parametric models under competing risks allow for estimation of both cumulative incidence-based contrasts and instantaneous rates. We outline the assumptions with which we can link the observable data to the causal contrasts of interest, propose hypothesis testing procedures, and evaluate our proposed methods in an extensive simulation study.

Social concerns related to HIV status disclosure and participation in the prevention of mother-to-child transmission of HIV care among pregnant women in Kenya.

BACKGROUND: Social concerns about unintentional HIV status disclosure and HIV-related stigma are barriers to pregnant women's access to prevention of mother-to-child transmission of HIV (PMTCT) care. There is limited quantitative evidence of women's social and emotional barriers to PMTCT care and HIV disclosure. We aimed to investigate how social concerns related to participation in PMTCT care are associated with HIV status disclosure to partners and relatives among pregnant women living with HIV in western Kenya. METHODS: A cross-sectional study, including 437 pregnant women living with HIV, was carried out at enrolment in a multicentre mobile phone intervention trial (WelTel PMTCT) in western Kenya. Women diagnosed with HIV on the day of enrolment were excluded. To investigate social concerns and their association with HIV disclosure we used multivariable-adjusted logistic regression, adjusted for sociodemographic and HIV-related characteristics, to estimate odds ratios (OR) and 95% confidence intervals (CI). RESULTS: The majority (80%) had disclosed their HIV status to a current partner and 46% to a relative. Older women (35-44 years) had lower odds of disclosure to a partner (OR = 0.15; 95% CI: 0.05-0.44) compared to women 18-24 years. The most common social concern was involuntary HIV status disclosure (reported by 21%). Concern about isolation or lack of support from family or friends was reported by 9%, and was associated with lower odds of disclosure to partners (OR = 0.33; 95% CI: 0.12-0.85) and relatives (OR = 0.37; 95% CI: 0.16-0.85). Concern about separation (reported by 5%; OR = 0.17; 95% CI: 0.05-0.57), and concern about conflict with a partner (reported by 5%; OR = 0.18; 95% CI: 0.05-0.67), was associated with lower odds of disclosure to a partner. CONCLUSIONS: Compared to previous reports from Kenya, our estimated disclosure rate to a partner is higher, suggesting a possible improvement over time in disclosure. Younger pregnant women appear to be more likely to disclose, suggesting a possible decreased stigma and more openness about HIV among younger couples. Healthcare providers and future interventional studies seeking to increase partner disclosure should consider supporting women regarding their concerns about isolation, lack of support, separation, and conflict with a partner. PMTCT care should be organized to ensure women's privacy and confidentiality.

Evaluation and comparison of predictive individual-level general surrogates.

An intermediate response measure that accurately predicts efficacy in a new setting at the individual level could be used both for prediction and personalized medical decisions. In this article, we define a predictive individual-level general surrogate (PIGS), which is an individual-level intermediate response that can be used to accurately predict individual efficacy in a new setting. While methods for evaluating trial-level general surrogates, which are predictors of trial-level efficacy, have been developed previously, few, if any, methods have been developed to evaluate individual-level general surrogates, and no methods have formalized the use of cross-validation to quantify the expected prediction error. Our proposed method uses existing methods of individual-level surrogate evaluation within a given clinical trial setting in combination with cross-validation over a set of clinical trials to evaluate surrogate quality and to estimate the absolute prediction error that is expected in a new trial setting when using a PIGS. Simulations show that our method performs well across a variety of scenarios. We use our method to evaluate and to compare candidate individual-level general surrogates over a set of multi-national trials of a pentavalent rotavirus vaccine.

Optimizing and evaluating biomarker combinations as trial-level general surrogates.

We extend the method proposed in a recent work by the Authors for trial-level general surrogate evaluation to allow combinations of biomarkers and provide a procedure for finding the "best" combination of biomarkers based on the absolute prediction error summary of surrogate quality. We use a nonparametric Bayesian model that allows us to select an optimal subset of biomarkers without having to consider a large number of explicit model specifications for that subset. This dramatically reduces the number of model comparisons needed. Given the model's flexibility, complex nonlinear relationships can be fit when enough data are available. We evaluate the operating characteristics of our proposed method in simulations designed to be similar to our motivating example. We use our method to compare and evaluate combinations of biomarkers as trial-level general surrogates for the pentavalent rotavirus vaccine RotaTeq™ (RV5) (Merck & Co, Inc, Kenilworth, New Jersey, USA), finding that the same single biomarker identified in our previously published analysis is likely the optimal subset.

A boundary-optimized rejection region test for the two-sample binomial problem.

Testing the equality of 2 proportions for a control group versus a treatment group is a well-researched statistical problem. In some settings, there may be strong historical data that allow one to reliably expect that the control proportion is one, or nearly so. While one-sample tests or comparisons to historical controls could be used, neither can rigorously control the type I error rate in the event the true control rate changes. In this work, we propose an unconditional exact test that exploits the historical information while controlling the type I error rate. We sequentially construct a rejection region by first maximizing the rejection region in the space where all controls have an event, subject to the constraint that our type I error rate does not exceed α for any true event rate; then with any remaining α we maximize the additional rejection region in the space where one control avoids the event, and so on. When the true control event rate is one, our test is the most powerful nonrandomized test for all points in the alternative space. When the true control event rate is nearly one, we demonstrate that our test has equal or higher mean power, averaging over the alternative space, than a variety of well-known tests. For the comparison of 4 controls and 4 treated subjects, our proposed test has higher power than all comparator tests. We demonstrate the properties of our proposed test by simulation and use our method to design a malaria vaccine trial.