Characterizing Imbalance in the Tails of the Propensity Score Distribution.

Understanding characteristics of patients with propensity scores in the tails of the propensity score (PS) distribution has relevance for inverse-probability-of-treatment-weighted and PS-based estimation in observational studies. Here we outline a method for identifying variables most responsible for extreme propensity scores. The approach is illustrated in 3 scenarios: 1) a plasmode simulation of adult patients in the National Ambulatory Medical Care Survey (2011-2015) and 2) timing of dexamethasone initiation and 3) timing of remdesivir initiation in patients hospitalized for coronavirus disease 2019 from February 2020 through January 2021. PS models were fitted using relevant baseline covariates, and tails of the PS distribution were defined using asymmetric first and 99th percentiles. After fitting of the PS model in each original data set, values of each key covariate were permuted and model-agnostic variable importance measures were examined. Visualization and variable importance techniques were helpful in identifying variables most responsible for extreme propensity scores and may help identify individual characteristics that might make patients inappropriate for inclusion in a study (e.g., off-label use). Subsetting or restricting the study sample based on variables identified using this approach may help investigators avoid the need for trimming or overlap weights in studies.

Comparing PCSK9 Monoclonal Antibody Treatment Strategies Following Myocardial Infarction Using Negative Control Outcomes: A Target Trial Emulation Study.

BACKGROUND: Initiation of proprotein convertase subtilisin/kexin type 9 monoclonal antibody (PCSK9 mAb) for lipid-lowering following myocardial infarction (MI) is likely affected by patients' prognostic factors, potentially leading to bias when comparing real-world treatment effects. METHODS: Using target-trial emulation, we assessed potential confounding when comparing two treatment strategies post-MI: initiation of PCSK9 mAb within 1 year and no initiation of PCSK9 mAb. We identified MI hospitalizations during July 2015-June 2020 for patients aged ≥18 years in Optum's de-identified Clinformatics Data Mart (CDM) and MarketScan, and those aged ≥66 in the US Medicare claims database. We estimated a 3-year counterfactual cumulative risk and risk difference (RD) for 10 negative control outcomes using the clone-censor-weight approach to address time-varying confounding and immortal person-time. RESULTS: PCSK9 mAb initiation within 1-year post-MI was low (0.7% in MarketScan and 0.4% in both CDM and Medicare databases). In CDM, there was a lower risk for cancer (RD = -3.6% [95% CI: -4.3%, -2.9%]), decubitus ulcer (RD = -7.7% [95% CI: -11.8%, -3.7%]), fracture (RD = -8.1% [95% CI: -9.6%, -6.6%]), influenza vaccine (RD = -9.3% [95% CI: -17.5%, -1.1%]), and visual test (RD = -0.6% [95% CI: -0.7%, -0.6%]) under the PCSK9 mAb initiation versus no initiation strategy. Similar differences persisted in the MarketScan and Medicare databases. In each database, ezetimibe and low-density lipoprotein testing were unbalanced between treatment strategies. CONCLUSION: A comparative effectiveness study of these treatments using the current approach would likely bias results due to the low number of PCSK9 mAb initiators.

Illustration of 2 Fusion Designs and Estimators.

"Fusion" study designs combine data from different sources to answer questions that could not be answered (as well) by subsets of the data. Studies that augment main study data with validation data, as in measurement-error correction studies or generalizability studies, are examples of fusion designs. Fusion estimators, here solutions to stacked estimating functions, produce consistent answers to identified research questions using data from fusion designs. In this paper, we describe a pair of examples of fusion designs and estimators, one where we generalize a proportion to a target population and one where we correct measurement error in a proportion. For each case, we present an example motivated by human immunodeficiency virus research and summarize results from simulation studies. Simulations demonstrate that the fusion estimators provide approximately unbiased results with appropriate 95% confidence interval coverage. Fusion estimators can be used to appropriately combine data in answering important questions that benefit from multiple sources of information.

Effectiveness of Remdesivir Treatment Protocols Among Patients Hospitalized with COVID-19: A Target Trial Emulation.

BACKGROUND: Remdesivir is recommended for certain hospitalized patients with COVID-19. However, these recommendations are based on evidence from small randomized trials, early observational studies, or expert opinion. Further investigation is needed to better inform treatment guidelines with regard to the effectiveness of remdesivir among these patients. METHODS: We emulated a randomized target trial using chargemaster data from 333 US hospitals from 1 May 2020 to 31 December 2021. We compared three treatment protocols: remdesivir within 2 days of hospital admission, no remdesivir within the first 2 days of admission, and no remdesivir ever. We used baseline comorbidities recorded from encounters up to 12 months before admission and identified the use of in-hospital medications, procedures, and oxygen supplementation from charges. We estimated the cumulative incidence of mortality or mechanical ventilation/extracorporeal membrane oxygenation with an inverse probability of censoring weighted estimator. We conducted analyses in the total population as well as in subgroups stratified by level of oxygen supplementation. RESULTS: A total of 274,319 adult patients met the eligibility criteria for the study. Thirty-day in-hospital mortality risk differences for patients adhering to the early remdesivir protocol were -3.1% (95% confidence interval = -3.5%, -2.7%) compared to no early remdesivir and -3.7% (95% confidence interval -4.2%, -3.2%) compared to never remdesivir, with the strongest effect in patients needing high-flow oxygen. For mechanical ventilation/extracorporeal membrane oxygenation, risk differences were minimal. CONCLUSIONS: We estimate that, among hospitalized patients with COVID-19, remdesivir treatment within 2 days of admission reduced 30-day in-hospital mortality, particularly for patients receiving supplemental oxygen on the day of admission.

Pragmatic considerations for negative control outcome studies to guide non-randomized comparative analyses: A narrative review.

PURPOSE: This narrative review describes the application of negative control outcome (NCO) methods to assess potential bias due to unmeasured or mismeasured confounders in non-randomized comparisons of drug effectiveness and safety. An NCO is assumed to have no causal relationship with a treatment under study while subject to the same confounding structure as the treatment and outcome of interest; an association between treatment and NCO then reflects the potential for uncontrolled confounding between treatment and outcome. METHODS: We focus on two recently completed NCO studies that assessed the comparability of outcome risk for patients initiating different osteoporosis medications and lipid-lowering therapies, illustrating several ways in which confounding may result. In these studies, NCO methods were implemented in claims-based data sources, with the results used to guide the decision to proceed with comparative effectiveness or safety analyses. RESULTS: Based on this research, we provide recommendations for future NCO studies, including considerations for the identification of confounding mechanisms in the target patient population, the selection of NCOs expected to satisfy required assumptions, the interpretation of NCO effect estimates, and the mitigation of uncontrolled confounding detected in NCO analyses. We propose the use of NCO studies prior to initiating comparative effectiveness or safety research, providing information on the potential presence of uncontrolled confounding in those comparative analyses. CONCLUSIONS: Given the increasing use of non-randomized designs for regulatory decision-making, the application of NCO methods will strengthen study design, analysis, and interpretation of real-world data and the credibility of the resulting real-world evidence.

Use of negative control outcomes to assess the comparability of patients initiating lipid-lowering therapies.

PURPOSE: Clinical trials have demonstrated efficacy of proprotein convertase subtilisin/kexin type 9 inhibitors (PCSK9i) in reducing risk of cardiovascular disease events, but effectiveness in routine clinical care has not been well-studied. We used negative control outcomes to assess potential confounding in an observational study of PCSK9i versus ezetimibe or high-intensity statin. METHODS: Using commercial claims, we identified U.S. adults initiating PCSK9i, ezetimibe, or high-intensity statin in 2015-2018, with other lipid-lowering therapy (LLT) use in the year prior (LLT cohort) or atherosclerotic cardiovascular disease (ASCVD) in the past 90 days (ASCVD cohort). We compared initiators of PCSK9i to ezetimibe and high-intensity statin by estimating one-year risks of negative control outcomes influenced by frailty or health-seeking behaviors. Inverse probability of treatment and censoring weighted estimators of risk differences (RDs) were used to evaluate residual confounding after controlling for covariates. RESULTS: PCSK9i initiators had lower one-year risks of negative control outcomes associated with frailty, such as decubitus ulcer in the ASCVD cohort (PCSK9i vs. high-intensity statin RD = -3.5%, 95% confidence interval (CI): -4.6%, -2.5%; PCSK9i vs. ezetimibe RD = -1.3%, 95% CI: -2.1%, -0.6%), with similar but attenuated associations in the LLT cohort. Lower risks of accidents and fractures were also observed for PCSK9i, varying by cohort. Risks were similar for outcomes associated with health-seeking behaviors, although trended higher for PCSK9i in the ASCVD cohort. CONCLUSIONS: Observed associations suggest lower frailty and potentially greater health-seeking behaviors among PCSK9i initiators, particularly those with a recent ASCVD diagnosis, with the potential to bias real-world analyses of treatment effectiveness.

Directed Acyclic Graphs, Effect Measure Modification, and Generalizability.

Directed acyclic graphs (DAGs) have had a major impact on the field of epidemiology by providing straightforward graphical rules for determining when estimates are expected to lack causally interpretable internal validity. Much less attention has been paid, however, to what DAGs can tell researchers about effect measure modification and external validity. In this work, we describe 2 rules based on DAGs related to effect measure modification. Rule 1 states that if a variable, $P$, is conditionally independent of an outcome, $Y$, within levels of a treatment, $X$, then $P$ is not an effect measure modifier for the effect of $X$ on $Y$ on any scale. Rule 2 states that if $P$ is not conditionally independent of $Y$ within levels of $X$, and there are open causal paths from $X$ to $Y$ within levels of $P$, then $P$ is an effect measure modifier for the effect of $X$ on $Y$ on at least 1 scale (given no exact cancelation of associations). We then show how Rule 1 can be used to identify sufficient adjustment sets to generalize nested trials studying the effect of $X$ on $Y$ to the total source population or to those who did not participate in the trial.

Comparing Parametric, Nonparametric, and Semiparametric Estimators: The Weibull Trials.

We use simple examples to show how the bias and standard error of an estimator depend in part on the type of estimator chosen from among parametric, nonparametric, and semiparametric candidates. We estimated the cumulative distribution function in the presence of missing data with and without an auxiliary variable. Simulation results mirrored theoretical expectations about the bias and precision of candidate estimators. Specifically, parametric maximum likelihood estimators performed best but must be "omnisciently" correctly specified. An augmented inverse probability-weighted (IPW) semiparametric estimator performed best among candidate estimators that were not omnisciently correct. In one setting, the augmented IPW estimator reduced the standard error by nearly 30%, compared with a standard Horvitz-Thompson IPW estimator; such a standard error reduction is equivalent to doubling the sample size. These results highlight the gains and losses that can be incurred when model assumptions are made in any analysis.

Machine Learning for Causal Inference: On the Use of Cross-fit Estimators.

BACKGROUND: Modern causal inference methods allow machine learning to be used to weaken parametric modeling assumptions. However, the use of machine learning may result in complications for inference. Doubly robust cross-fit estimators have been proposed to yield better statistical properties. METHODS: We conducted a simulation study to assess the performance of several different estimators for the average causal effect. The data generating mechanisms for the simulated treatment and outcome included log-transforms, polynomial terms, and discontinuities. We compared singly robust estimators (g-computation, inverse probability weighting) and doubly robust estimators (augmented inverse probability weighting, targeted maximum likelihood estimation). We estimated nuisance functions with parametric models and ensemble machine learning separately. We further assessed doubly robust cross-fit estimators. RESULTS: With correctly specified parametric models, all of the estimators were unbiased and confidence intervals achieved nominal coverage. When used with machine learning, the doubly robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage. CONCLUSIONS: Due to the difficulty of properly specifying parametric models in high-dimensional data, doubly robust estimators with ensemble learning and cross-fitting may be the preferred approach for estimation of the average causal effect in most epidemiologic studies. However, these approaches may require larger sample sizes to avoid finite-sample issues.

Estimating the Effectiveness of Rotavirus Vaccine Schedules.

BACKGROUND: Important questions exist regarding the comparative effectiveness of alternative childhood vaccine schedules; however, optimal approaches to studying this complex issue are unclear. METHODS: We applied methods for studying dynamic treatment regimens to estimate the comparative effectiveness of different rotavirus vaccine (RV) schedules for preventing acute gastroenteritis-related emergency department (ED) visits or hospitalization. We studied the effectiveness of six separate protocols: one- and two-dose monovalent rotavirus vaccine (RV1); one-, two-, and three-dose pentavalent rotavirus vaccine (RV5); and no RV vaccine. We used data on all infants to estimate the counterfactual cumulative risk for each protocol. Infants were censored when vaccine receipt deviated from the protocol. Inverse probability of censoring-weighted estimation addressed potentially informative censoring by protocol deviations. A nonparametric group-based bootstrap procedure provided statistical inference. RESULTS: The method yielded similar 2-year effectiveness estimates for the full-series protocols; weighted risk difference estimates comparing unvaccinated children to those adherent to either full-series (two-dose RV1, three-dose RV5) corresponded to four fewer hospitalizations and 12 fewer ED visits over the 2-year period per 1,000 children. We observed dose-response relationships, such that additional doses further reduced risk of acute gastroenteritis. Under a theoretical intervention to fully vaccinate all children, the 2-year risk differences comparing full to observed adherence were 0.04% (95% CI = 0.03%, 0.05%) for hospitalizations and 0.17% (95% CI = 0.14%, 0.19%) for ED visits. CONCLUSIONS: The proposed approach can generate important evidence about the consequences of delaying or skipping vaccine doses, and the impact of interventions to improve vaccine schedule adherence.

Poverty, Deprivation, and Mortality Risk Among Women With HIV in the United States.

BACKGROUND: Prior studies suggest neighborhood poverty and deprivation are associated with adverse health outcomes including death, but evidence is limited among persons with HIV, particularly women. We estimated changes in mortality risk from improvement in three measures of area-level socioeconomic context among participants of the Women's Interagency HIV Study. METHODS: Starting in October 2013, we linked geocoded residential census block groups to the 2015 Area Deprivation Index (ADI) and two 2012-2016 American Community Survey poverty variables, categorized into national tertiles. We used parametric g-computation to estimate, through March 2018, impacts on mortality of improving each income or poverty measure by one and two tertiles maximum versus no improvement. RESULTS: Of 1596 women with HIV (median age 49), 91 (5.7%) were lost to follow-up and 83 (5.2%) died. Most women (62%) lived in a block group in the tertile with the highest proportions of individuals with income:poverty <1; 13% lived in areas in the tertile with the lowest proportions. Mortality risk differences comparing a one-tertile improvement (for those in the two highest poverty tertiles) in income:poverty <1 versus no improvement increased over time; the risk difference was -2.2% (95% confidence interval [CI] = -3.7, -0.64) at 4 years. Estimates from family income below poverty level (-1.0%; 95% CI = -2.7, 0.62) and ADI (-1.5%; 95% CI = -2.8, -0.21) exposures were similar. CONCLUSIONS: Consistent results from three distinct measures of area-level socioeconomic environment support the hypothesis that interventions to ameliorate neighborhood poverty or deprivation reduce mortality risk for US women with HIV. See video abstract at, http://links.lww.com/EDE/B863.

Fusion designs and estimators for treatment effects.

While randomized trials remain the best evidence for treatment effectiveness, lack of generalizability often remains an important concern. Additionally, when new treatments are compared against existing standards of care, the potentially small benefit of the new treatment may be difficult to detect in a trial without extremely large sample sizes and long follow-up times. Recent advances in "data fusion" provide a framework to combine results across studies that are applicable to a given population of interest and allow treatment comparisons that may not be feasible with traditional study designs. We propose a data fusion-based estimator that can be used to combine information from two studies: (1) a study comparing a new treatment to the standard of care in the local population of interest, and (2) a study comparing the standard of care to placebo in a separate, distal population. We provide conditions under which the parameter of interest can be identified from the two studies described and explore properties of the estimator through simulation. Finally, we apply the estimator to estimate the effect of triple- vs monotherapy for the treatment of HIV using data from two randomized trials. The proposed estimator can account for underlying population structures that induce differences in case mix, adherence, and outcome prevalence between the local and distal populations, and the estimator can also account for potentially informative loss to follow-up. Approaches like those detailed here are increasingly important to speed the approval and adoption of effective new therapies by leveraging multiple sources of information.

When Is a Complete-Case Approach to Missing Data Valid? The Importance of Effect-Measure Modification.

When estimating causal effects, careful handling of missing data is needed to avoid bias. Complete-case analysis is commonly used in epidemiologic analyses. Previous work has shown that covariate-stratified effect estimates from complete-case analysis are unbiased when missingness is independent of the outcome conditional on the exposure and covariates. Here, we assess the bias of complete-case analysis for adjusted marginal effects when confounding is present under various causal structures of missing data. We show that estimation of the marginal risk difference requires an unbiased estimate of the unconditional joint distribution of confounders and any other covariates required for conditional independence of missingness and outcome. The dependence of missing data on these covariates must be considered to obtain a valid estimate of the covariate distribution. If none of these covariates are effect-measure modifiers on the absolute scale, however, the marginal risk difference will equal the stratified risk differences and the complete-case analysis will be unbiased when the stratified effect estimates are unbiased. Estimation of unbiased marginal effects in complete-case analysis therefore requires close consideration of causal structure and effect-measure modification.

The Effects of Hepatitis C Infection and Treatment on All-cause Mortality Among People Living With Human Immunodeficiency Virus.

BACKGROUND: Persons living with human immunodeficiency virus (HIV; PLwH) are commonly co-infected with hepatitis C virus (HCV). Most co-infected individuals can achieve a sustained HCV virologic response after treatment with direct-acting antivirals (DAA). However, the effect of HCV co-infection and DAA treatment on mortality after initiating antiretroviral therapy (ART) is unknown for PLwH. METHODS: We analyzed data from the Women's Interagency HIV Study and the Multicenter AIDS Cohort Study. Participants included those who had prevalent HIV or seroconverted during follow-up; all were antiretroviral-naive and acquired immunodeficiency syndrome (AIDS)-free prior to their first visit after 1 October 1994. The follow-up lasted 10 years or until 30 September 2015. We used parametric g-computation to estimate the effects of HCV infection and DAA treatment on mortality had participants initiated ART at study entry. RESULTS: Of the 3056 eligible participants, 58% were female and 18% had HCV. The estimated 10-year all-cause mortality risk in the scenario in which no PLwH had HCV was 10.4% (95% confidence interval [CI] 6.0-18.0%). The 10-year mortality risk difference for HCV infection was 4.3% (95% CI 0.4-8.9%) and the risk ratio was 1.4 (95% CI 1.0-1.9). The risk difference for DAA treatment was -3.8% (95% CI -9.2-0.9%) and the risk ratio was 0.8 (95% CI 0.6-1.1). CONCLUSIONS: HCV co-infection remains an important risk factor for mortality among PLwH after initiating ART according to modern guidelines, and DAAs are effective at reducing mortality in this population. HCV prevention and treatment interventions should be prioritized to reduce mortality among PLwH.

The Effects of Hepatitis C Treatment Eligibility Criteria on All-cause Mortality Among People With Human Immunodeficiency Virus.

BACKGROUND: The cost of direct-acting antivirals (DAAs) for hepatitis C virus (HCV) prompted many payers to restrict treatment to patients who met non-evidence-based criteria. These restrictions have implications for survival of people with HCV, especially for people with human immunodeficiency virus (HIV)/HCV coinfection who are at high risk for liver disease progression. The goal of this work was to estimate the effects of DAA access policies on 10-year all-cause mortality among people with HIV. METHODS: The study population included 3056 adults with HIV in the Women's Interagency HIV Study and Multicenter AIDS Cohort Study from 1 October 1994 through 30 September 2015. We used the parametric g-formula to estimate 10-year all-cause mortality under DAA access policies that included treating (i) all people with HCV; (ii) only people with suppressed HIV; (iii) only people with severe fibrosis; and (iv) only people with HIV suppression and severe fibrosis. RESULTS: The 10-year risk difference of treating all coinfected persons with DAAs compared with no treatment was -3.7% (95% confidence interval [CI], -9.1% to .6%). Treating only those with suppressed HIV and severe fibrosis yielded a risk difference of -1.1% (95% CI, -2.8% to .6%), with 51% (95% CI, 38%-59%) of coinfected persons receiving DAAs. Treating a random selection of 51% of coinfected persons at baseline decreased the risk by 1.9% (95% CI, -4.7% to .3%). CONCLUSIONS: Restrictive DAA access policies may decrease survival compared to treating similar proportions of people with HIV/HCV coinfection with DAAs at random. These findings suggest that lives could be saved by thoughtfully revising access policies.

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity.

In the absence of strong assumptions (e.g., exchangeability), only bounds for causal effects can be identified. Here we describe bounds for the risk difference for an effect of a binary exposure on a binary outcome in 4 common study settings: observational studies and randomized studies, each with and without simple random selection from the target population. Through these scenarios, we introduce randomizations for selection and treatment, and the widths of the bounds are narrowed from 2 (the width of the range of the risk difference) to 0 (point identification). We then assess the strength of the assumptions of exchangeability for internal and external validity by comparing their contributions to the widths of the bounds in the setting of an observational study without random selection from the target population. We find that when less than two-thirds of the target population is selected into the study, the assumption of exchangeability for external validity of the risk difference is stronger than that for internal validity. The relative strength of these assumptions should be considered when designing, analyzing, and interpreting observational studies and will aid in determining the best methods for estimating the causal effects of interest.

Exploring the Subtleties of Inverse Probability Weighting and Marginal Structural Models.

Since being introduced to epidemiology in 2000, marginal structural models have become a commonly used method for causal inference in a wide range of epidemiologic settings. In this brief report, we aim to explore three subtleties of marginal structural models. First, we distinguish marginal structural models from the inverse probability weighting estimator, and we emphasize that marginal structural models are not only for longitudinal exposures. Second, we explore the meaning of the word "marginal" in "marginal structural model." Finally, we show that the specification of a marginal structural model can have important implications for the interpretation of its parameters. Each of these concepts have important implications for the use and understanding of marginal structural models, and thus providing detailed explanations of them may lead to better practices for the field of epidemiology.