Emulating Target Trials With Real-World Data to Inform Health Technology Assessment: Findings and Lessons From an Application to Emergency Surgery.

OBJECTIVES: International health technology assessment (HTA) agencies recommend that real-world data (RWD) are used in some circumstances to add to the evidence base about the effectiveness and cost-effectiveness of health interventions. The target trial framework applies the design principles of randomized-controlled trials to RWD and can help alleviate inevitable concerns about bias and design flaws with nonrandomized studies. This article aimed to tackle the lack of guidance and exemplar applications on how this methodology can be applied to RWD to inform HTA decision making. METHODS: We use Hospital Episode Statistics data from England on emergency hospital admissions from 2010 to 2019 to evaluate the cost-effectiveness of emergency surgery for 2 acute gastrointestinal conditions. We draw on the case study to describe the main challenges in applying the target trial framework alongside RWD and provide recommendations for how these can be addressed in practice. RESULTS: The 4 main challenges when applying the target trial framework to RWD are (1) defining the study population, (2) defining the treatment strategies, (3) establishing time zero (baseline), and (4) adjusting for unmeasured confounding. The recommendations for how to address these challenges, mainly around the incorporation of expert judgment and use of appropriate methods for handling unmeasured confounding, are illustrated within the case study. CONCLUSIONS: The recommendations outlined in this study could help future studies seeking to inform HTA decision processes. These recommendations can complement checklists for economic evaluations and design tools for estimating treatment effectiveness in nonrandomized studies.

How does a local instrumental variable method perform across settings with instruments of differing strengths? A simulation study and an evaluation of emergency surgery.

Local instrumental variable (LIV) approaches use continuous/multi-valued instrumental variables (IV) to generate consistent estimates of average treatment effects (ATEs) and Conditional Average Treatment Effects (CATEs). There is little evidence on how LIV approaches perform according to the strength of the IV or with different sample sizes. Our simulation study examined the performance of an LIV method, and a two-stage least squares (2SLS) approach across different sample sizes and IV strengths. We considered four 'heterogeneity' scenarios: homogeneity, overt heterogeneity (over measured covariates), essential heterogeneity (unmeasured), and overt and essential heterogeneity combined. In all scenarios, LIV reported estimates with low bias even with the smallest sample size, provided that the instrument was strong. Compared to 2SLS, LIV provided estimates for ATE and CATE with lower levels of bias and Root Mean Squared Error. With smaller sample sizes, both approaches required stronger IVs to ensure low bias. We considered both methods in evaluating emergency surgery (ES) for three acute gastrointestinal conditions. Whereas 2SLS found no differences in the effectiveness of ES according to subgroup, LIV reported that frailer patients had worse outcomes following ES. In settings with continuous IVs of moderate strength, LIV approaches are better suited than 2SLS to estimate policy-relevant treatment effect parameters.

Impact of the first wave of COVID-19 on outcomes following emergency admissions for common acute surgical conditions: analysis of a national database in England.

BACKGROUND: This study assessed the impact of the first COVID-19 wave in England on outcomes for acute appendicitis, gallstone disease, intestinal obstruction, diverticular disease, and abdominal wall hernia. METHODS: Emergency surgical admissions for patients aged 18 years and older to 124 NHS Trust hospitals between January and June in 2019 and 2020 were extracted from Hospital Episode Statistics. The risk of 90-day mortality after admission during weeks 11-19 in 2020 (national lockdown) and 2019 (pre-COVID-19) was estimated using multilevel logistic regression with case-mix adjustment. The primary outcome was all-cause mortality at 90 days. RESULTS: There were 12 231 emergency admissions and 564 deaths within 90 days during weeks 11-19 in 2020, compared with 18 428 admissions and 542 deaths in the same interval in 2019. Overall, 90-day mortality was higher in 2020 versus 2019, with an adjusted OR of 1.95 (95 per cent c.i. 0.78 to 4.89) for appendicitis, 2.66 (1.81 to 3.92) for gallstone disease, 1.99 (1.44 to 2.74) for diverticular disease, 1.70 (1.13 to 2.55) for hernia, and 1.22 (1.01 to 1.47) for intestinal obstruction. After emergency surgery, 90-day mortality was higher in 2020 versus 2019 for gallstone disease (OR 3.37, 1.26 to 9.02), diverticular disease (OR 2.35, 1.16 to 4.73), and hernia (OR 2.34, 1.23 to 4.45). For intestinal obstruction, the corresponding OR was 0.91 (0.59 to 1.41). For admissions not leading to emergency surgery, mortality was higher in 2020 versus 2019 for gallstone disease (OR 2.55, 1.67 to 3.88), diverticular disease (1.90, 1.32 to 2.73), and intestinal obstruction (OR 1.30, 1.06 to 1.60). CONCLUSION: Emergency admission was reduced during the first lockdown in England and this was associated with higher 90-day mortality.

Estimating the Health Effects of Expansions in Health Expenditure in Indonesia: A Dynamic Panel Data Approach.

BACKGROUND: The marginal productivity of a country's healthcare system refers to the health gains produced per unit change in the level of spending. In budget-constrained settings, this metric reflects the opportunity cost, in terms of health gains forgone, of committing additional or existing resources to alternative uses within the healthcare system. It can therefore assist in evidence-based decisions on whether different interventions represent good value for money. OBJECTIVE: The aim of this paper was to estimate the marginal productivity of the Indonesian healthcare system using subnational data, and to use this to inform health opportunity costs in the country. METHODS: We define a dynamic health production function to model the stream of effects of current and prior public health spending decisions on population under-five mortality. To estimate the model, we use data from the 33 Indonesian provinces for the 2004-2012 period. The estimated elasticity is then translated into gains in terms of cost per DALY (disability-adjusted life-year) averted. We use dynamic panel data methods to address potential endogeneity issues in the model. RESULTS: Our base-case estimates suggest that a 1% expansion in the level of health spending reduces under-five mortality by 0.38% (95% CI 0.00-0.76), which translates into a cost of averting one DALY of $235 (2019 US$). CONCLUSION: With Indonesia aiming for universal health coverage, our results support these efforts by highlighting the associated benefits resulting from increases in public health expenditure and have the potential to inform the decision-making process about a suitable locally relevant cost-effectiveness threshold.

Local Instrumental Variable Methods to Address Confounding and Heterogeneity when Using Electronic Health Records: An Application to Emergency Surgery.

BACKGROUND: Electronic health records (EHRs) offer opportunities for comparative effectiveness research to inform decision making. However, to provide useful evidence, these studies must address confounding and treatment effect heterogeneity according to unmeasured prognostic factors. Local instrumental variable (LIV) methods can help studies address these challenges, but have yet to be applied to EHR data. This article critically examines a LIV approach to evaluate the cost-effectiveness of emergency surgery (ES) for common acute conditions from EHRs. METHODS: This article uses hospital episodes statistics (HES) data for emergency hospital admissions with acute appendicitis, diverticular disease, and abdominal wall hernia to 175 acute hospitals in England from 2010 to 2019. For each emergency admission, the instrumental variable for ES receipt was each hospital's ES rate in the year preceding the emergency admission. The LIV approach provided individual-level estimates of the incremental quality-adjusted life-years, costs and net monetary benefit of ES, which were aggregated to the overall population and subpopulations of interest, and contrasted with those from traditional IV and risk-adjustment approaches. RESULTS: The study included 268,144 (appendicitis), 138,869 (diverticular disease), and 106,432 (hernia) patients. The instrument was found to be strong and to minimize covariate imbalance. For diverticular disease, the results differed by method; although the traditional approaches reported that, overall, ES was not cost-effective, the LIV approach reported that ES was cost-effective but with wide statistical uncertainty. For all 3 conditions, the LIV approach found heterogeneity in the cost-effectiveness estimates across population subgroups: in particular, ES was not cost-effective for patients with severe levels of frailty. CONCLUSIONS: EHRs can be combined with LIV methods to provide evidence on the cost-effectiveness of routinely provided interventions, while fully recognizing heterogeneity. HIGHLIGHTS: This article addresses the confounding and heterogeneity that arise when assessing the comparative effectiveness from electronic health records (EHR) data, by applying a local instrumental variable (LIV) approach to evaluate the cost-effectiveness of emergency surgery (ES) versus alternative strategies, for patients with common acute conditions (appendicitis, diverticular disease, and abdominal wall hernia).The instrumental variable, the hospital's tendency to operate, was found to be strongly associated with ES receipt and to minimize imbalances in baseline characteristics between the comparison groups.The LIV approach found that, for each condition, there was heterogeneity in the estimates of cost-effectiveness according to baseline characteristics.The study illustrates how an LIV approach can be applied to EHR data to provide cost-effectiveness estimates that recognize heterogeneity and can be used to inform decision making as well as to generate hypotheses for further research.

Variation in the rates of emergency surgery amongst emergency admissions to hospital for common acute conditions.

BACKGROUND: This paper assesses variation in rates of emergency surgery (ES) amongst emergency admissions to hospital in patients with acute appendicitis, cholelithiasis, diverticular disease, abdominal wall hernia, and intestinal obstruction. METHODS: Records of emergency admissions between 1 April 2010 and 31 December 2019 for the five conditions were extracted from Hospital Episode Statistics for 136 acute National Health Service (NHS) trusts in England. Patients who had ES were identified using Office of Population Censuses and Surveys (OPCS) procedure codes, selected by consensus of a clinical panel. The differences in ES rates according to patient characteristics, and unexplained variations across NHS trusts were estimated by multilevel logistic regression, adjusting for year of emergency admission, age, sex, ethnicity, diagnostic subcategories, index of multiple deprivation, number of co-morbidities, and frailty. RESULTS: The cohort sizes ranged from 107 325 (hernia) to 268 253 (appendicitis) patients, and the proportion of patients who received ES from 11.0 per cent (diverticular disease) to 92.3 per cent (appendicitis). Older patients were generally less likely to receive ES, with adjusted odds ratios (ORs) of ES for those aged 75-79 versus those aged 45-49 years: 0.34 (appendicitis), 0.49 (cholelithiasis), 0.87 (hernia), and 0.91 (intestinal obstruction). Patients with diverticular disease aged 75-79 were more likely to receive ES than those aged 45-49 (OR 1.40). Variation in ES rates across NHS trusts remained after case mix adjustment and was greatest for cholelithiasis (trust median 18 per cent, 10th to 90th centile 7-35 per cent). CONCLUSION: For patients presenting as emergency hospital admissions with common acute conditions, variation in ES rates between NHS trusts remained after adjustment for demographic and clinical characteristics. Age was strongly associated with the likelihood of ES receipt for some procedures.