Integrating decision modeling and machine learning to inform treatment stratification.

There is increasing interest in moving away from "one size fits all (OSFA)" approaches toward stratifying treatment decisions. Understanding how expected effectiveness and cost-effectiveness varies with patient covariates is a key aspect of stratified decision making. Recently proposed machine learning (ML) methods can learn heterogeneity in outcomes without pre-specifying subgroups or functional forms, enabling the construction of decision rules ('policies') that map individual covariates into a treatment decision. However, these methods do not yet integrate ML estimates into a decision modeling framework in order to reflect long-term policy-relevant outcomes and synthesize information from multiple sources. In this paper, we propose a method to integrate ML and decision modeling, when individual patient data is available to estimate treatment-specific survival time. We also propose a novel implementation of policy tree algorithms to define subgroups using decision model output. We demonstrate these methods using the SPRINT (Systolic Blood Pressure Intervention Trial), comparing outcomes for "standard" and "intensive" blood pressure targets. We find that including ML into a decision model can impact the estimate of incremental net health benefit (INHB) for OSFA policies. We also find evidence that stratifying treatment using subgroups defined by a tree-based algorithm can increase the estimates of the INHB.

The effect of conflict-related violence intensity and alcohol use on mental health: The case of Colombia.

We investigated the causal impact of conflict-related violence on individual mental health and its potential pathways in Colombia. Using data from before and after the 2016 peace accord between the Colombian government and the Revolutionary Armed Forces of Colombia (FARC), we adopted a difference-in-differences empirical design combined with instrumental variables estimation. We also used formal mediation analysis to investigate a possible mediating role of alcohol consumption in the relationship between conflict exposure and mental health. Our results did not support the hypothesis that changes in exposure to conflict violence after the peace accord causally led to any changes in individual mental health. We were unable to identify a statistically significant mediating effect of alcohol consumption in the relationship between exposure to conflict violence and mental health.

Estimating the Health Effects of Expansions in Health Expenditure in Indonesia: A Dynamic Panel Data Approach.

BACKGROUND: The marginal productivity of a country's healthcare system refers to the health gains produced per unit change in the level of spending. In budget-constrained settings, this metric reflects the opportunity cost, in terms of health gains forgone, of committing additional or existing resources to alternative uses within the healthcare system. It can therefore assist in evidence-based decisions on whether different interventions represent good value for money. OBJECTIVE: The aim of this paper was to estimate the marginal productivity of the Indonesian healthcare system using subnational data, and to use this to inform health opportunity costs in the country. METHODS: We define a dynamic health production function to model the stream of effects of current and prior public health spending decisions on population under-five mortality. To estimate the model, we use data from the 33 Indonesian provinces for the 2004-2012 period. The estimated elasticity is then translated into gains in terms of cost per DALY (disability-adjusted life-year) averted. We use dynamic panel data methods to address potential endogeneity issues in the model. RESULTS: Our base-case estimates suggest that a 1% expansion in the level of health spending reduces under-five mortality by 0.38% (95% CI 0.00-0.76), which translates into a cost of averting one DALY of $235 (2019 US$). CONCLUSION: With Indonesia aiming for universal health coverage, our results support these efforts by highlighting the associated benefits resulting from increases in public health expenditure and have the potential to inform the decision-making process about a suitable locally relevant cost-effectiveness threshold.

Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force.

Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.

Reflection on modern methods: constructing directed acyclic graphs (DAGs) with domain experts for health services research.

Directed acyclic graphs (DAGs) are a useful tool to represent, in a graphical format, researchers' assumptions about the causal structure among variables while providing a rationale for the choice of confounding variables to adjust for. With origins in the field of probabilistic graphical modelling, DAGs are yet to be widely adopted in applied health research, where causal assumptions are frequently made for the purpose of evaluating health services initiatives. In this context, there is still limited practical guidance on how to construct and use DAGs. Some progress has recently been made in terms of building DAGs based on studies from the literature, but an area that has received less attention is how to create DAGs from information provided by domain experts, an approach of particular importance when there is limited published information about the intervention under study. This approach offers the opportunity for findings to be more robust and relevant to patients, carers and the public, and more likely to inform policy and clinical practice. This article draws lessons from a stakeholder workshop involving patients, health care professionals, researchers, commissioners and representatives from industry, whose objective was to draw DAGs for a complex intervention-online consultation, i.e. written exchange between the patient and health care professional using an online system-in the context of the English National Health Service. We provide some initial, practical guidance to those interested in engaging with domain experts to develop DAGs.

The long-run effects of diagnosis related group payment on hospital lengths of stay in a publicly funded health care system: Evidence from 15 years of micro data.

Diagnosis Related Group (DRG) payment systems are a common means of paying for hospital services. They reward greater activity and therefore potentially encourage more rapid treatment. This paper uses 15 years of administrative data to examine the impact of a DRG system introduced in England on hospital lengths of stay. We utilize different econometric models, exploiting within and cross jurisdiction variation, to identify policy effects, finding that the reduction of lengths of stay was greater than previously estimated and grew over time. This constitutes new and important evidence of the ability of financing reform to generate substantial and persistent change in healthcare delivery.

Propensity score methods for comparative-effectiveness analysis: A case study of direct oral anticoagulants in the atrial fibrillation population.

OBJECTIVE: To explore methodological challenges when using real-world evidence (RWE) to estimate comparative-effectiveness in the context of Health Technology Assessment of direct oral anticoagulants (DOACs) in Scotland. METHODS: We used linkage data from the Prescribing Information System (PIS), Scottish Morbidity Records (SMR) and mortality records for newly anticoagulated patients to explore methodological challenges in the use of Propensity score (PS) matching, Inverse Probability Weighting (IPW) and covariate adjustment with PS. Model performance was assessed by standardised difference. Clinical outcomes (stroke and major bleeding) and mortality were compared for all DOACs (including apixaban, dabigatran and rivaroxaban) versus warfarin. Patients were followed for 2 years from first oral anticoagulant prescription to first clinical event or death. Censoring was applied for treatment switching or discontinuation. RESULTS: Overall, a good balance of patients' covariates was obtained with every PS model tested. IPW was found to be the best performing method in assessing covariate balance when applied to subgroups with relatively large sample sizes (combined-DOACs versus warfarin). With the IPTW-IPCW approach, the treatment effect tends to be larger, but still in line with the treatment effect estimated using other PS methods. Covariate adjustment with PS in the outcome model performed well when applied to subgroups with smaller sample sizes (dabigatran versus warfarin), as this method does not require further reduction of sample size, and trimming or truncation of extreme weights. CONCLUSION: The choice of adequate PS methods may vary according to the characteristics of the data. If assumptions of unobserved confounding hold, multiple approaches should be identified and tested. PS based methods can be implemented using routinely collected linked data, thus supporting Health Technology decision-making.

The impact of civil conflict on child health: Evidence from Colombia.

Internal armed conflicts have become more common and more physically destructive since the mid-20th century, with devastating consequences for health and development in low- and middle-income countries. This paper investigates the causal impacts of the long-term internal conflict on child health in Colombia, following an identification strategy based on the temporal and geographic variation of conflict intensity. We estimate the effect of different levels of conflict intensity on height-for-age (HAZ), weight-for-age (WAZ), and weight-for-height z-scores among children under five years old, and explore the underlying potential mechanisms, through maternal health behavior and health care utilization. We find a harmful effect of exposure to conflict violence in utero and in early childhood for HAZ and WAZ, in the full sample and even more strongly in the rural sample, yet these estimates are smaller than those found for shorter term conflicts. The underlying pathways appear to operate around the time of the pregnancy and birth (in the form of maternal alcohol use, use of antenatal care and skilled birth attendance), rather than during the post-birth period (via breastfeeding or vaccination), and the impacts accumulate over the childhood. The most adverse impacts of conflict violence on child health and utilization of maternal healthcare were observed in municipalities which suffered from intermittent presence of armed groups.

Inequalities in catastrophic health expenditures in conflict-affected areas and the Colombian peace agreement: an oaxaca-blinder change decomposition analysis.

BACKGROUND: The present study analyzes inequalities in catastrophic health expenditures in conflict-affected regions of Meta, Colombia and socioeconomic factors contributing to the existence and changes in catastrophic expenditures before and after the sign of Colombian Peace Agreement with FARC-EP guerilla group in 2016. METHODS: The study uses the results of the survey Conflicto, Paz y Salud (CONPAS) conducted in 1309 households of Meta, Colombia, a territory historically impacted by armed conflict, for the years 2014 and 2018. We define catastrophic expenditures as health expenditures above 20% of the capacity to pay of a household. We disaggregate the changes in inequalities in catastrophic expenditures through the Oaxaca-Blinder change decomposition method. RESULTS: The incidence of catastrophic expenditures slightly increased between 2014 to 2018, from 29.3 to 30.7%. Inequalities in catastrophic expenditures, measured through concentration indexes (CI), also increased from 2014 (CI: -0.152) to 2018 (CI: -0.232). Results show that differences in catastrophic expenditures between socioeconomic groups are mostly attributed to an increased influence of specific sociodemographic variables such as living in rural zones, being a middle-aged person, living in conflict-affected territories, or presenting any type of mental and physical disability. CONCLUSIONS: Conflict-deescalation and the peace agreement may have facilitated lower-income groups to have access to health services, especially in territories highly impacted by conflict. This, consequently, may have led to higher levels of out-of-pocket expenditures and, therefore, to higher chances of experiencing catastrophic expenditures for lower-income groups in comparison to higher-income groups. Therefore, results indicate the importance of designing policies that guarantee access to health services for people in conflict -affected regions but also, that minimize health care inequalities in out-of-pocket payments that may arouse between people at different socioeconomic groups.

The effect of distance on maternal institutional delivery choice: Evidence from Malawi.

In many low- and middle-income countries, geographical accessibility continues to be a barrier to health care utilization. In this paper, we aim to better understand the full relationship between distance to providers and utilization of maternal delivery services. We address three methodological challenges: non-linear effects between distance and utilization; unobserved heterogeneity through non-random distance "assignment"; and heterogeneous effects of distance. Linking Malawi Demographic Health Survey household data to Service Provision Assessment facility data, we consider distance as a continuous treatment variable, estimating a Dose-Response Function based on generalized propensity scores, allowing exploration of non-linearities in the effect of an increment in distance at different distance exposures. Using an instrumental variables approach, we examine the potential for unobserved differences between women residing at different distances to health facilities. Our results suggest distance significantly reduces the probability of having a facility delivery, with evidence of non-linearities in the effect. The negative relationship is shown to be particularly strong for women with poor health knowledge and lower socio-economic status, with important implications for equity. We also find evidence of potential unobserved confounding, suggesting that methods that ignore such confounding may underestimate the effect of distance on the utilization of health services.

Facility standards and the quality of public sector primary care: Evidence from South Africa's "Ideal Clinics" program.

Primary healthcare systems are central to achieving universal healthcare coverage. However, in many low- and middle-income country settings, primary care quality is challenged by inadequate facility infrastructure and equipment, limited human resources, and poor provider process. We study the effects of a recent large-scale quality improvement policy in South Africa, the Ideal Clinics Realization and Maintenance Program (ICRMP). The ICRMP introduced a set of standards for facilities and a quality improvement process involving manuals, district-based support, and external assessment. Exploiting differential prioritization of facilities for the ICRMP's quality improvement process, we apply differences-in-differences methods to identify the effects of the program's efforts on standards scores and primary care quality indicators over the first 12 months of implementation. We find large and statistically significant increases in standards scores, but mixed effects on care outcomes-a small magnitude improvement in early antenatal care usage, null effects on childhood immunization and cervical cancer screening, and small negative effect of human immunodeficiency virus (HIV) care. While the ICRMP process has led to significant improvements in facilities' satisfaction of the program's standards, we were unable to detect meaningful change in care quality indicators.

A light of hope? Inequalities in mental health before and after the peace agreement in Colombia: a decomposition analysis.

BACKGROUND: The present study seeks to evaluate the change in mental health inequalities in the department of Meta after the signing of Colombia's Peace Agreement in 2016 with the FARC guerrilla group. Using a validated survey instrument composed of 20 questions ('SRQ-20'), we measure changes in mental health inequalities from 2014, before the signing of the agreement, to 2018, after the signing. We then decompose the changes in inequalities to establish which socioeconomic factors explain differences in mental health inequalities over time. METHODS: Our study uses information from the Conflicto, Salud y Paz (CONPAS) survey conducted in the department of Meta, Colombia, in 1309 households in 2018, with retrospective information for 2014. To measure inequalities, we calculate the concentration indices for both years. Through the Oaxaca change decomposition method, we disaggregate changes in mental health inequalities into its underlying factors. This method allows us to explain the relationship between changes in mental health inequalities and changes in inequalities in several sociodemographic factors. It also identifies the extent to which these factors help explain the changes in mental health inequalities. RESULTS: Mental health inequalities in Meta were reduced almost by half from 2014 to 2018. In 2018, the population at the lower and middle socioeconomic levels had fewer chances of experiencing mental health disorders in comparison to 2014. The reduction in mental health differences is mostly attributed to reductions in the influence of certain sociodemographic variables, such as residence in rural zones and conflict-affected territories, working in the informal sector, or experiencing internal displacement. However, even though mental health inequalities have diminished, overall mental health outcomes have worsened in these years. CONCLUSIONS: The reduction in the contribution of conflict-related variables for explaining mental health inequalities could mean that the negative consequences of conflict on mental health have started to diminish in the short run after the peace agreement. Nevertheless, conflict and the presence of other socioeconomic inequalities still contribute to persistent adverse mental health outcomes in the overall population. Thus, public policy should be oriented towards improving mental health care services in these territories, given the post-accord context.

Exploiting nonsystematic covariate monitoring to broaden the scope of evidence about the causal effects of adaptive treatment strategies.

In studies based on electronic health records (EHR), the frequency of covariate monitoring can vary by covariate type, across patients, and over time, which can limit the generalizability of inferences about the effects of adaptive treatment strategies. In addition, monitoring is a health intervention in itself with costs and benefits, and stakeholders may be interested in the effect of monitoring when adopting adaptive treatment strategies. This paper demonstrates how to exploit nonsystematic covariate monitoring in EHR-based studies to both improve the generalizability of causal inferences and to evaluate the health impact of monitoring when evaluating adaptive treatment strategies. Using a real world, EHR-based, comparative effectiveness research (CER) study of patients with type II diabetes mellitus, we illustrate how the evaluation of joint dynamic treatment and static monitoring interventions can improve CER evidence and describe two alternate estimation approaches based on inverse probability weighting (IPW). First, we demonstrate the poor performance of the standard estimator of the effects of joint treatment-monitoring interventions, due to a large decrease in data support and concerns over finite-sample bias from near-violations of the positivity assumption (PA) for the monitoring process. Second, we detail an alternate IPW estimator using a no direct effect assumption. We demonstrate that this estimator can improve efficiency but at the potential cost of increase in bias from violations of the PA for the treatment process.

A comparison of methods for health policy evaluation with controlled pre-post designs.

OBJECTIVE: To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re-evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010. DATA SOURCES: Simulations and Hospital Episode Statistics. STUDY DESIGN: Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference-in-differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study. PRINCIPAL FINDINGS: Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: -1.4 to 0.4) percentage point reduction in 30-day mortality. CONCLUSIONS: The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective.

What next after GDP-based cost-effectiveness thresholds?

Public payers around the world are increasingly using cost-effectiveness thresholds (CETs) to assess the value-for-money of an intervention and make coverage decisions. However, there is still much confusion about the meaning and uses of the CET, how it should be calculated, and what constitutes an adequate evidence base for its formulation. One widely referenced and used threshold in the last decade has been the 1-3 GDP per capita, which is often attributed to the Commission on Macroeconomics and  WHO guidelines on Choosing Interventions that are Cost Effective (WHO-CHOICE). For many reasons, however, this threshold has been widely criticised; which has led experts across the world, including the WHO, to discourage its use. This has left a vacuum for policy-makers and technical staff at a time when countries are wanting to move towards Universal Health Coverage . This article seeks to address this gap by offering five practical options for decision-makers in low- and middle-income countries that can be used instead of the 1-3 GDP rule, to combine existing evidence with fair decision-rules or develop locally relevant CETs. It builds on existing literature as well as an engagement with a group of experts and decision-makers working in low, middle and high income countries.

Paying for efficiency: Incentivising same-day discharges in the English NHS.

We study a pay-for-efficiency scheme that encourages hospitals to admit and discharge patients on the same calendar day when clinically appropriate. Since 2010, hospitals in the English NHS are incentivised by a higher price for patients treated as same-day discharge than for overnight stays, despite the former being less costly. We analyse administrative data for patients treated during 2006-2014 for 191 conditions for which same-day discharge is clinically appropriate - of which 32 are incentivised. Using difference-in-difference and synthetic control methods, we find that the policy had generally a positive impact with a statistically significant effect in 14 out of the 32 conditions. The median elasticity is 0.24 for planned and 0.01 for emergency conditions. Condition-specific design features explain some, but not all, of the differential responses.

A framework for conducting economic evaluations alongside natural experiments.

Internationally, policy makers are increasingly focussed on reducing the detrimental consequences and rising costs associated with unhealthy diets, inactivity, smoking, alcohol and other risk factors on the health of their populations. This has led to an increase in the demand for evidence-based, cost-effective Population Health Interventions (PHIs) to reverse this trend. Given that research designs such as randomised controlled trials (RCTs) are often not suited to the evaluation of PHIs, Natural Experiments (NEs) are now frequently being used as a design to evaluate such complex, preventive PHIs. However, current guidance for economic evaluation focusses on RCT designs and therefore does not address the specific challenges of NE designs. Using such guidance can lead to sub-optimal design, data collection and analysis for NEs, leading to bias in the estimated effectiveness and cost-effectiveness of the PHI. As a consequence, there is a growing recognition of the need to identify a robust methodological framework for the design and conducting of economic evaluations alongside such NEs. This paper outlines the challenges inherent to the design and conduct of economic evaluations of PHIs alongside NEs, providing a comprehensive framework and outlining a research agenda in this area.

Estimating the Comparative Effectiveness of Feeding Interventions in the Pediatric Intensive Care Unit: A Demonstration of Longitudinal Targeted Maximum Likelihood Estimation.

Longitudinal data sources offer new opportunities for the evaluation of sequential interventions. To adjust for time-dependent confounding in these settings, longitudinal targeted maximum likelihood based estimation (TMLE), a doubly robust method that can be coupled with machine learning, has been proposed. This paper provides a tutorial in applying longitudinal TMLE, in contrast to inverse probability of treatment weighting and g-computation based on iterative conditional expectations. We apply these methods to estimate the causal effect of nutritional interventions on clinical outcomes among critically ill children in a United Kingdom study (Control of Hyperglycemia in Paediatric Intensive Care, 2008-2011). We estimate the probability of a child's being discharged alive from the pediatric intensive care unit by a given day, under a range of static and dynamic feeding regimes. We find that before adjustment, patients who follow the static regime "never feed" are discharged by the end of the fifth day with a probability of 0.88 (95% confidence interval: 0.87, 0.90), while for the patients who follow the regime "feed from day 3," the probability of discharge is 0.64 (95% confidence interval: 0.62, 0.66). After adjustment for time-dependent confounding, most of this difference disappears, and the statistical methods produce similar results. TMLE offers a flexible estimation approach; hence, we provide practical guidance on implementation to encourage its wider use.