Objective causal predictions from observational data.

Many recent articles in public health risk assessment have stated that causal conclusions drawn from observational data must rely on inherently untestable assumptions. They claim that such assumptions ultimately can only be evaluated by informed human judgments. We call this the subjective approach to causal interpretation of observational results. Its theoretical and conceptual foundation is a potential outcomes model of causation in which counterfactual outcomes cannot be observed. It risks depriving decision-makers and the public of the key benefits of traditional objective science, which invites scrutiny and independent verification through testable causal models and interventional hypotheses. We introduce an alternative objective approach to causal analysis of exposure-response relationships in observational data. This is designed to be more objective in the specific sense that it is independently verifiable (or refutable) and data-driven, requiring no inherently untestable assumptions. This approach uses empirically testable interventional causal models, specifically causal Bayesian networks (CBNs), instead of untestable potential outcomes models. It enables empirical validation of causal claims through Invariant Causal Prediction (ICP) tests across multiple studies. We explain how to use CBNs and individual conditional expectation (ICE) plots to quantify the effects on health risks of changing exposures while taking into account realistic complexities such as imperfectly controlled confounding, missing data, and measurement error. By ensuring that all causal assumptions are explicit and empirically testable, our framework may help to improve the reliability and transparency of causal inferences in health risk assessments.

Smartphone Screen Time Characteristics in People With Suicidal Thoughts: Retrospective Observational Data Analysis Study.

Background: Smartphone-based monitoring in natural settings provides opportunities to monitor mental health behaviors, including suicidal thoughts and behaviors. To date, most suicidal thoughts and behaviors research using smartphones has primarily relied on collecting so-called "active" data, requiring participants to engage by completing surveys. Data collected passively from smartphone sensors and logs may offer an objectively measured representation of an individual's behavior, including smartphone screen time. Objective: This study aims to present methods for identifying screen-on bouts and deriving screen time characteristics from passively collected smartphone state logs and to estimate daily smartphone screen time in people with suicidal thinking, providing a more reliable alternative to traditional self-report. Methods: Participants (N=126; median age 22, IQR 16-33 years) installed the Beiwe app (Harvard University) on their smartphones, which passively collected phone state logs for up to 6 months after discharge from an inpatient psychiatric unit (adolescents) or emergency department visit (adults). We derived daily screen time measures from these logs, including screen-on time, screen-on bout duration, screen-off bout duration, and screen-on bout count. We estimated the mean of these measures across age subgroups (adults and adolescents), phone operating systems (Android and iOS), and monitoring stages after the discharge (first 4 weeks vs subsequent weeks). We evaluated the sensitivity of daily screen time measures to changes in the parameters of the screen-on bout identification method. Additionally, we estimated the impact of a daylight time change on minute-level screen time using function-on-scalar generalized linear mixed-effects regression. Results: The median monitoring period was 169 (IQR 42-169) days. For adolescents and adults, mean daily screen-on time was 254.6 (95% CI 231.4-277.7) and 271.0 (95% CI 252.2-289.8) minutes, mean daily screen-on bout duration was 4.233 (95% CI 3.565-4.902) and 4.998 (95% CI 4.455-5.541) minutes, mean daily screen-off bout duration was 25.90 (95% CI 20.09-31.71) and 26.90 (95% CI 22.18-31.66) minutes, and mean daily screen-on bout count (natural logarithm transformed) was 4.192 (95% CI 4.041-4.343) and 4.090 (95% CI 3.968-4.213), respectively; there were no significant differences between smartphone operating systems (all P values were >.05). The daily measures were not significantly different for the first 4 weeks compared to the fifth week onward (all P values were >.05), except average screen-on bout in adults (P value = .018). Our sensitivity analysis indicated that in the screen-on bout identification method, the cap on an individual screen-on bout duration has a substantial effect on the resulting daily screen time measures. We observed time windows with a statistically significant effect of daylight time change on screen-on time (based on 95% joint confidence intervals bands), plausibly attributable to sleep time adjustments related to clock changes. Conclusions: Passively collected phone logs offer an alternative to self-report measures for studying smartphone screen time characteristics in people with suicidal thinking. Our work demonstrates the feasibility of this approach, opening doors for further research on the associations between daily screen time, mental health, and other factors.

Measuring the impact of influenza vaccination in the Netherlands using retrospective observational primary care, hospitalisation and mortality data.

We aimed to estimate the impact of influenza vaccination in the Netherlands using general practitioner medical records for 2011-2020. We found that vaccinees had higher consultation rates for influenza-like-illness, acute respiratory infections, and pneumonia, as well as antibiotic use, hospitalisations, and several control diagnoses (i.e. illnesses for which there was no a priori expectation that influenza vaccination would play a protective effect). We found similar rates for respiratory mortality and lower all-cause mortality in the vaccinees versus non-vaccinees, mainly driven by the 75+ age group. These results expand, but are fairly consistent with those of previous investigations, and highlight the difficulty of using registry data to assess the impact of vaccination, because of underlying differences between vaccinees and non-vaccinees. Whether these biases also play a role for hospitalisations and mortality remains unclear. Our findings support the implementation of randomized studies to assess the impact of influenza vaccination.

Trends in the Completeness and Validity of Sources of Death Data Against the National Death Index From 2010 to 2018.

PURPOSE: We investigated time trends in validation performance characteristics for six sources of death data available within the Healthcare Integrated Research Database (HIRD) over 8 years. METHODS: We conducted a secondary analysis of a cohort of advanced cancer patients with linked National Death Index (NDI) data identified in the HIRD between 2010 and 2018. We calculated sensitivity, specificity, positive predictive value, and negative predictive value for six sources of death status data and an algorithm combining data from available sources using NDI data as the reference standard. Measures were calculated for each year of the study including all members in the cohort for at least 1 day in that year. RESULTS: We identified 27 396 deaths from any source among 40 692 cohort members. Between 2010 and 2018, the sensitivity of the Death Master File (DMF) decreased from 0.77 (95% CI = 0.76, 0.79) to 0.12 (95% CI = 0.11, 0.14). In contrast, the sensitivity of online obituary data increased from 0.43 (95% CI = 0.41, 0.45) in 2012 to 0.71 (95% CI = 0.68, 0.73) in 2018. The sensitivity of the composite algorithm remained above 0.83 throughout the study period. PPV was observed to be high from 2010 to 2016 and decrease thereafter for all sources. Specificity and NPV remained at high levels throughout the study. CONCLUSIONS: We observed that the sensitivity of mortality data sources compared with the NDI could change substantially between 2010 and 2018. Other validation characteristics were less variable. Combining multiple sources of mortality data may be necessary to achieve adequate performance particularly for multiyear studies.

Real-World Data in Greece: Mapping the Papageorgiou General Hospital Data to the OMOP Common Data Model.

This paper showcases the results of the Extract-Transform-Load process mapping the Electronic Health Record of Papageorgiou General Hospital in Thessaloniki, Greece, to the Observational Medical Outcomes Partnership Common Data Model. We describe the staged process utilized to account for the intricate structure of the database, along with some general findings from the mapping. Finally, we investigate potential directions for future research.

Machine Learning Methods to Estimate Individualized Treatment Effects for Use in Health Technology Assessment.

BACKGROUND: Recent developments in causal inference and machine learning (ML) allow for the estimation of individualized treatment effects (ITEs), which reveal whether treatment effectiveness varies according to patients' observed covariates. ITEs can be used to stratify health policy decisions according to individual characteristics and potentially achieve greater population health. Little is known about the appropriateness of available ML methods for use in health technology assessment. METHODS: In this scoping review, we evaluate ML methods available for estimating ITEs, aiming to help practitioners assess their suitability in health technology assessment. We present a taxonomy of ML approaches, categorized by key challenges in health technology assessment using observational data, including handling time-varying confounding and time-to event data and quantifying uncertainty. RESULTS: We found a wide range of algorithms for simpler settings with baseline confounding and continuous or binary outcomes. Not many ML algorithms can handle time-varying or unobserved confounding, and at the time of writing, no ML algorithm was capable of estimating ITEs for time-to-event outcomes while accounting for time-varying confounding. Many of the ML algorithms that estimate ITEs in longitudinal settings do not formally quantify uncertainty around the point estimates. LIMITATIONS: This scoping review may not cover all relevant ML methods and algorithms as they are continuously evolving. CONCLUSIONS: Existing ML methods available for ITE estimation are limited in handling important challenges posed by observational data when used for cost-effectiveness analysis, such as time-to-event outcomes, time-varying and hidden confounding, or the need to estimate sampling uncertainty around the estimates. IMPLICATIONS: ML methods are promising but need further development before they can be used to estimate ITEs for health technology assessments. HIGHLIGHTS: Estimating individualized treatment effects (ITEs) using observational data and machine learning (ML) can support personalized treatment advice and help deliver more customized information on the effectiveness and cost-effectiveness of health technologies.ML methods for ITE estimation are mostly designed for handling confounding at baseline but not time-varying or unobserved confounding. The few models that account for time-varying confounding are designed for continuous or binary outcomes, not time-to-event outcomes.Not all ML methods for estimating ITEs can quantify the uncertainty of their predictions.Future work on developing ML that addresses the concerns summarized in this review is needed before these methods can be widely used in clinical and health technology assessment-like decision making.

Development and validation of a patient-level model to predict dementia across a network of observational databases.

BACKGROUND: A prediction model can be a useful tool to quantify the risk of a patient developing dementia in the next years and take risk-factor-targeted intervention. Numerous dementia prediction models have been developed, but few have been externally validated, likely limiting their clinical uptake. In our previous work, we had limited success in externally validating some of these existing models due to inadequate reporting. As a result, we are compelled to develop and externally validate novel models to predict dementia in the general population across a network of observational databases. We assess regularization methods to obtain parsimonious models that are of lower complexity and easier to implement. METHODS: Logistic regression models were developed across a network of five observational databases with electronic health records (EHRs) and claims data to predict 5-year dementia risk in persons aged 55-84. The regularization methods L1 and Broken Adaptive Ridge (BAR) as well as three candidate predictor sets to optimize prediction performance were assessed. The predictor sets include a baseline set using only age and sex, a full set including all available candidate predictors, and a phenotype set which includes a limited number of clinically relevant predictors. RESULTS: BAR can be used for variable selection, outperforming L1 when a parsimonious model is desired. Adding candidate predictors for disease diagnosis and drug exposure generally improves the performance of baseline models using only age and sex. While a model trained on German EHR data saw an increase in AUROC from 0.74 to 0.83 with additional predictors, a model trained on US EHR data showed only minimal improvement from 0.79 to 0.81 AUROC. Nevertheless, the latter model developed using BAR regularization on the clinically relevant predictor set was ultimately chosen as best performing model as it demonstrated more consistent external validation performance and improved calibration. CONCLUSIONS: We developed and externally validated patient-level models to predict dementia. Our results show that although dementia prediction is highly driven by demographic age, adding predictors based on condition diagnoses and drug exposures further improves prediction performance. BAR regularization outperforms L1 regularization to yield the most parsimonious yet still well-performing prediction model for dementia.

Assessing Etiologic Heterogeneity for Multinomial Outcome with Two-Phase Outcome-Dependent Sampling Design.

Etiologic heterogeneity occurs when distinct sets of events or exposures give rise to different subtypes of disease. Inference about subtype-specific exposure effects from two-phase outcome-dependent sampling data requires adjustment for both confounding and the sampling design. Common approaches to inference for these effects do not necessarily appropriately adjust for these sources of bias, or allow for formal comparisons of effects across different subtypes. Herein, using inverse probability weighting (IPW) to fit a multinomial model is shown to yield valid inference with this sampling design for subtype-specific exposure effects and contrasts thereof. The IPW approach is compared to common regression-based methods for assessing exposure effect heterogeneity using simulations. The methods are applied to estimate subtype-specific effects of various exposures on breast cancer risk in the Carolina Breast Cancer Study.

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances.

PURPOSE: Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology. METHODS: We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology. RESULTS: Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping. CONCLUSIONS: Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

Detecting responsible nodes in differential Bayesian networks.

To study the roles that different nodes play in differentiating Bayesian networks under two states, such as control versus disease, we formulate two node-specific scores to facilitate such assessment. The first score is motivated by the prediction invariance property of a causal model. The second score results from modifying an existing score constructed for differential analysis of undirected networks. We develop strategies based on these scores to identify nodes responsible for topological differences between two Bayesian networks. Synthetic data and real-life data from designed experiments are used to demonstrate the efficacy of the proposed methods in detecting responsible nodes.

Application of causal inference methods in individual-participant data meta-analyses in medicine: addressing data handling and reporting gaps with new proposed reporting guidelines.

Observational data provide invaluable real-world information in medicine, but certain methodological considerations are required to derive causal estimates. In this systematic review, we evaluated the methodology and reporting quality of individual-level patient data meta-analyses (IPD-MAs) conducted with non-randomized exposures, published in 2009, 2014, and 2019 that sought to estimate a causal relationship in medicine. We screened over 16,000 titles and abstracts, reviewed 45 full-text articles out of the 167 deemed potentially eligible, and included 29 into the analysis. Unfortunately, we found that causal methodologies were rarely implemented, and reporting was generally poor across studies. Specifically, only three of the 29 articles used quasi-experimental methods, and no study used G-methods to adjust for time-varying confounding. To address these issues, we propose stronger collaborations between physicians and methodologists to ensure that causal methodologies are properly implemented in IPD-MAs. In addition, we put forward a suggested checklist of reporting guidelines for IPD-MAs that utilize causal methods. This checklist could improve reporting thereby potentially enhancing the quality and trustworthiness of IPD-MAs, which can be considered one of the most valuable sources of evidence for health policy.

Unfulfilled and method-specific contraceptive preferences among reproductive-aged contraceptive users in Arizona, Iowa, New Jersey, and Wisconsin.

OBJECTIVE: To identify characteristics associated with unfulfilled contraceptive preferences, document reasons for these unfulfilled preferences, and examine how these unfulfilled preferences vary across specific method users. DATA SOURCES AND STUDY SETTING: We draw on secondary baseline data from 4660 reproductive-aged contraceptive users in the Arizona, Iowa, New Jersey, and Wisconsin Surveys of Women (SoWs), state-representative surveys fielded between October 2018 and August 2020 across the four states. STUDY DESIGN: This is an observational cross-sectional study, which examined associations between individuals' reproductive health-related experiences and contraceptive preferences, adjusting for sociodemographic characteristics. Our primary outcome of interest is having an unfulfilled contraceptive preference, and a key independent variable is experience of high-quality contraceptive care. We also examine specific contraceptive method preferences according to current method used, as well as reasons for not using a preferred method. DATA COLLECTION/EXTRACTION METHODS: Survey respondents who indicated use of any contraceptive method within the last 3 months prior to the survey were eligible for inclusion in this analysis. PRINCIPAL FINDINGS: Overall, 23% reported preferring to use a method other than their current method, ranging from 17% in Iowa to 26% in New Jersey. Young age (18-24), using methods not requiring provider involvement, and not receiving quality contraceptive care were key attributes associated with unfulfilled contraceptive preferences. Those using emergency contraception and fertility awareness-based methods had some of the highest levels of unfulfilled contraceptive preferences, while pills, condoms, partner vasectomy, and IUDs were identified as the most preferred methods. Reasons for not using preferred contraceptive methods fell largely into one of two buckets: system-level or interpersonal/individual reasons. CONCLUSIONS: Our findings highlight that avenues for decreasing the gap between contraceptive methods used and those preferred to be used may lie with healthcare providers and funding streams that support the delivery of contraceptive care.

Using joint models for longitudinal and time-to-event data to investigate the causal effect of salvage therapy after prostatectomy.

Prostate cancer patients who undergo prostatectomy are closely monitored for recurrence and metastasis using routine prostate-specific antigen measurements. When prostate-specific antigen levels rise, salvage therapies are recommended in order to decrease the risk of metastasis. However, due to the side effects of these therapies and to avoid over-treatment, it is important to understand which patients and when to initiate these salvage therapies. In this work, we use the University of Michigan Prostatectomy Registry Data to tackle this question. Due to the observational nature of this data, we face the challenge that prostate-specific antigen is simultaneously a time-varying confounder and an intermediate variable for salvage therapy. We define different causal salvage therapy effects defined conditionally on different specifications of the longitudinal prostate-specific antigen history. We then illustrate how these effects can be estimated using the framework of joint models for longitudinal and time-to-event data. All proposed methodology is implemented in the freely-available R package JMbayes2.

Association of hospital-based substance use supports on emergency department revisits: a retrospective cohort study in Sudbury, Canada from 2018 to 2022.

BACKGROUND: This study compares emergency department (ED) revisits for patients receiving hospital-based substance-use support compared to those who did not receive specialized addiction services at Health Sciences North in Sudbury, Ontario, Canada. METHODS: The study is a retrospective observational study using administrative data from all patients presenting with substance use disorder (SUD) at Health Sciences North from January 1, 2018, and August 31, 2022 with ICD-10 codes from the Discharge Abstract Database (DAD) and the National Ambulatory Care Database (NACRS). There were two interventions under study: addiction medicine consult services (AMCS group), and specialized addiction medicine unit (AMU group). The AMCS is a consult service offered for patients in the ED and those who are admitted to the hospital. The AMU is a specialized inpatient medical unit designed to offer addiction support to stabilize patients that operates under a harm-reduction philosophy. The primary outcome was all cause ED revisit within 30 days of the index ED or hospital visit. The secondary outcome was all observed ED revisits in the study period. Kaplan-Meier curves were used to measure the proportion of 30-day revisits by exposure group. Odds ratios and Hazard Ratios were calculated using logistic regression models with random effects and Cox-proportional hazard model respectively. RESULTS: A total of 5,367 patients with 10,871 ED index visits, and 2,127 revisits between 2018 and 2022 are included in the study. 45% (2,340/5,367) of patient were not admitted to hospital. 30-day revisits were less likely among the intervention group: Addiction Medicine Consult Services (AMCS) in the ED significantly reduced the odds of revisits (OR 0.53, 95% CI 0.39-0.71, p < 0.01) and first revisits (OR 0.42, 95% CI 0.33-0.53, p < 0.01). The AMU group was associated with lower revisits odds (OR 0.80, 95% CI 0.66-0.98, p = 0.03). For every additional year of age, the odds of revisits slightly decreased (OR 0.99, 95% CI 0.98-1.00, p = 0.01) and males were found to have an increased risk compared to females (OR 1.50, 95% CI 1.35-1.67, p < 0.01). INTERPRETATION: We observe statistically significant differences in ED revisits for patients receiving hospital-based substance-use support at Health Sciences North. Hospital-based substance-use supports could be applied to other hospitals to reduce 30-day revisits.

Use and reporting of inverse-probability-of-treatment weighting for multicategory treatments in medical research: a systematic review.

OBJECTIVES: Causal inference methods for observational data represent an alternative to randomised controlled trials when they are not feasible or when real-world evidence is sought. Inverse-probability-of-treatment weighting (IPTW) is one of the most popular approaches to account for confounding in observational studies. In medical research, IPTW is mainly applied to estimate the causal effect of a binary treatment, even when the treatment has in fact multiple categories, despite the availability of IPTW estimators for multiple treatment categories. This raises questions about the appropriateness of the use of IPTW in this context. Therefore, we conducted a systematic review of medical publications reporting the use of IPTW in the presence of a multi-category treatment. Our objectives were to investigate the frequency of use and the implementation of these methods in practice, and to assess the quality of their reporting. STUDY DESIGN AND SETTING: Using Pubmed, Embase and Web of Science, we screened 5660 articles and retained 106 articles in the final analysis that were from 17 different medical areas. This systematic review is registered on PROSPERO (CRD42022352669). RESULTS: The number of treatment groups varied between 3 and 9, with a large majority of articles (90 [84.9%]) including 3 or 4 groups. The most commonly used method for estimating the weights was multinomial regression (51 [48.1%]) and generalized boosted models (48 [45.3%]). The covariates of the weight model were reported in 91 articles (85.9 %). Twenty-six articles (24.5 %) did not discuss the balance of covariates after weighting, and only 16 articles (15.1 %) referred to the assumptions needed to obtain correct inferences. CONCLUSION: The results of this systematic review illustrate that medical publications scarcely use IPTW methods for more than two treatment categories. Among the publications that did, the quality of reporting was suboptimal, in particular in regard to the assumptions and model building. IPTW for multi-category treatments could be applied more broadly in medical research, and the application of the proposed guidelines in this context will help researchers to report their results and to ensure reproducibility of their research.

Markov modeling for cost-effectiveness using federated health data network.

OBJECTIVE: To introduce 2 R-packages that facilitate conducting health economics research on OMOP-based data networks, aiming to standardize and improve the reproducibility, transparency, and transferability of health economic models. MATERIALS AND METHODS: We developed the software tools and demonstrated their utility by replicating a UK-based heart failure data analysis across 5 different international databases from Estonia, Spain, Serbia, and the United States. RESULTS: We examined treatment trajectories of 47 163 patients. The overall incremental cost-effectiveness ratio (ICER) for telemonitoring relative to standard of care was 57 472 €/QALY. Country-specific ICERs were 60 312 €/QALY in Estonia, 58 096 €/QALY in Spain, 40 372 €/QALY in Serbia, and 90 893 €/QALY in the US, which surpassed the established willingness-to-pay thresholds. DISCUSSION: Currently, the cost-effectiveness analysis lacks standard tools, is performed in ad-hoc manner, and relies heavily on published information that might not be specific for local circumstances. Published results often exhibit a narrow focus, central to a single site, and provide only partial decision criteria, limiting their generalizability and comprehensive utility. CONCLUSION: We created 2 R-packages to pioneer cost-effectiveness analysis in OMOP CDM data networks. The first manages state definitions and database interaction, while the second focuses on Markov model learning and profile synthesis. We demonstrated their utility in a multisite heart failure study, comparing telemonitoring and standard care, finding telemonitoring not cost-effective.

Evaluating the Practicality of Causal Inference From Non-randomized Observational Data in Small-Scale Clinical Settings: A Study on the Effects of Ninjin'yoeito.

Objective The primary goal of this study was to demonstrate the practical application of causal inference using non-randomized observational data, adapting this approach to smaller populations, such as those in hospitals or community healthcare. This adaptation seeks a more effective and practical research method than randomized controlled trials (RCTs), with the goal of revealing novel insights unexplored by traditional research and enhancing understanding within the realm of causal inference. Methods This study evaluated the effects of Ninjin'yoeito (NYT), a traditional Japanese Kampo medicine, on Overactive Bladder Symptom Score (OABSS) and the frailty scores. Employing new statistical methods, this study sought to illustrate the efficacy of estimating causal relationships from non-randomized data in a clinical setting. The database included 985 women aged 65-90 years who visited a clinic between November 2016 and November 2022. By utilizing various statistical techniques, including regression analysis, inverse probability of treatment weighting (IPTW), instrumental variable (IV), and difference-in-differences (DiD) analysis, this study aimed to provide insights beyond traditional methods, attempting to bridge the gap between theory and practice in causal inference. Results After applying propensity score matching, the NYT treatment group (220 participants) and non-treatment group (182 participants) were each adjusted to two groups of 159 individuals. NYT significantly improved OABSS and frailty scores. IPTW analysis highlighted that on average, the NYT treatment group showed an improvement of 0.8671 points in OABSS and 0.1339 points in the frailty scores, surpassing the non-treatment group (p<0.05). IV analysis indicated that NYT treatment is predicted to increase ΔOABSS by an average of approximately 4.86 points, highlighting its significant positive impact on OABSS improvement. The DiD analysis showed that the NYT treatment group demonstrated an average improvement of 0.5457 points in OABSS, which was significantly higher than that of the control group. The adjusted R² value for the model is 0.025. Conclusion This study successfully implemented a practical application of causal inference using non-randomized observational data in a relatively small population. NYT showed a significant improvement in OABSS and vulnerability, and this result was confirmed using a new statistical method. The relatively low adjusted R² of the model suggests the existence of other unmeasured variables that influence OABSS and vulnerability improvement. In particular, the use of diverse statistical techniques, including IPTW, IV, and DiD analysis, is an important step toward revealing the effectiveness of inferring causal relationships from non-randomized data and narrowing the gap between theory and practice. This study provides a valid and practical alternative to RCTs and reveals new insights that have not been explored in traditional research.