A conceptual framework for external validity.

Evidence-Based Medicine (EBM) encourages clinicians to seek the most reputable evidence. The quality of evidence is organized in a hierarchy in which randomized controlled trials (RCTs) are regarded as least biased. However, RCTs are plagued by poor generalizability, impeding the translation of clinical research to practice. Though the presence of poor external validity is known, the factors that contribute to poor generalizability have not been summarized and placed in a framework. We propose a new population-oriented conceptual framework to facilitate consistent and comprehensive evaluation of generalizability, replicability, and assessment of RCT study quality.

The Counterfactual χ-GAN: Finding comparable cohorts in observational health data.

Causal inference often relies on the counterfactual framework, which requires that treatment assignment is independent of the outcome, known as strong ignorability. Approaches to enforcing strong ignorability in causal analyses of observational data include weighting and matching methods. Effect estimates, such as the average treatment effect (ATE), are then estimated as expectations under the re-weighted or matched distribution, P. The choice of P is important and can impact the interpretation of the effect estimate and the variance of effect estimates. In this work, instead of specifying P, we learn a distribution that simultaneously maximizes coverage and minimizes variance of ATE estimates. In order to learn this distribution, this research proposes a generative adversarial network (GAN)-based model called the Counterfactual χ-GAN (cGAN), which also learns feature-balancing weights and supports unbiased causal estimation in the absence of unobserved confounding. Our model minimizes the Pearson χ2-divergence, which we show simultaneously maximizes coverage and minimizes the variance of importance sampling estimates. To our knowledge, this is the first such application of the Pearson χ2-divergence. We demonstrate the effectiveness of cGAN in achieving feature balance relative to established weighting methods in simulation and with real-world medical data.

Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations.

Randomized controlled trials (RCTs) are regarded as the most reputable source of evidence. In some studies, factors beyond the intervention itself may contribute to the measured effect, an occurrence known as heterogeneity of treatment effect (HTE). If the RCT population differs from the real-world population on factors that induce HTE, the trials effect will not replicate. The RCTs eligibility criteria should identify the sub-population in which its evidence will replicate. However, the extent to which the eligibility criteria identify the appropriate population is unknown, which raises concerns for generalizability. We compared reported data from RCTs with real-world data from the electronic health records of a large, academic medical center that was curated according to RCT eligibility criteria. Our results show fundamental differences between the RCT population and our observational cohorts, which suggests that eligibility criteria may be insufficient for identifying the applicable real-world population in which RCT evidence will replicate.

Characterizing non-heroin opioid overdoses using electronic health records.

INTRODUCTION: The opioid epidemic is a modern public health emergency. Common interventions to alleviate the opioid epidemic aim to discourage excessive prescription of opioids. However, these methods often take place over large municipal areas (state-level) and may fail to address the diversity that exists within each opioid case (individual-level). An intervention to combat the opioid epidemic that takes place at the individual-level would be preferable. METHODS: This research leverages computational tools and methods to characterize the opioid epidemic at the individual-level using the electronic health record data from a large, academic medical center. To better understand the characteristics of patients with opioid use disorder (OUD) we leveraged a self-controlled analysis to compare the healthcare encounters before and after an individual's first overdose event recorded within the data. We further contrast these patients with matched, non-OUD controls to demonstrate the unique qualities of the OUD cohort. RESULTS: Our research confirms that the rate of opioid overdoses in our hospital significantly increased between 2006 and 2015 (P < 0.001), at an average rate of 9% per year. We further found that the period just prior to the first overdose is marked by conditions of pain or malignancy, which may suggest that overdose stems from pharmaceutical opioids prescribed for these conditions. CONCLUSIONS: Informatics-based methodologies, like those presented here, may play a role in better understanding those individuals who suffer from opioid dependency and overdose, and may lead to future research and interventions that could successfully prevent morbidity and mortality associated with this epidemic.

Economic burden of inadequate symptom control among US commercially insured patients with irritable bowel syndrome with diarrhea.

AIMS: To assess healthcare resource use and costs among irritable bowel syndrome (IBS) with diarrhea (IBS-D) patients with and without evidence of inadequate symptom control on current prescription therapies and estimate incremental all-cause costs associated with inadequate symptom control. METHODS: IBS-D patients aged ≥18 years with ≥1 medical claim for IBS (ICD-9-CM 564.1x) and either ≥2 claims for diarrhea (ICD-9-CM 787.91, 564.5x), ≥1 claim for diarrhea plus ≥1 claim for abdominal pain (ICD-9-CM 789.0x), or ≥1 claim for diarrhea plus ≥1 pharmacy claim for a symptom-related prescription within 1 year of an IBS diagnosis were identified from the Truven Health MarketScan database. Inadequate symptom control, resource use, and costs were assessed up to 1 year following the index date. Inadequate symptom control included any of the following: (1) switch or (2) addition of new symptom-related therapy; (3) IBS-D-related inpatient or emergency room (ER) admission; (4) IBS-D-related medical procedure; (5) diagnosis of condition indicating treatment failure; or (6) use of a more aggressive prescription. Generalized linear models assessed incremental costs of inadequate symptom control. RESULTS: Of 20,624 IBS-D patients (mean age = 48.5 years; 77.8% female), 66.4% had evidence of inadequate symptom control. Compared to those without inadequate symptom control, patients with evidence of inadequate symptom control had significantly more hospitalizations (12.0% vs 6.0%), ER visits (37.1% vs 22.6%), use of outpatient services (73.0% vs 60.7%), physician office visits (mean 11.0 vs 8.1), and prescription fills (mean 40.0 vs 26.7) annually (all p < .01). Incremental costs associated with inadequate symptom control were $3,065 (2013 US dollars), and were driven by medical service costs ($2,391; 78%). LIMITATIONS: Study included US commercially insured patients only and inferred IBS-D status and inadequate symptom control from claims. CONCLUSIONS: Inadequate symptom control associated with available IBS-D therapies represents a significant economic burden for both payers and IBS-D patients.

Economic Burden of Irritable Bowel Syndrome with Diarrhea: Retrospective Analysis of a U.S. Commercially Insured Population.

BACKGROUND: The economic burden associated with irritable bowel syndrome with diarrhea (IBS-D) is not well understood. OBJECTIVES: To (a) evaluate total annual all-cause, gastrointestinal (GI)-related, and symptom-related (i.e., IBS, diarrhea, abdominal pain) health care resource use and costs among IBS-D patients in a U.S. commercially insured population and (b) estimate incremental all-cause health care costs of IBS-D patients versus matched controls. METHODS: Patients aged ≥ 18 years with 12 months of continuous medical and pharmacy benefit eligibility in 2013 were identified from the Truven Health MarketScan research database. The study sample included patients with ≥ 1 medical claim with an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code in any position for IBS (ICD-9-CM 564.1x) and either (a) ≥ 2 claims for diarrhea (ICD-9-CM 787.91, 564.5x) on different service dates in 2013, or (b) ≥ 1 claim for diarrhea plus ≥ 1 claim for abdominal pain (ICD-9-CM 789.0x) on different service dates in 2013, or (c) ≥ 1 claim for diarrhea plus ≥ 1 pharmacy claim for a symptom-related prescription on different service dates in 2013. Controls included patients with no claims for IBS, diarrhea, abdominal pain, or symptom-related prescriptions in 2013. Controls were randomly selected and matched with IBS-D patients in a 1:1 ratio based on age (± 4 years), gender, geographic location, and health plan type. All-cause health care resource utilization included medical and pharmacy claims for health care services associated with any condition. Total health care costs were defined as the sum of health plan-paid and patient-paid direct health care costs from prescriptions and medical services, including inpatient, emergency department (ED), and physician office visits, and other outpatient services. A total cost approach was used to assess all-cause, GI-related, and symptom-related health care costs for IBS-D patients. An incremental cost approach via generalized linear models was used to assess the excess all-cause costs attributable to IBS-D after adjusting for demographics and general and GI comorbidities. RESULTS: Of 39,306 patients (n = 19,653 each for IBS-D and matched controls) included, mean (± SD) age was 47 (± 17) years and 76.5% were female. Compared with controls, IBS-D patients had a significantly higher mean annual number of hospitalizations, ED visits, office visits, and monthly (30-day) prescription fills. Mean annual all-cause health care costs for IBS-D patients were $13,038, with over half (58.4%) attributable to office visits and other outpatient services (e.g., diagnostic tests and laboratory or radiology services), and remaining costs attributable to prescriptions (19.5%), inpatient admissions (13.6%), and ED visits (8.5%). GI-related ($3,817) and symptom-related ($1,693) costs were also primarily driven by other outpatient service costs. After adjusting for demographics and comorbidities, incremental annual all-cause costs associated with IBS-D were $2,268 ($9,436 for IBS-D patients vs. $7,169 for matched controls; P < 0.001) per patient/year, of which 78% were from medical costs and 22% were from prescription costs. CONCLUSIONS: IBS-D was associated with a substantial burden in direct costs in this population. Compared with matched controls, IBS-D patients had greater medical service use and incurred significantly more annual all-cause health care costs, even after controlling for demographics and comorbidities. Incremental costs associated with IBS-D were primarily attributable to increased use of medical services rather than pharmacy costs. DISCLOSURES: This study was funded by Allergan. The authors received no compensation related to the development of the manuscript. Buono and Andrae are employees of Allergan. Mathur is an employee of Axtria. Averitt was an employee of Axtria at the time this study was conducted. Data from this manuscript have previously been presented in poster format by Buono at the American College of Gastroenterology Annual Scientific Meeting; Honolulu, Hawaii; October 16-21, 2015. Mathur and Averitt were involved in conducting the study analyses. All authors were involved in the study design, interpretation of the data, and preparation of the manuscript. The authors take full responsibility for the scope, direction, and content of the manuscript and have approved the submitted manuscript.