Algorithmic Identification of Treatment-Emergent Adverse Events From Clinical Notes Using Large Language Models: A Pilot Study in Inflammatory Bowel Disease.

Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT) have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event (AE) detection. We adapted a new clinical LLM, University of California - San Francisco (UCSF)-BERT, to identify serious AEs (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. We annotated 928 outpatient IBD notes corresponding to 928 individual patients with IBD for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of eight candidate models, UCSF-BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF-BERT was significantly superior at identifying hospitalization events emergent to medication use (P < 0.01). LLMs like UCSF-BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared with prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multicenter data and newer architectures like Generative pre-trained transformer (GPT). Our findings support the potential value of using large language models to enhance pharmacovigilance.

Real-world effectiveness of ustekinumab and vedolizumab in TNF-exposed pediatric patients with ulcerative colitis.

OBJECTIVES: Vedolizumab (VDZ) and ustekinumab (UST) are second-line treatments in pediatric patients with ulcerative colitis (UC) refractory to antitumor necrosis factor (anti-TNF) therapy. Pediatric studies comparing the effectiveness of these medications are lacking. Using a registry from ImproveCareNow (ICN), a global research network in pediatric inflammatory bowel disease, we compared the effectiveness of UST and VDZ in anti-TNF refractory UC. METHODS: We performed a propensity-score weighted regression analysis to compare corticosteroid-free clinical remission (CFCR) at 6 months from starting second-line therapy. Sensitivity analyses tested the robustness of our findings to different ways of handling missing outcome data. Secondary analyses evaluated alternative proxies of response and infection risk. RESULTS: Our cohort included 262 patients on VDZ and 74 patients on UST. At baseline, the two groups differed on their mean pediatric UC activity index (PUCAI) (p = 0.03) but were otherwise similar. At Month 6, 28.3% of patients on VDZ and 25.8% of those on UST achieved CFCR (p = 0.76). Our primary model showed no difference in CFCR (odds ratio: 0.81; 95% confidence interval [CI]: 0.41-1.59) (p = 0.54). The time to biologic discontinuation was similar in both groups (hazard ratio: 1.26; 95% CI: 0.76-2.08) (p = 0.36), with the reference group being VDZ, and we found no differences in clinical response, growth parameters, hospitalizations, surgeries, infections, or malignancy risk. Sensitivity analyses supported these findings of similar effectiveness. CONCLUSIONS: UST and VDZ are similarly effective for inducing clinical remission in anti-TNF refractory UC in pediatric patients. Providers should consider safety, tolerability, cost, and comorbidities when deciding between these therapies.

Accurate, Robust, and Scalable Machine Abstraction of Mayo Endoscopic Subscores From Colonoscopy Reports.

BACKGROUND: The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. METHODS: We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. RESULTS: Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). CONCLUSIONS: We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.

Our accurate pair of models automatically classify colonoscopy reports by Mayo endoscopic subscore and abstain from prediction appropriately. Our methods can enable large-scale electronic health record studies of treatment effectiveness, prospective identification of patients for clinical trials, and quality improvement efforts in ulcerative colitis.

Characterisation of digital therapeutic clinical trials: a systematic review with natural language processing.

Digital therapeutics (DTx) are a somewhat novel class of US Food and Drug Administration-regulated software that help patients prevent, manage, or treat disease. Here, we use natural language processing to characterise registered DTx clinical trials and provide insights into the clinical development landscape for these novel therapeutics. We identified 449 DTx clinical trials, initiated or expected to be initiated between 2010 and 2030, from ClinicalTrials.gov using 27 search terms, and available data were analysed, including trial durations, locations, MeSH categories, enrolment, and sponsor types. Topic modelling of eligibility criteria, done with BERTopic, showed that DTx trials frequently exclude patients on the basis of age, comorbidities, pregnancy, language barriers, and digital determinants of health, including smartphone or data plan access. Our comprehensive overview of the DTx development landscape highlights challenges in designing inclusive DTx clinical trials and presents opportunities for clinicians and researchers to address these challenges. Finally, we provide an interactive dashboard for readers to conduct their own analyses.

Assessing the Impact of COVID-19 on IBD Outcomes Among Vulnerable Patient Populations in a Large Metropolitan Center.

BACKGROUND: With the onset of COVID-19, there were rapid changes in healthcare delivery as remote access became the norm. The aim of this study was to determine the impact of changes in healthcare delivery during the COVID-19 pandemic on patients with inflammatory bowel disease (IBD), in both well-resourced and vulnerable populations. METHODS: Using a mixed methods, observational study design, patients receiving IBD care at a university or a safety-net hospital were identified by the electronic health record. Patient demographics, IBD history, and disease activity were acquired from the electronic health record. IBD-related outcomes were compared from the onset of the pandemic in the United States until December 2020 (COVID-19 pandemic year 1) and compared with outcomes in the previous year. A subset of participants provided their perspective on how changes in healthcare delivery and financial stability impacted their IBD through a standardized questionnaire and semi-structured interview. RESULTS: Data from a total of 1449 participants were captured, 1324 at the tertiary care university hospital and 125 at the safety-net hospital. During COVID-19, there was a decrease in healthcare utilization at both sites. Race/ethnicity and primary language were not associated with IBD-related hospitalizations or admissions. Patients that were employed and those with insurance had a higher number of IBD-related emergency department visits at both the university and safety-net hospitals (P = .03 and P = .01, respectively). Patients who did not speak English were more likely to report challenges using technology with telehealth and difficulty contacting IBD providers. CONCLUSIONS: For IBD populations, during COVID-19, in both hospital settings, emergency department visits, hospitalizations, outpatient surgery, and clinic visits were reduced compared with the year prior. Patients with lower socioeconomic status and limited English proficiency reported facing more challenges with changes to healthcare delivery, healthcare access, and conveying changes in IBD activity. These results highlight the need for payors and providers to specifically attend to those populations most susceptible to these systemic and lasting changes in care delivery and promote greater equity in healthcare.

Personalizing treatment selection in Crohn's disease: a meta-analysis of individual participant data from fifteen randomized controlled trials.

BACKGROUND: Meta-analyses have found anti-TNF drugs to be the best treatment, on average, for Crohn's disease. We performed a subgroup analysis to determine if it is possible to achieve more efficacious outcomes by individualizing treatment selection. METHODS: We obtained participant-level data from 15 trials of FDA-approved treatments (N=5703). We used sequential regression and simulation to model week six disease activity as a function of drug class, demographics, and disease-related features. We performed hypothesis testing to define subgroups based on rank-ordered preferences for treatments. We queried health records from University of California Health (UCH) to estimate the impacts these models could have on practice. We computed the sample size needed to prospectively test a prediction of our models. RESULTS: 45% of the participants (N=2561) showed greater efficacy with at least one drug class (anti-TNF, anti-IL-12/23, anti-integrin) over another. They were classifiable into 6 subgroups, two showing greatest efficacy with anti-TNFs (36%, N=2064). Women over 50 showed superior responses with anti-IL-12/23s. Although they represented only 2% of the trial-based cohort, 25% of Crohn's patients at UCH are women over 50 (N=5,647), consistent with potential selection bias in trials. Moreover, 75% of biologic-exposed women over 50 did not receive an anti-IL12/23 first-line, supporting the potential value of these models. A future trial with 250 patients per arm will have 97% power to confirm the superiority of anti-IL-12/23s over anti-TNFs in these patients. A treatment recommendation tool is available at https://crohnsrx.org. CONCLUSIONS: Personalizing treatment can improve outcomes in Crohn's disease. Future work is needed to confirm these findings, and improve representativeness in Crohn's trials.

Sequential regression and simulation: a method for estimating causal effects from heterogeneous clinical trials without a common control group.

BACKGROUND: The advent of clinical trial data sharing platforms has created opportunities for making new discoveries and answering important questions using already collected data. However, existing methods for meta-analyzing these data require the presence of shared control groups across studies, significantly limiting the number of questions that can be confidently addressed. We sought to develop a method for meta-analyzing potentially heterogeneous clinical trials even in the absence of a common control group. METHODS: This work was conducted within the context of a broader effort to study comparative efficacy in Crohn's disease. Following a search of clnicaltrials.gov we obtained access to the individual participant data from nine trials of FDA-approved treatments in Crohn's Disease (N = 3392). We developed a method involving sequences of regression and simulation to separately model the placebo- and drug-attributable effects, and to simulate head-to-head trials against an appropriately normalized background. We validated this method by comparing the outcome of a simulated trial comparing the efficacies of adalimumab and ustekinumab against the recently published results of SEAVUE, an actual head-to-head trial of these drugs. This study was pre-registered on PROSPERO (#157,827) prior to the completion of SEAVUE. RESULTS: Using our method of sequential regression and simulation, we compared the week eight outcomes of two virtual cohorts subject to the same patient selection criteria as SEAVUE and treated with adalimumab or ustekinumab. Our primary analysis replicated the corresponding published results from SEAVUE (p = 0.9). This finding proved stable under multiple sensitivity analyses. CONCLUSIONS: This new method may help reduce the bias of individual participant data meta-analyses, expand the scope of what can be learned from these already-collected data, and reduce the costs of obtaining high-quality evidence to guide patient care.

Reducing diagnostic delays in Acute Hepatic Porphyria using electronic health records data and machine learning: a multicenter development and validation study.

Importance: Acute Hepatic Porphyria (AHP) is a group of rare but treatable conditions associated with diagnostic delays of fifteen years on average. The advent of electronic health records (EHR) data and machine learning (ML) may improve the timely recognition of rare diseases like AHP. However, prediction models can be difficult to train given the limited case numbers, unstructured EHR data, and selection biases intrinsic to healthcare delivery. Objective: To train and characterize models for identifying patients with AHP. Design Setting and Participants: This diagnostic study used structured and notes-based EHR data from two centers at the University of California, UCSF (2012-2022) and UCLA (2019-2022). The data were split into two cohorts (referral, diagnosis) and used to develop models that predict: 1) who will be referred for testing of acute porphyria, amongst those who presented with abdominal pain (a cardinal symptom of AHP), and 2) who will test positive, amongst those referred. The referral cohort consisted of 747 patients referred for testing and 99,849 contemporaneous patients who were not. The diagnosis cohort consisted of 72 confirmed AHP cases and 347 patients who tested negative. Cases were female predominant and 6-75 years old at the time of diagnosis. Candidate models used a range of architectures. Feature selection was semi-automated and incorporated publicly available data from knowledge graphs. Main Outcomes and Measures: F-score on an outcome-stratified test set. Results: The best center-specific referral models achieved an F-score of 86-91%. The best diagnosis model achieved an F-score of 92%. To further test our model, we contacted 372 current patients who lack an AHP diagnosis but were predicted by our models as potentially having it (≥ 10% probability of referral, ≥ 50% of testing positive). However, we were only able to recruit 10 of these patients for biochemical testing, all of whom were negative. Nonetheless, post hoc evaluations suggested that these models could identify 71% of cases earlier than their diagnosis date, saving 1.2 years. Conclusions and Relevance: ML can reduce diagnostic delays in AHP and other rare diseases. Robust recruitment strategies and multicenter coordination will be needed to validate these models before they can be deployed.

Algorithmic identification of treatment-emergent adverse events from clinical notes using large language models: a pilot study in inflammatory bowel disease.

Background and Aims: Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLM) like BERT have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event detection. Methods: We adapted a new clinical LLM, UCSF BERT, to identify serious adverse events (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. Results: We annotated 928 outpatient IBD notes corresponding to 928 individual IBD patients for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of 8 candidate models, UCSF BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF BERT was significantly superior at identifying hospitalization events emergent to medication use (p < 0.01). Conclusions: LLMs like UCSF BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared to prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multi-center data and newer architectures like GPT. Our findings support the potential value of using large language models to enhance pharmacovigilance.

ALK Inhibitor Treatment Patterns and Outcomes in Real-World Patients with ALK-Positive Non-Small-Cell Lung Cancer: A Retrospective Cohort Study.

BACKGROUND: Randomized trials have demonstrated that anaplastic lymphoma kinase (ALK) tyrosine kinase inhibitors (TKIs) can be safe and efficacious treatments for patients with ALK-positive advanced non-small-cell lung cancer (aNSCLC). However, their safety, tolerability, effectiveness, and patterns of use in real-world patients remain understudied. OBJECTIVE: We sought to assess the overall treatment pattern characteristics, safety, and effectiveness outcomes of real-world patients with ALK-positive aNSCLC receiving ALK TKIs. PATIENTS AND METHODS: This retrospective cohort study using electronic health record data included adult patients with ALK-positive aNSCLC receiving ALK TKIs between January 2012 and November 2021 at a large tertiary medical center, University of California, San Francisco (UCSF), with alectinib or crizotinib as the initial ALK TKI therapy. Our primary endpoints included the incidence of treatment changes (treatment dose adjustments, interruptions, and discontinuations) during the initial ALK TKI treatment, the count and type of subsequent treatments, rates of serious adverse events (sAEs), and major adverse events (mAEs) leading to any ALK TKI treatment changes. Secondary endpoints included the hazard ratios (HRs) for median mAE-free survival (mAEFS), real-world progression-free survival (rwPFS), and overall survival (OS) when comparing alectinib with crizotinib. RESULTS: The cohort consisted of 117 adult patients (70 alectinib and 47 crizotinib) with ALK-positive aNSCLC, with 24.8%, 17.9%, and 6.0% experiencing treatment dose adjustments, interruptions, and discontinuation, respectively. Of the 73 patients whose ALK TKI treatments were discontinued, 68 received subsequent treatments including newer generations of ALK TKIs, immune checkpoint inhibitors, and chemotherapies. The most common mAEs were rash (9.9%) and bradycardia (7.0%) for alectinib and liver toxicity (19.1%) for crizotinib. The most common sAEs were pericardial effusion (5.6%) and pleural effusion (5.6%) for alectinib and pulmonary embolism (6.4%) for crizotinib. Patients receiving alectinib versus crizotinib as their first ALK TKI treatment experienced significantly prolonged median rwPFS (29.3 versus 10.4 months) with an HR of 0.38 (95% CI 0.21-0.67), while prolonged median mAEFS (not reached versus 91.3 months) and OS (54.1 versus 45.8 months) were observed in patients receiving alectinib versus crizotinib but did not reach statistical significance. Yet, it is worth noting that there was a high degree of cross-over post-progression, which could significantly confound the overall survival measures. CONCLUSIONS: We found that ALK TKIs were highly tolerable, and alectinib was associated with favorable survival outcomes with longer time to adverse events (AE) requiring medical interventions, disease progression, and death, in the context of real-world use. Proactive monitoring for adverse events such as rash, bradycardia, and hepatotoxicity may help further promote the safe and optimal use of ALK TKIs in the treatment of patients with aNSCLC.

Creation of an ustekinumab external control arm for Crohn's disease using electronic health records data: A pilot study.

BACKGROUND: Randomized trials are the gold-standard for clinical evidence generation, but they can sometimes be limited by infeasibility and unclear generalizability to real-world practice. External control arm (ECA) studies may help address this evidence gaps by constructing retrospective cohorts that closely emulate prospective ones. Experience in constructing these outside the context of rare diseases or cancer is limited. We piloted an approach for developing an ECA in Crohn's disease using electronic health records (EHR) data. METHODS: We queried EHR databases and manually screened records at the University of California, San Francisco to identify patients meeting the eligibility criteria of TRIDENT, a recently completed interventional trial involving an ustekinumab reference arm. We defined timepoints to balance missing data and bias. We compared imputation models by their impacts on cohort membership and outcomes. We assessed the accuracy of algorithmic data curation against manual review. Lastly, we assessed disease activity following treatment with ustekinumab. RESULTS: Screening identified 183 patients. 30% of the cohort had missing baseline data. Nonetheless, cohort membership and outcomes were robust to the method of imputation. Algorithms for ascertaining non-symptom-based elements of disease activity using structured data were accurate against manual review. The cohort consisted of 56 patients, exceeding planned enrollment in TRIDENT. 34% of the cohort was in steroid-free remission at week 24. CONCLUSION: We piloted an approach for creating an ECA in Crohn's disease from EHR data by using a combination of informatics and manual methods. However, our study reveals significant missing data when standard-of-care clinical data are repurposed. More work will be needed to improve the alignment of trial design with typical patterns of clinical practice, and thereby enable a future of more robust ECAs in chronic diseases like Crohn's disease.

Comparative Safety and Effectiveness of Biologic Therapy for Crohn's Disease: A CA-IBD Cohort Study.

BACKGROUND & AIMS: We compared the safety and effectiveness of tumor necrosis factor α (TNF-α) antagonists vs vedolizumab vs ustekinumab in patients with Crohn's disease (CD) in a multicenter cohort (CA-IBD). METHODS: We created an electronic health record-based cohort of adult patients with CD who were initiating a new biologic agent (TNF-α antagonists, ustekinumab, vedolizumab) from 5 health systems in California between 2010 and 2017. We compared the risk of serious infections (safety) and all-cause hospitalization and inflammatory bowel disease-related surgery (effectiveness) between different biologic classes using propensity score (PS) matching. RESULTS: As compared with TNF-α antagonists (n = 1030), 2:1 PS-matched, ustekinumab-treated patients with CD (n = 515) experienced a lower risk of serious infections (hazard ratio [HR], 0.36; 95% CI, 0.20-0.64), without any difference in the risk of hospitalization (HR, 0.99; 95% CI, 0.89-1.21) or surgery (HR, 1.08; 95% CI, 0.69-1.70). Compared with vedolizumab (n = 221), 1:1 PS-matched, ustekinumab-treated patients with CD (n = 221) experienced a lower risk of serious infections (HR, 0.20; 95% CI, 0.07-0.60), without significant differences in risk of hospitalization (HR, 0.76; 95% CI, 0.54-1.07) or surgery (HR, 1.42; 95% CI, 0.54-3.72). Compared with TNF-α antagonists (n = 442), 2:1 PS-matched, vedolizumab-treated patients with CD (n = 221) had a similar risk of serious infections (HR, 1.53; 95% CI, 0.84-2.78), hospitalization (HR, 1.32; 95% CI, 0.98-1.77), and surgery (HR, 0.63; 95% CI, 0.27-1.47). High comorbidity burden, concomitant opiate use, and prior hospitalization were associated with serious infections and hospitalization in biologic-treated patients with CD. CONCLUSION: In a multicenter cohort of biologic-treated patients with CD, ustekinumab was associated with a lower risk of serious infections compared with TNF-α antagonists and vedolizumab, without any differences in risk of hospitalization or surgery. The risk of serious infections was similar for TNF-α antagonists vs vedolizumab.

Effectiveness and Safety of Biologic Therapy in Hispanic Vs Non-Hispanic Patients With Inflammatory Bowel Diseases: A CA-IBD Cohort Study.

BACKGROUND & AIMS: There are limited data on outcomes of biologic therapy in Hispanic patients with inflammatory bowel diseases (IBDs). We compared risk of hospitalization, surgery, and serious infections in Hispanic vs non-Hispanic patients with IBD in a multicenter, electronic health record-based cohort of biologic-treated patients. METHODS: We identified adult patients with IBD who were new users of biologic agents (tumor necrosis factor α [TNF-α] antagonists, ustekinumab, vedolizumab) from 5 academic institutions in California between 2010 and 2017. We compared the risk of all-cause hospitalization, IBD-related surgery, and serious infections in Hispanic vs non-Hispanic patients using 1:4 propensity score matching and survival analysis. RESULTS: We compared 240 Hispanic patients (53% male; 45% with ulcerative colitis; 73% TNF-α antagonist-treated; 20% with prior biologic exposure) with 960 non-Hispanic patients (51% male; 44% with ulcerative colitis; 67% TNF-α antagonist-treated; 27% with prior biologic exposure). After propensity score matching, Hispanic patients were younger (37 ± 15 vs 40 ± 16 y; P = .02) and had a higher burden of comorbidities (Elixhauser index, >0; 37% vs 26%; P < .01), without any differences in patterns of medication use, burden of inflammation, and hospitalizations. Within 1 year of biologic initiation, Hispanic patients had higher rates of hospitalizations (31% vs 23%; adjusted hazard ratio [aHR], 1.32; 95% CI, 1.01-1.74) and IBD-related surgery (7.1% vs 4.6%; aHR, 2.00; 95% CI, 1.07-3.72), with a trend toward higher risk of serious infections (8.8% vs 4.9%; aHR, 1.74; 95% CI, 0.99-3.05). CONCLUSIONS: In a multicenter, propensity score-matched cohort of biologic-treated patients with IBD, Hispanic patients experienced higher rates of hospitalization, surgery, and serious infections. Future studies are needed to investigate the biological, social, and environmental drivers of these differences.

Effect of Obesity on Risk of Hospitalization, Surgery, and Serious Infection in Biologic-Treated Patients With Inflammatory Bowel Diseases: A CA-IBD Cohort Study.

INTRODUCTION: Obesity is variably associated with treatment response in biologic-treated patients with inflammatory bowel diseases (IBD). We evaluated the association between obesity and risk of hospitalization, surgery, or serious infections in patients with IBD in new users of biologic agents in a large, multicenter, electronic health record (EHR)-based cohort (CA-IBD). METHODS: We created an EHR-based cohort of adult patients with IBD who were new users of biologic agents (tumor necrosis factor [TNF-α] antagonists, ustekinumab, and vedolizumab) between January 1, 2010, and June 30, 2017, from 5 health systems in California. Patients were classified as those with normal body mass index (BMI), overweight, or obese based on the World Health Organization classification. We compared the risk of all-cause hospitalization, IBD-related surgery, or serious infections among patients with obesity vs those overweight vs those with normal BMI, using Cox proportional hazard analyses, adjusting for baseline demographic, disease, and treatment characteristics. RESULTS: Of 3,038 biologic-treated patients with IBD (69% with Crohn's disease and 76% on TNF-α antagonists), 28.2% (n = 858) were overweight, and 13.7% (n = 416) were obese. On a follow-up after biologic initiation, obesity was not associated with an increased risk of hospitalization (adjusted hazard ratio [aHR] vs normal BMI, 0.90; [95% confidence interval, 0.72-1.13]); IBD-related surgery (aHR, 0.62 [0.31-1.22]); or serious infection (aHR, 1.11 [0.73-1.71]). Similar results were observed on stratified analysis by disease phenotype (Crohn's disease vs ulcerative colitis) and index biologic therapy (TNF-α antagonists vs non-TNF-α antagonists). DISCUSSION: In a multicenter, EHR-based cohort of biologic-treated patients with IBD, obesity was not associated with hospitalization, surgery, or serious infections. Further studies examining the effect of visceral obesity on patient-reported and endoscopic outcomes are needed.

Case-control study of the association of chronic acid suppression and social determinants of health with COVID-19 infection.

Acid suppressants are widely-used classes of medications linked to increased risks of aerodigestive infections. Prior studies of these medications as potentially reversible risk factors for COVID-19 have been conflicting. We aimed to determine the impact of chronic acid suppression use on COVID-19 infection risk while simultaneously evaluating the influence of social determinants of health to validate known and discover novel risk factors. We assessed the association of chronic acid suppression with incident COVID-19 in a 1:1 case-control study of 900 patients tested across three academic medical centers in California, USA. Medical comorbidities and history of chronic acid suppression use were manually extracted from health records by physicians following a pre-specified protocol. Socio-behavioral factors by geomapping publicly-available data to patient zip codes were incorporated. We identified no evidence to support an association between chronic acid suppression and COVID-19 (adjusted odds ratio 1.04, 95% CI 0.92-1.17, P = 0.515). However, several medical and social features were positive (Latinx ethnicity, BMI ≥ 30, dementia, public transportation use, month of the pandemic) and negative (female sex, concurrent solid tumor, alcohol use disorder) predictors of new infection. These findings demonstrate the value of integrating publicly-available databases with medical data to identify critical features of communicable diseases.

Utility of routinely collected electronic health records data to support effectiveness evaluations in inflammatory bowel disease: a pilot study of tofacitinib.

OBJECTIVES: Electronic health records (EHR) are receiving growing attention from regulators, biopharmaceuticals and payors as a potential source of real-world evidence. However, their suitability for the study of diseases with complex activity measures is unclear. We sought to evaluate the use of EHR data for estimating treatment effectiveness in inflammatory bowel disease (IBD), using tofacitinib as a use case. METHODS: Records from the University of California, San Francisco (6/2012 to 4/2019) were queried to identify tofacitinib-treated IBD patients. Disease activity variables at baseline and follow-up were manually abstracted according to a preregistered protocol. The proportion of patients meeting the endpoints of recent randomised trials in ulcerative colitis (UC) and Crohn's disease (CD) was assessed. RESULTS: 86 patients initiated tofacitinib. Baseline characteristics of the real-world and trial cohorts were similar, except for universal failure of tumour necrosis factor inhibitors in the former. 54% (UC) and 62% (CD) of patients had complete capture of disease activity at baseline (month -6 to 0), while only 32% (UC) and 69% (CD) of patients had complete follow-up data (month 2 to 8). Using data imputation, we estimated the proportion achieving the trial primary endpoints as being similar to the published estimates for both UC (16%, p value=0.5) and CD (38%, p-value=0.8). DISCUSSION/CONCLUSION: This pilot study reproduced trial-based estimates of tofacitinib efficacy despite its use in a different cohort but revealed substantial missingness in routinely collected data. Future work is needed to strengthen EHR data and enable real-world evidence in complex diseases like IBD.

CovidCounties is an interactive real time tracker of the COVID19 pandemic at the level of US counties.

Management of the COVID-19 pandemic has proven to be a significant challenge to policy makers. This is in large part due to uneven reporting and the absence of open-access visualization tools to present local trends and infer healthcare needs. Here we report the development of CovidCounties.org, an interactive web application that depicts daily disease trends at the level of US counties using time series plots and maps. This application is accompanied by a manually curated dataset that catalogs all major public policy actions made at the state-level, as well as technical validation of the primary data. Finally, the underlying code for the site is also provided as open source, enabling others to validate and learn from this work.

CovidCounties - an interactive, real-time tracker of the COVID-19 pandemic at the level of US counties.

Management of the COVID-19 pandemic has proven to be a significant challenge to policy makers. This is in large part due to uneven reporting and the absence of open-access visualization tools to present local trends and infer healthcare needs. Here we report the development of CovidCounties.org, an interactive web application that depicts daily disease trends at the level of US counties using time series plots and maps. This application is accompanied by a manually curated dataset that catalogs all major public policy actions made at the state-level, as well as technical validation of the primary data. Finally, the underlying code for the site is also provided as open source, enabling others to validate and learn from this work.

Accuracy of medical billing data against the electronic health record in the measurement of colorectal cancer screening rates.

OBJECTIVE: Medical billing data are an attractive source of secondary analysis because of their ease of use and potential to answer population-health questions with statistical power. Although these datasets have known susceptibilities to biases, the degree to which they can distort the assessment of quality measures such as colorectal cancer screening rates are not widely appreciated, nor are their causes and possible solutions. METHODS: Using a billing code database derived from our institution's electronic health records, we estimated the colorectal cancer screening rate of average-risk patients aged 50-74 years seen in primary care or gastroenterology clinic in 2016-2017. 200 records (150 unscreened, 50 screened) were sampled to quantify the accuracy against manual review. RESULTS: Out of 4611 patients, an analysis of billing data suggested a 61% screening rate, an estimate that matches the estimate by the Centers for Disease Control. Manual review revealed a positive predictive value of 96% (86%-100%), negative predictive value of 21% (15%-29%) and a corrected screening rate of 85% (81%-90%). Most false negatives occurred due to examinations performed outside the scope of the database-both within and outside of our institution-but 21% of false negatives fell within the database's scope. False positives occurred due to incomplete examinations and inadequate bowel preparation. Reasons for screening failure include ordered but incomplete examinations (48%), lack of or incorrect documentation by primary care (29%) including incorrect screening intervals (13%) and patients declining screening (13%). CONCLUSIONS: Billing databases are prone to substantial bias that may go undetected even in the presence of confirmatory external estimates. Caution is recommended when performing population-level inference from these data. We propose several solutions to improve the use of these data for the assessment of healthcare quality.

Opportunities and challenges in using real-world data for health care.

Real-world data (RWD) continue to emerge as a new source of clinical evidence. Although the best-known use case of RWD has been in drug regulation, RWD are being generated and used by many other parties, including biopharmaceutical companies, payors, clinical researchers, providers, and patients. In this Review, we describe 21 potential uses for RWD across the spectrum of health care. We also discuss important challenges and limitations relevant to the translation of these data into evidence.