Validation of an ICD-Code-Based Case Definition for Psychotic Illness Across Three Health Systems.

BACKGROUND AND HYPOTHESIS: Psychosis-associated diagnostic codes are increasingly being utilized as case definitions for electronic health record (EHR)-based algorithms to predict and detect psychosis. However, data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis. STUDY DESIGN: Using EHRs at 3 health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into 5 higher-order groups. 1133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings. STUDY RESULTS: PPVs across all diagnostic groups and hospital systems exceeded 70%: Mass General Brigham 0.72 [95% CI 0.68-0.77], Boston Children's Hospital 0.80 [0.75-0.84], and Boston Medical Center 0.83 [0.79-0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80-0.92) and major depressive disorder with psychosis were the most variable (0.57-0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08-0.62). CONCLUSIONS: We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the case definitions used in the development of risk prediction models designed to predict or detect undiagnosed psychosis.

Development and Validation of a Colorectal Cancer Prediction Model: A Nationwide Cohort-Based Study.

BACKGROUND: Early diagnosis of colorectal cancer (CRC) is critical to increasing survival rates. Computerized risk prediction models hold great promise for identifying individuals at high risk for CRC. In order to utilize such models effectively in a population-wide screening setting, development and validation should be based on cohorts that are similar to the target population. AIM: Establish a risk prediction model for CRC diagnosis based on electronic health records (EHR) from subjects eligible for CRC screening. METHODS: A retrospective cohort study utilizing the EHR data of Clalit Health Services (CHS). The study includes CHS members aged 50-74 who were eligible for CRC screening from January 2013 to January 2019. The model was trained to predict receiving a CRC diagnosis within 2 years of the index date. Approximately 20,000 EHR demographic and clinical features were considered. RESULTS: The study includes 2935 subjects with CRC diagnosis, and 1,133,457 subjects without CRC diagnosis. Incidence values of CRC among subjects in the top 1% risk scores were higher than baseline (2.3% vs 0.3%; lift 8.38; P value < 0.001). Cumulative event probabilities increased with higher model scores. Model-based risk stratification among subjects with a positive FOBT, identified subjects with more than twice the risk for CRC compared to FOBT alone. CONCLUSIONS: We developed an individualized risk prediction model for CRC that can be utilized as a complementary decision support tool for healthcare providers to precisely identify subjects at high risk for CRC and refer them for confirmatory testing.

Validation of an ICD-code-based case definition for psychotic illness across three health systems.

Background and Hypothesis: Early detection of psychosis is critical for improving outcomes. Algorithms to predict or detect psychosis using electronic health record (EHR) data depend on the validity of the case definitions used, typically based on diagnostic codes. Data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis. Study Design: Using EHRs at three health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into five higher-order groups. 1,133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings. Study Results: PPVs across all diagnostic groups and hospital systems exceeded 70%: Massachusetts General Brigham 0.72 [95% CI 0.68-0.77], Boston Children's Hospital 0.80 [0.75-0.84], and Boston Medical Center 0.83 [0.79-0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80-0.92) and major depressive disorder with psychosis were the most variable (0.57-0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08-0.62). Conclusions: We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the development of risk prediction models designed to predict or detect undiagnosed psychosis.

Harnessing the Power of Generative AI for Clinical Summaries: Perspectives From Emergency Physicians.

STUDY OBJECTIVE: The workload of clinical documentation contributes to health care costs and professional burnout. The advent of generative artificial intelligence language models presents a promising solution. The perspective of clinicians may contribute to effective and responsible implementation of such tools. This study sought to evaluate 3 uses for generative artificial intelligence for clinical documentation in pediatric emergency medicine, measuring time savings, effort reduction, and physician attitudes and identifying potential risks and barriers. METHODS: This mixed-methods study was performed with 10 pediatric emergency medicine attending physicians from a single pediatric emergency department. Participants were asked to write a supervisory note for 4 clinical scenarios, with varying levels of complexity, twice without any assistance and twice with the assistance of ChatGPT Version 4.0. Participants evaluated 2 additional ChatGPT-generated clinical summaries: a structured handoff and a visit summary for a family written at an 8th grade reading level. Finally, a semistructured interview was performed to assess physicians' perspective on the use of ChatGPT in pediatric emergency medicine. Main outcomes and measures included between subjects' comparisons of the effort and time taken to complete the supervisory note with and without ChatGPT assistance. Effort was measured using a self-reported Likert scale of 0 to 10. Physicians' scoring of and attitude toward the ChatGPT-generated summaries were measured using a 0 to 10 Likert scale and open-ended questions. Summaries were scored for completeness, accuracy, efficiency, readability, and overall satisfaction. A thematic analysis was performed to analyze the content of the open-ended questions and to identify key themes. RESULTS: ChatGPT yielded a 40% reduction in time and a 33% decrease in effort for supervisory notes in intricate cases, with no discernible effect on simpler notes. ChatGPT-generated summaries for structured handoffs and family letters were highly rated, ranging from 7.0 to 9.0 out of 10, and most participants favored their inclusion in clinical practice. However, there were several critical reservations, out of which a set of general recommendations for applying ChatGPT to clinical summaries was formulated. CONCLUSION: Pediatric emergency medicine attendings in our study perceived that ChatGPT can deliver high-quality summaries while saving time and effort in many scenarios, but not all.

The value of parental medical records for the prediction of diabetes and cardiovascular disease: a novel method for generating and incorporating family histories.

OBJECTIVE: To determine whether data-driven family histories (DDFH) derived from linked EHRs of patients and their parents can improve prediction of patients' 10-year risk of diabetes and atherosclerotic cardiovascular disease (ASCVD). MATERIALS AND METHODS: A retrospective cohort study using data from Israel's largest healthcare organization. A random sample of 200 000 subjects aged 40-60 years on the index date (January 1, 2010) was included. Subjects with insufficient history (<1 year) or insufficient follow-up (<10 years) were excluded. Two separate XGBoost models were developed-1 for diabetes and 1 for ASCVD-to predict the 10-year risk for each outcome based on data available prior to the index date of January 1, 2010. RESULTS: Overall, the study included 110 734 subject-father-mother triplets. There were 22 153 cases of diabetes (20%) and 11 715 cases of ASCVD (10.6%). The addition of parental information significantly improved prediction of diabetes risk (P < .001), but not ASCVD risk. For both outcomes, maternal medical history was more predictive than paternal medical history. A binary variable summarizing parental disease state delivered similar predictive results to the full parental EHR. DISCUSSION: The increasing availability of EHRs for multiple family generations makes DDFH possible and can assist in delivering more personalized and precise medicine to patients. Consent frameworks must be established to enable sharing of information across generations, and the results suggest that sharing the full records may not be necessary. CONCLUSION: DDFH can address limitations of patient self-reported family history, and it improves clinical predictions for some conditions, but not for all, and particularly among younger adults.

An efficient landmark model for prediction of suicide attempts in multiple clinical settings.

Growing evidence has shown that applying machine learning models to large clinical data sources may exceed clinician performance in suicide risk stratification. However, many existing prediction models either suffer from "temporal bias" (a bias that stems from using case-control sampling) or require training on all available patient visit data. Here, we adopt a "landmark model" framework that aligns with clinical practice for prediction of suicide-related behaviors (SRBs) using a large electronic health record database. Using the landmark approach, we developed models for SRB prediction (regularized Cox regression and random survival forest) that establish a time-point (e.g., clinical visit) from which predictions are made over user-specified prediction windows using historical information up to that point. We applied this approach to cohorts from three clinical settings: general outpatient, psychiatric emergency department, and psychiatric inpatients, for varying prediction windows and lengths of historical data. Models achieved high discriminative performance (area under the Receiver Operating Characteristic curve 0.74-0.93 for the Cox model) across different prediction windows and settings, even with relatively short periods of historical data. In short, we developed accurate, dynamic SRB risk prediction models with the landmark approach that reduce bias and enhance the reliability and portability of suicide risk prediction models.

Improving risk prediction for target subpopulations: Predicting suicidal behaviors among multiple sclerosis patients.

Several recent studies have applied machine learning techniques to develop risk algorithms that predict subsequent suicidal behavior based on electronic health record data. In this study we used a retrospective cohort study design to test whether developing more tailored predictive models-within specific subpopulations of patients-would improve predictive accuracy. A retrospective cohort of 15,117 patients diagnosed with multiple sclerosis (MS), a diagnosis associated with increased risk of suicidal behavior, was used. The cohort was randomly divided into equal sized training and validation sets. Overall, suicidal behavior was identified among 191 (1.3%) of the patients with MS. A Naïve Bayes Classifier model was trained on the training set to predict future suicidal behavior. With 90% specificity, the model detected 37% of subjects who later demonstrated suicidal behavior, on average 4.6 years before the first suicide attempt. The performance of a model trained only on MS patients was better at predicting suicide in MS patients than that a model trained on a general patient sample of a similar size (AUC of 0.77 vs. 0.66). Unique risk factors for suicidal behavior among patients with MS included pain-related codes, gastroenteritis and colitis, and history of smoking. Future studies are needed to further test the value of developing population-specific risk models.

Estimated Travel Time and Spatial Access to Abortion Facilities in the US Before and After the Dobbs v Jackson Women's Health Decision.

Importance: Abortion facility closures resulted in a substantial decrease in access to abortion care in the US. Objectives: To investigate the changes in travel time to the nearest abortion facility after the Dobbs v Jackson Women's Health Organization (referred to hereafter as Dobbs) US Supreme Court decision. Design, Setting, and Participants: Repeated cross-sectional spatial analysis of travel time from each census tract in the contiguous US (n = 82 993) to the nearest abortion facility (n = 1134) listed in the Advancing New Standards in Reproductive Health database. Census tract boundaries and demographics were defined by the 2020 American Community Survey. The spatial analysis compared access during the pre-Dobbs period (January-December 2021) with the post-Dobbs period (September 2022) for the estimated 63 718 431 females aged 15 to 44 years (reproductive age for this analysis) in the US (excluding Alaska and Hawaii). Exposures: The Dobbs ruling and subsequent state laws restricting abortion procedures. The pre-Dobbs period measured abortion access to all facilities providing abortions in 2021. Post-Dobbs abortion access was measured by simulating the closure of all facilities in the 15 states with existing total or 6-week abortion bans in effect as of September 30, 2022. Main Outcomes and Measures: Median and mean changes in surface travel time (eg, car, public transportation) to an abortion facility in the post-Dobbs period compared with the pre-Dobbs period and the total percentage of females of reproductive age living more than 60 minutes from abortion facilities during the pre- and post-Dobbs periods. Results: Of 1134 abortion facilities in the US (at least 1 in every state; 8 in Alaska and Hawaii excluded), 749 were considered active during the pre-Dobbs period and 671 were considered active during a simulated post-Dobbs period. Median (IQR) and mean (SD) travel times to pre-Dobbs abortion facilities were estimated to be 10.9 (4.3-32.4) and 27.8 (42.0) minutes. Travel time to abortion facilities in the post-Dobbs period significantly increased (paired sample t test P <.001) to an estimated median (IQR) of 17.0 (4.9-124.5) minutes and a mean (SD) of and 100.4 (161.5) minutes. In the post-Dobbs period, an estimated 33.3% (sensitivity interval, 32.3%-34.8%) of females of reproductive age lived in a census tract more than 60 minutes from an abortion facility compared with 14.6.% (sensitivity interval, 13.0%-16.9%) of females of reproductive age in the pre-Dobbs period. Conclusions and Relevance: In this repeated cross-sectional spatial analysis, estimated travel time to abortion facilities in the US was significantly greater in the post-Dobbs period after accounting for the closure of abortion facilities in states with total or 6-week abortion bans compared with the pre-Dobbs period, during which all facilities providing abortions in 2021 were considered active.

Effectiveness of REGEN-COV antibody combination in preventing severe COVID-19 outcomes.

REGEN-COV, a combination of the monoclonal antibodies casirivimab and imdevimab, has been approved as a treatment for high-risk patients infected with SARS-CoV-2 within five days of their diagnosis. We performed a retrospective cohort study, and used data repositories of Israel's largest healthcare organization to determine the real-world effectiveness of REGEN-COV treatment against COVID-19-related hospitalization, severe disease, and death. We compared patients infected with Delta variant and treated with REGEN-COV (n = 289) to those infected but not-treated with REGEN-COV (n = 1,296). Demographic and clinical characteristics were used to match patients and for further adjustment as part of the C0x model. Estimated treatment effectiveness was defined as one minus the hazard ratio. Treatment effectiveness of REGEN-COV was 56.4% (95% CI: 23.7-75.1%) in preventing COVID-19 hospitalization, 59.2% (95% CI: 19.9-79.2%) in preventing severe COVID-19, and 93.5% (95% CI: 52.1-99.1%) in preventing COVID-19 death in the 28 days after treatment. In conclusion, REGEN-COV was effective in reducing the risk of severe sequelae in high-risk COVID-19 patients.

BNT162b2 Vaccine Effectiveness against Omicron in Children 5 to 11 Years of Age.

BACKGROUND: Limited evidence is available on the real-world effectiveness of the BNT162b2 vaccine against coronavirus disease 2019 (Covid-19) and specifically against infection with the omicron variant among children 5 to 11 years of age. METHODS: Using data from the largest health care organization in Israel, we identified a cohort of children 5 to 11 years of age who were vaccinated on or after November 23, 2021, and matched them with unvaccinated controls to estimate the vaccine effectiveness of BNT162b2 among newly vaccinated children during the omicron wave. Vaccine effectiveness against documented severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and symptomatic Covid-19 was estimated after the first and second vaccine doses. The cumulative incidence of each outcome in the two study groups through January 7, 2022, was estimated with the use of the Kaplan-Meier estimator, and vaccine effectiveness was calculated as 1 minus the risk ratio. Vaccine effectiveness was also estimated in age subgroups. RESULTS: Among 136,127 eligible children who had been vaccinated during the study period, 94,728 were matched with unvaccinated controls. The estimated vaccine effectiveness against documented infection was 17% (95% confidence interval [CI], 7 to 25) at 14 to 27 days after the first dose and 51% (95% CI, 39 to 61) at 7 to 21 days after the second dose. The absolute risk difference between the study groups at days 7 to 21 after the second dose was 1905 events per 100,000 persons (95% CI, 1294 to 2440) for documented infection and 599 events per 100,000 persons (95% CI, 296 to 897) for symptomatic Covid-19. The estimated vaccine effectiveness against symptomatic Covid-19 was 18% (95% CI, -2 to 34) at 14 to 27 days after the first dose and 48% (95% CI, 29 to 63) at 7 to 21 days after the second dose. We observed a trend toward higher vaccine effectiveness in the youngest age group (5 or 6 years of age) than in the oldest age group (10 or 11 years of age). CONCLUSIONS: Our findings suggest that as omicron was becoming the dominant variant, two doses of the BNT162b2 messenger RNA vaccine provided moderate protection against documented SARS-CoV-2 infection and symptomatic Covid-19 in children 5 to 11 years of age. (Funded by the European Union through the VERDI project and others.).

Comparing COVID-19-related hospitalization rates among individuals with infection-induced and vaccine-induced immunity in Israel.

With the COVID-19 pandemic ongoing, accurate assessment of population immunity and the effectiveness of booster and enhancer vaccine doses is critical. We compare COVID-19-related hospitalization incidence rates in 2,412,755 individuals across four exposure levels: non-recent vaccine immunity (two BNT162b2 COVID-19 vaccine doses five or more months prior), boosted vaccine immunity (three BNT162b2 doses), infection-induced immunity (previous COVID-19 without a subsequent BNT162b2 dose), and enhanced infection-induced immunity (previous COVID-19 with a subsequent BNT162b2 dose). Rates, adjusted for potential demographic, clinical and health-seeking-behavior confounders, were assessed from July-November 2021 when the Delta variant was predominant. Compared with non-recent vaccine immunity, COVID-19-related hospitalization incidence rates were reduced by 89% (87-91%) for boosted vaccine immunity, 66% (50-77%) for infection-induced immunity and 75% (61-83%) for enhanced infection-induced immunity. We demonstrate that infection-induced immunity (enhanced or not) provides more protection against COVID-19-related hospitalization than non-recent vaccine immunity, but less protection than booster vaccination. Additionally, our results suggest that vaccinating individuals with infection-induced immunity further enhances their protection.

Fourth Dose of BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting.

BACKGROUND: With large waves of infection driven by the B.1.1.529 (omicron) variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), alongside evidence of waning immunity after the booster dose of coronavirus disease 2019 (Covid-19) vaccine, several countries have begun giving at-risk persons a fourth vaccine dose. METHODS: To evaluate the early effectiveness of a fourth dose of the BNT162b2 vaccine for the prevention of Covid-19-related outcomes, we analyzed data recorded by the largest health care organization in Israel from January 3 to February 18, 2022. We evaluated the relative effectiveness of a fourth vaccine dose as compared with that of a third dose given at least 4 months earlier among persons 60 years of age or older. We compared outcomes in persons who had received a fourth dose with those in persons who had not, individually matching persons from these two groups with respect to multiple sociodemographic and clinical variables. A sensitivity analysis was performed with the use of parametric Poisson regression. RESULTS: The primary analysis included 182,122 matched pairs. Relative vaccine effectiveness in days 7 to 30 after the fourth dose was estimated to be 45% (95% confidence interval [CI], 44 to 47) against polymerase-chain-reaction-confirmed SARS-CoV-2 infection, 55% (95% CI, 53 to 58) against symptomatic Covid-19, 68% (95% CI, 59 to 74) against Covid-19-related hospitalization, 62% (95% CI, 50 to 74) against severe Covid-19, and 74% (95% CI, 50 to 90) against Covid-19-related death. The corresponding estimates in days 14 to 30 after the fourth dose were 52% (95% CI, 49 to 54), 61% (95% CI, 58 to 64), 72% (95% CI, 63 to 79), 64% (95% CI, 48 to 77), and 76% (95% CI, 48 to 91). In days 7 to 30 after a fourth vaccine dose, the difference in the absolute risk (three doses vs. four doses) was 180.1 cases per 100,000 persons (95% CI, 142.8 to 211.9) for Covid-19-related hospitalization and 68.8 cases per 100,000 persons (95% CI, 48.5 to 91.9) for severe Covid-19. In sensitivity analyses, estimates of relative effectiveness against documented infection were similar to those in the primary analysis. CONCLUSIONS: A fourth dose of the BNT162b2 vaccine was effective in reducing the short-term risk of Covid-19-related outcomes among persons who had received a third dose at least 4 months earlier. (Funded by the Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute.).

Implementing Machine Learning Models for Suicide Risk Prediction in Clinical Practice: Focus Group Study With Hospital Providers.

BACKGROUND: Interest in developing machine learning models that use electronic health record data to predict patients' risk of suicidal behavior has recently proliferated. However, whether and how such models might be implemented and useful in clinical practice remain unknown. To ultimately make automated suicide risk-prediction models useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders, including the frontline providers who will be using such tools, at each stage of the implementation process. OBJECTIVE: The aim of this focus group study is to inform ongoing and future efforts to deploy suicide risk-prediction models in clinical practice. The specific goals are to better understand hospital providers' current practices for assessing and managing suicide risk; determine providers' perspectives on using automated suicide risk-prediction models in practice; and identify barriers, facilitators, recommendations, and factors to consider. METHODS: We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by 2 independent study staff members. All coded text was reviewed and discrepancies were resolved in consensus meetings with doctoral-level staff. RESULTS: Although most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers' general attitudes toward the practical use of automated suicide risk-prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the health care system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider training. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings. CONCLUSIONS: Providers were dissatisfied with current suicide risk assessment methods and were open to the use of a machine learning-based risk-prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of these new approaches in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.

An integrated framework for identifying clinical-laboratory indicators for novel pandemics: COVID-19 and MIS-C.

During the critical early stages of an emerging pandemic, limited availability of pathogen-specific testing can severely inhibit individualized risk screening and pandemic tracking. Standard clinical laboratory tests offer a widely available complementary data source for first-line risk screening and pandemic surveillance. Here, we propose an integrated framework for developing clinical-laboratory indicators for novel pandemics that combines population-level and individual-level analyses. We apply this framework to 7,520,834 clinical laboratory tests recorded over five years and find clinical-lab-test combinations that are strongly associated with SARS-CoV-2 PCR test results and Multisystem Inflammatory Syndrome in Children (MIS-C) diagnoses: Interleukin-related tests (e.g. IL4, IL10) were most strongly associated with SARS-CoV-2 infection and MIS-C, while other more widely available tests (ferritin, D-dimer, fibrinogen, alanine transaminase, and C-reactive protein) also had strong associations. When novel pandemics emerge, this framework can be used to identify specific combinations of clinical laboratory tests for public health tracking and first-line individualized risk screening.

Indirect protection of children from SARS-CoV-2 infection through parental vaccination.

Children not vaccinated against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) may still benefit from vaccines through protection from vaccinated contacts. We estimated the protection provided to children through parental vaccination with the BNT162b2 vaccine. We studied households without prior infection consisting of two parents and unvaccinated children, estimating the effect of parental vaccination on the risk of infection for unvaccinated children. We studied two periods separately-an early period (17 January 2021 to 28 March 2021; Alpha variant, two doses versus no vaccination) and a late period (11 July 2021 to 30 September 2021; Delta variant, booster dose versus two vaccine doses). We found that having a single vaccinated parent was associated with a 26.0 and a 20.8% decreased risk in the early and late periods, respectively, and having two vaccinated parents was associated with a 71.7 and a 58.1% decreased risk, respectively. Thus, parental vaccination confers substantial protection on unvaccinated children in the household.

Predictive structured-unstructured interactions in EHR models: A case study of suicide prediction.

Clinical risk prediction models powered by electronic health records (EHRs) are becoming increasingly widespread in clinical practice. With suicide-related mortality rates rising in recent years, it is becoming increasingly urgent to understand, predict, and prevent suicidal behavior. Here, we compare the predictive value of structured and unstructured EHR data for predicting suicide risk. We find that Naive Bayes Classifier (NBC) and Random Forest (RF) models trained on structured EHR data perform better than those based on unstructured EHR data. An NBC model trained on both structured and unstructured data yields similar performance (AUC = 0.743) to an NBC model trained on structured data alone (0.742, p = 0.668), while an RF model trained on both data types yields significantly better results (AUC = 0.903) than an RF model trained on structured data alone (0.887, p < 0.001), likely due to the RF model's ability to capture interactions between the two data types. To investigate these interactions, we propose and implement a general framework for identifying specific structured-unstructured feature pairs whose interactions differ between case and non-case cohorts, and thus have the potential to improve predictive performance and increase understanding of clinical risk. We find that such feature pairs tend to capture heterogeneous pairs of general concepts, rather than homogeneous pairs of specific concepts. These findings and this framework can be used to improve current and future EHR-based clinical modeling efforts.

The language of crisis: spatiotemporal effects of COVID-19 pandemic dynamics on health crisis communications by political leaders.

In times of crisis, communication by leaders is essential for mobilizing an effective public response. During the COVID-19 pandemic, compliance with public health guidelines has been critical for the prevention of infections and deaths. We assembled a corpus of over 1500 pandemic-related speeches, containing over 4 million words, delivered by all 50 US state governors during the initial months of the COVID-19 pandemic. We analyzed the semantic, grammatical and linguistic-complexity properties of these speeches, and examined their relationships to COVID-19 case rates over space and time. We found that as COVID-19 cases rose, governors used stricter language to issue guidance, employed greater negation to defend their actions and highlight prevailing uncertainty, and used more extreme descriptive adjectives. As cases surged to their highest levels, governors used shorter words with fewer syllables. Investigating and understanding such characteristic responses to stress is important for improving effective public communication during major health crises.

Prediction across healthcare settings: a case study in predicting emergency department disposition.

Several approaches exist today for developing predictive models across multiple clinical sites, yet there is a lack of comparative data on their performance, especially within the context of EHR-based prediction models. We set out to provide a framework for prediction across healthcare settings. As a case study, we examined an ED disposition prediction model across three geographically and demographically diverse sites. We conducted a 1-year retrospective study, including all visits in which the outcome was either discharge-to-home or hospitalization. Four modeling approaches were compared: a ready-made model trained at one site and validated at other sites, a centralized uniform model incorporating data from all sites, multiple site-specific models, and a hybrid approach of a ready-made model re-calibrated using site-specific data. Predictions were performed using XGBoost. The study included 288,962 visits with an overall admission rate of 16.8% (7.9-26.9%). Some risk factors for admission were prominent across all sites (e.g., high-acuity triage emergency severity index score, high prior admissions rate), while others were prominent at only some sites (multiple lab tests ordered at the pediatric sites, early use of ECG at the adult site). The XGBoost model achieved its best performance using the uniform and site-specific approaches (AUC = 0.9-0.93), followed by the calibrated-model approach (AUC = 0.87-0.92), and the ready-made approach (AUC = 0.62-0.85). Our results show that site-specific customization is a key driver of predictive model performance.

Effectiveness of a third dose of the BNT162b2 mRNA COVID-19 vaccine for preventing severe outcomes in Israel: an observational study.

BACKGROUND: Many countries are experiencing a resurgence of COVID-19, driven predominantly by the delta (B.1.617.2) variant of SARS-CoV-2. In response, these countries are considering the administration of a third dose of mRNA COVID-19 vaccine as a booster dose to address potential waning immunity over time and reduced effectiveness against the delta variant. We aimed to use the data repositories of Israel's largest health-care organisation to evaluate the effectiveness of a third dose of the BNT162b2 mRNA vaccine for preventing severe COVID-19 outcomes. METHODS: Using data from Clalit Health Services, which provides mandatory health-care coverage for over half of the Israeli population, individuals receiving a third vaccine dose between July 30, 2020, and Sept 23, 2021, were matched (1:1) to demographically and clinically similar controls who did not receive a third dose. Eligible participants had received the second vaccine dose at least 5 months before the recruitment date, had no previous documented SARS-CoV-2 infection, and had no contact with the health-care system in the 3 days before recruitment. Individuals who are health-care workers, live in long-term care facilities, or are medically confined to their homes were excluded. Primary outcomes were COVID-19-related admission to hospital, severe disease, and COVID-19-related death. The third dose effectiveness for each outcome was estimated as 1 - risk ratio using the Kaplan-Meier estimator. FINDINGS: 1 158 269 individuals were eligible to be included in the third dose group. Following matching, the third dose and control groups each included 728 321 individuals. Participants had a median age of 52 years (IQR 37-68) and 51% were female. The median follow-up time was 13 days (IQR 6-21) in both groups. Vaccine effectiveness evaluated at least 7 days after receipt of the third dose, compared with receiving only two doses at least 5 months ago, was estimated to be 93% (231 events for two doses vs 29 events for three doses; 95% CI 88-97) for admission to hospital, 92% (157 vs 17 events; 82-97) for severe disease, and 81% (44 vs seven events; 59-97) for COVID-19-related death. INTERPRETATION: Our findings suggest that a third dose of the BNT162b2 mRNA vaccine is effective in protecting individuals against severe COVID-19-related outcomes, compared with receiving only two doses at least 5 months ago. FUNDING: The Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute.

Temporally informed random forests for suicide risk prediction.

OBJECTIVE: Suicide is one of the leading causes of death worldwide, yet clinicians find it difficult to reliably identify individuals at high risk for suicide. Algorithmic approaches for suicide risk detection have been developed in recent years, mostly based on data from electronic health records (EHRs). Significant room for improvement remains in the way these models take advantage of temporal information to improve predictions. MATERIALS AND METHODS: We propose a temporally enhanced variant of the random forest (RF) model-Omni-Temporal Balanced Random Forests (OT-BRFs)-that incorporates temporal information in every tree within the forest. We develop and validate this model using longitudinal EHRs and clinician notes from the Mass General Brigham Health System recorded between 1998 and 2018, and compare its performance to a baseline Naive Bayes Classifier and 2 standard versions of balanced RFs. RESULTS: Temporal variables were found to be associated with suicide risk: Elevated suicide risk was observed in individuals with a higher total number of visits as well as those with a low rate of visits over time, while lower suicide risk was observed in individuals with a longer period of EHR coverage. RF models were more accurate than Naive Bayesian classifiers at predicting suicide risk in advance (area under the receiver operating curve = 0.824 vs. 0.754, respectively). The proposed OT-BRF model performed best among all RF approaches, yielding a sensitivity of 0.339 at 95% specificity, compared to 0.290 and 0.286 for the other 2 RF models. Temporal variables were assigned high importance by the models that incorporated them. DISCUSSION: We demonstrate that temporal variables have an important role to play in suicide risk detection and that requiring their inclusion in all RF trees leads to increased predictive performance. Integrating temporal information into risk prediction models helps the models interpret patient data in temporal context, improving predictive performance.