The infant health effects of doulas: Leveraging big data and machine learning to inform cost-effective targeting.

Doula services represent an underutilized maternal and child health intervention with the potential to improve outcomes through the provision of physical, emotional, and informational support. However, there is limited evidence of the infant health effects of doulas despite well-established connections between maternal and infant health. Moreover, because the availability of doulas is limited and often not covered by insurers, existing evidence leaves unclear if or how doula services should be allocated to achieve the greatest improvements in outcomes. We use unique data and machine learning to develop accurate predictive models of infant health and doula service participation. We then combine these predictive models within the double machine learning method to estimate the effects of doula services. We show that while doula services reduce risk on average, the benefits of doula services increase as the risk of negative infant health outcomes increases. We compare these benefits to the costs of doula services under alternative allocation schemes and show that leveraging the risk predictions dramatically increases the cost effectiveness of doula services. Our results show the potential of big data and novel analytic methods to provide cost-effective support to those at greatest risk of poor outcomes.

Variation in the infant health effects of the women, infants, and children program by predicted risk using novel machine learning methods.

The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) has an extensive literature documenting positive effects on infant health outcomes, specifically preterm birth, low birthweight, small size for gestational age, and infant mortality. However, existing studies focus on average effects for these relatively infrequent outcomes, thus providing no evidence for how WIC affects those at greatest risk of negative infant health outcomes. Our study focuses on documenting how WIC's infant health effects vary by level of risk. In doing so, we leverage a uniquely rich database describing maternal and infant outcomes and risk factors. Additionally, we use high dimensional data to generate predictions of risk and combine these predictions with the novel double machine learning method to stratify the effects of WIC by predicted risk. Our estimates of WIC's average treatment effects align with those in the existing literature. More importantly, we document significant variation in the effects of WIC on infant health by predicted risk level. Our results show that WIC is most beneficial among those at greatest risk of poor outcomes.

Reducing the environmental impact of mask inductions in children: A quality improvement report.

BACKGROUND: Inhalational anesthetic agents are potent greenhouse gases with global warming potential that far exceed that of carbon dioxide. Traditionally, pediatric inhalation inductions are achieved with a volatile anesthetic delivered to the patient in oxygen and nitrous oxide at high fresh gas flows. While contemporary volatile anesthetics and anesthesia machines allow for a more environmentally conscious induction, practice has not changed. We aimed to reduce the environmental impact of our inhalation inductions by decreasing the use of nitrous oxide and fresh gas flows. METHODS: Through a series of four plan-do-study-act cycles, the improvement team used content experts to demonstrate the environmental impact of the current inductions and to provide practical ways to reduce this, by focusing on nitrous oxide use and fresh gas flows, with visual reminders introduced at point of delivery. The primary measures were the percentage of inhalation inductions that used nitrous oxide and the maximum fresh gas flows/kg during the induction period. Statistical process control charts were used to measure improvement over time. RESULTS: 33 285 inhalation inductions were included over a 20-month period. nitrous oxide use decreased from 80% to <20% and maximum fresh gas flows/kg decreased from a rate of 0.53 L/min/kg to 0.38 L/min/kg, an overall reduction of 28%. Reduction in fresh gas flows was greatest in the lightest weight groups. Induction times and behaviors remained unchanged over the duration of this project. CONCLUSIONS: Our quality improvement group decreased the environmental impact of inhalation inductions and created cultural change within our department to sustain change and foster the pursuit of future environmental efforts.

Comparing different wavelet transforms on removing electrocardiogram baseline wanders and special trends.

BACKGROUND: Electrocardiogram (ECG) signal, an important indicator for heart problems, is commonly corrupted by a low-frequency baseline wander (BW) artifact, which may cause interpretation difficulty or inaccurate analysis. Unlike current state-of-the-art approach using band-pass filters, wavelet transforms can accurately capture both time and frequency information of a signal. However, extant literature is limited in applying wavelet transforms (WTs) for baseline wander removal. In this study, we aimed to evaluate 5 wavelet families with a total of 14 wavelets for removing ECG baseline wanders from a semi-synthetic dataset. METHODS: We created a semi-synthetic ECG dataset based on a public QT Database on Physionet repository with ECG data from 105 patients. The semi-synthetic ECG dataset comprised ECG excerpts from the QT database superimposed with artificial baseline wanders. We extracted one ECG excerpt from each of 105 patients, and the ECG excerpt comprised 14 s of randomly selected ECG data. Twelve baseline wanders were manually generated, including sinusoidal waves, spikes and step functions. We implemented and evaluated 14 commonly used wavelets up to 12 WT levels. The evaluation metric was mean-square-error (MSE) between the original ECG excerpt and the processed signal with artificial BW removed. RESULTS: Among the 14 wavelets, Daubechies-3 wavelet and Symlets-3 wavelet with 7 levels of WT had best performance, MSE = 0.0044. The average MSEs for sinusoidal waves, step, and spike functions were 0.0271, 0.0304, 0.0199 respectively. For artificial baseline wanders with spikes or step functions, wavelet transforms in general had lower performance in removing the BW; however, WTs accurately located the temporal position of an impulse edge. CONCLUSIONS: We found wavelet transforms in general accurately removed various baseline wanders. Daubechies-3 and Symlets-3 wavelets performed best. The study could facilitate future real-time processing of streaming ECG signals for clinical decision support systems.

Predictive modeling for classification of positive valence system symptom severity from initial psychiatric evaluation records.

In response to the challenges set forth by the CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing, we describe a framework to automatically classify initial psychiatric evaluation records to one of four positive valence system severities: absent, mild, moderate, or severe. We used a dataset provided by the event organizers to develop a framework comprised of natural language processing (NLP) modules and 3 predictive models (two decision tree models and one Bayesian network model) used in the competition. We also developed two additional predictive models for comparison purpose. To evaluate our framework, we employed a blind test dataset provided by the 2016 CEGS N-GRID. The predictive scores, measured by the macro averaged-inverse normalized mean absolute error score, from the two decision trees and Naïve Bayes models were 82.56%, 82.18%, and 80.56%, respectively. The proposed framework in this paper can potentially be applied to other predictive tasks for processing initial psychiatric evaluation records, such as predicting 30-day psychiatric readmissions.

Comparison of machine learning classifiers for influenza detection from emergency department free-text reports.

Influenza is a yearly recurrent disease that has the potential to become a pandemic. An effective biosurveillance system is required for early detection of the disease. In our previous studies, we have shown that electronic Emergency Department (ED) free-text reports can be of value to improve influenza detection in real time. This paper studies seven machine learning (ML) classifiers for influenza detection, compares their diagnostic capabilities against an expert-built influenza Bayesian classifier, and evaluates different ways of handling missing clinical information from the free-text reports. We identified 31,268 ED reports from 4 hospitals between 2008 and 2011 to form two different datasets: training (468 cases, 29,004 controls), and test (176 cases and 1620 controls). We employed Topaz, a natural language processing (NLP) tool, to extract influenza-related findings and to encode them into one of three values: Acute, Non-acute, and Missing. Results show that all ML classifiers had areas under ROCs (AUC) ranging from 0.88 to 0.93, and performed significantly better than the expert-built Bayesian model. Missing clinical information marked as a value of missing (not missing at random) had a consistently improved performance among 3 (out of 4) ML classifiers when it was compared with the configuration of not assigning a value of missing (missing completely at random). The case/control ratios did not affect the classification performance given the large number of training cases. Our study demonstrates ED reports in conjunction with the use of ML and NLP with the handling of missing value information have a great potential for the detection of infectious diseases.

Electroencephalogram-based machine learning models to predict neurologic outcome after cardiac arrest: A systematic review.

AIM OF THE REVIEW: The primary aim of this systematic review was to investigate the most common electroencephalogram (EEG)-based machine learning (ML) model with the highest Area Under Receiver Operating Characteristic Curve (AUC) in two ML categories, conventional ML and Deep Neural Network (DNN), to predict the neurologic outcomes after cardiac arrest; the secondary aim was to investigate common EEG features applied to ML models. METHODS: Systematic search of medical literature from PubMed and engineering literature from Compendex up to June 2, 2023. One reviewer screened studies that used EEG-based ML models to predict the neurologic outcomes after cardiac arrest. Four reviewers validated that the studies met selection criteria. Nine variables were manually extracted. The top-five common EEG features were calculated. We evaluated each study's risk of bias using the Quality in Prognosis Studies guideline. RESULTS: Out of 351 identified studies, 17 studies met the inclusion criteria. Random Forest (RF) (n = 7) was the most common ML model in the conventional ML category (n = 11), followed by Convolutional Neural Network (CNN) (n = 4) in the DNN category (n = 6). The AUCs for RF ranged between 0.8 and 0.97, while CNN had AUCs between 0.7 and 0.92. The top-three commonly used EEG features were band power (n = 12), Shannon's Entropy (n = 11), burst-suppression ratio (n = 9). CONCLUSIONS: RF and CNN were the two most common ML models with the highest AUCs for predicting the neurologic outcomes after cardiac arrest. Using a multimodal model that combines EEG features and electronic health record data may further improve prognostic performance.

Predictive Validity of the K-CAT-SS in High-Risk Adolescents and Young Adults.

OBJECTIVE: Suicide is a leading cause of death in adolescents and young adults and has increased substantially in the past 15 years. Accurate suicide risk stratification based on rapid screening can help reverse these trends. This study aimed to assess the ability of the Kiddie Computerized Adaptive Test Suicide Scale (K-CAT-SS), a brief computerized adaptive test of suicidality, to predict suicide attempts (SAs) in high-risk youth. METHOD: A total of 652 participants (age range, 12-24 years), 78% of whom presented with suicidal ideation or behavior, were recruited within 1 month of mental health care contact. The K-CAT-SS, scaled from 0 to 100, was administered at baseline, and participants were assessed at about 1, 3, and 6 months after intake. Weekly incidence of SAs was assessed using the Adolescent Longitudinal Interval Follow-up Evaluation and Columbia-Suicide Severity Rating Scale. A secondary outcome was suicidal behavior, including aborted, interrupted, and actual SAs. RESULTS: The K-CAT-SS showed a 4.91-fold increase in SAs for every 25-point increase in the baseline score (95% CI 2.83-8.52) and a 3.51-fold increase in suicidal behaviors (95% CI 2.32-5.30). These relations persisted following adjustment for prior attempts; demographic variables including age, sex, gender identity, sexual orientation, and race/ethnicity; and other measures of psychopathology. No moderating effects were identified. At 3 months, area under the receiver operating characteristic curve was 0.83 (95% CI 0.72-0.93) for 1 or more SAs. CONCLUSION: The K-CAT-SS is an excellent tool for suicide risk stratification, particularly in higher-risk populations where other measures have shown lower predictive validity.

Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition.

OBJECTIVE: Social determinants of health (SDOH) are nonclinical, socioeconomic conditions that influence patient health and quality of life. Identifying SDOH may help clinicians target interventions. However, SDOH are more frequently available in narrative notes compared to structured electronic health records. The 2022 n2c2 Track 2 competition released clinical notes annotated for SDOH to promote development of NLP systems for extracting SDOH. We developed a system addressing 3 limitations in state-of-the-art SDOH extraction: the inability to identify multiple SDOH events of the same type per sentence, overlapping SDOH attributes within text spans, and SDOH spanning multiple sentences. MATERIALS AND METHODS: We developed and evaluated a 2-stage architecture. In stage 1, we trained a BioClinical-BERT-based named entity recognition system to extract SDOH event triggers, that is, text spans indicating substance use, employment, or living status. In stage 2, we trained a multitask, multilabel NER to extract arguments (eg, alcohol "type") for events extracted in stage 1. Evaluation was performed across 3 subtasks differing by provenance of training and validation data using precision, recall, and F1 scores. RESULTS: When trained and validated on data from the same site, we achieved 0.87 precision, 0.89 recall, and 0.88 F1. Across all subtasks, we ranked between second and fourth place in the competition and always within 0.02 F1 from first. CONCLUSIONS: Our 2-stage, deep-learning-based NLP system effectively extracted SDOH events from clinical notes. This was achieved with a novel classification framework that leveraged simpler architectures compared to state-of-the-art systems. Improved SDOH extraction may help clinicians improve health outcomes.

Addressing Health-Related Social Needs and Mental Health Needs in the Neonatal Intensive Care Unit: Exploring Challenges and the Potential of Technology.

Unaddressed health-related social needs (HRSNs) and parental mental health needs in an infant's environment can negatively affect their health outcomes. This study examines the challenges and potential technological solutions for addressing these needs in the neonatal intensive care unit (NICU) setting and beyond. In all, 22 semistructured interviews were conducted with members of the NICU care team and other relevant stakeholders, based on an interpretive description approach. The participants were selected from three safety net hospitals in the U.S. with level IV NICUs. The challenges identified include navigating the multitude of burdens families in the NICU experience, resource constraints within and beyond the health system, a lack of streamlined or consistent processes, no closed-loop referrals to track status and outcomes, and gaps in support postdischarge. Opportunities for leveraging technology to facilitate screening and referral include automating screening, initiating risk-based referrals, using remote check-ins, facilitating resource navigation, tracking referrals, and providing language support. However, technological implementations should avoid perpetuating disparities and consider potential privacy or data-sharing concerns. Although advances in technological health tools alone cannot address all the challenges, they have the potential to offer dynamic tools to support the healthcare setting in identifying and addressing the unique needs and circumstances of each family in the NICU.

Predicting pediatric emergence delirium using data-driven machine learning applied to electronic health record dataset at a quaternary care pediatric hospital.

Objectives: Pediatric emergence delirium is an undesirable outcome that is understudied. Development of a predictive model is an initial step toward reducing its occurrence. This study aimed to apply machine learning (ML) methods to a large clinical dataset to develop a predictive model for pediatric emergence delirium. Materials and Methods: We performed a single-center retrospective cohort study using electronic health record data from February 2015 to December 2019. We built and evaluated 4 commonly used ML models for predicting emergence delirium: least absolute shrinkage and selection operator, ridge regression, random forest, and extreme gradient boosting. The primary outcome was the occurrence of emergence delirium, defined as a Watcha score of 3 or 4 recorded at any time during recovery. Results: The dataset included 54 776 encounters across 43 830 patients. The 4 ML models performed similarly with performance assessed by the area under the receiver operating characteristic curves ranging from 0.74 to 0.75. Notable variables associated with increased risk included adenoidectomy with or without tonsillectomy, decreasing age, midazolam premedication, and ondansetron administration, while intravenous induction and ketorolac were associated with reduced risk of emergence delirium. Conclusions: Four different ML models demonstrated similar performance in predicting postoperative emergence delirium using a large pediatric dataset. The prediction performance of the models draws attention to our incomplete understanding of this phenomenon based on the studied variables. The results from our modeling could serve as a first step in designing a predictive clinical decision support system, but further optimization and validation are needed. Clinical trial number and registry URL: Not applicable.

Early prediction of clinical deterioration using data-driven machine-learning modeling of electronic health records.

OBJECTIVES: To develop and evaluate a high-dimensional, data-driven model to identify patients at high risk of clinical deterioration from routinely collected electronic health record (EHR) data. MATERIALS AND METHODS: In this single-center, retrospective cohort study, 488 patients with single-ventricle and shunt-dependent congenital heart disease <6 months old were admitted to the cardiac intensive care unit before stage 2 palliation between 2014 and 2019. Using machine-learning techniques, we developed the Intensive care Warning Index (I-WIN), which systematically assessed 1028 regularly collected EHR variables (vital signs, medications, laboratory tests, and diagnoses) to identify patients in the cardiac intensive care unit at elevated risk of clinical deterioration. An ensemble of 5 extreme gradient boosting models was developed and validated on 203 cases (130 emergent endotracheal intubations, 34 cardiac arrests requiring cardiopulmonary resuscitation, 10 extracorporeal membrane oxygenation cannulations, and 29 cardiac arrests requiring cardiopulmonary resuscitation onto extracorporeal membrane oxygenation) and 378 control periods from 446 patients. RESULTS: At 4 hours before deterioration, the model achieved an area under the receiver operating characteristic curve of 0.92 (95% confidence interval, 0.84-0.98), 0.881 sensitivity, 0.776 positive predictive value, 0.862 specificity, and 0.571 Brier skill score. Performance remained high at 8 hours before deterioration with 0.815 (0.688-0.921) area under the receiver operating characteristic curve. CONCLUSIONS: I-WIN accurately predicted deterioration events in critically-ill infants with high-risk congenital heart disease up to 8 hours before deterioration, potentially allowing clinicians to target interventions. We propose a paradigm shift from conventional expert consensus-based selection of risk factors to a data-driven, machine-learning methodology for risk prediction. With the increased availability of data capture in EHRs, I-WIN can be extended to broader applications in data-rich environments in critical care.

An argument for reporting data standardization procedures in multi-site predictive modeling: case study on the impact of LOINC standardization on model performance.

OBJECTIVES: We aimed to gain a better understanding of how standardization of laboratory data can impact predictive model performance in multi-site datasets. We hypothesized that standardizing local laboratory codes to logical observation identifiers names and codes (LOINC) would produce predictive models that significantly outperform those learned utilizing local laboratory codes. MATERIALS AND METHODS: We predicted 30-day hospital readmission for a set of heart failure-specific visits to 13 hospitals from 2008 to 2012. Laboratory test results were extracted and then manually cleaned and mapped to LOINC. We extracted features to summarize laboratory data for each patient and used a training dataset (2008-2011) to learn models using a variety of feature selection techniques and classifiers. We evaluated our hypothesis by comparing model performance on an independent test dataset (2012). RESULTS: Models that utilized LOINC performed significantly better than models that utilized local laboratory test codes, regardless of the feature selection technique and classifier approach used. DISCUSSION AND CONCLUSION: We quantitatively demonstrated the positive impact of standardizing multi-site laboratory data to LOINC prior to use in predictive models. We used our findings to argue for the need for detailed reporting of data standardization procedures in predictive modeling, especially in studies leveraging multi-site datasets extracted from electronic health records.

The effects of natural language processing on cross-institutional portability of influenza case detection for disease surveillance.

OBJECTIVES: This study evaluates the accuracy and portability of a natural language processing (NLP) tool for extracting clinical findings of influenza from clinical notes across two large healthcare systems. Effectiveness is evaluated on how well NLP supports downstream influenza case-detection for disease surveillance. METHODS: We independently developed two NLP parsers, one at Intermountain Healthcare (IH) in Utah and the other at University of Pittsburgh Medical Center (UPMC) using local clinical notes from emergency department (ED) encounters of influenza. We measured NLP parser performance for the presence and absence of 70 clinical findings indicative of influenza. We then developed Bayesian network models from NLP processed reports and tested their ability to discriminate among cases of (1) influenza, (2) non-influenza influenza-like illness (NI-ILI), and (3) 'other' diagnosis. RESULTS: On Intermountain Healthcare reports, recall and precision of the IH NLP parser were 0.71 and 0.75, respectively, and UPMC NLP parser, 0.67 and 0.79. On University of Pittsburgh Medical Center reports, recall and precision of the UPMC NLP parser were 0.73 and 0.80, respectively, and IH NLP parser, 0.53 and 0.80. Bayesian case-detection performance measured by AUROC for influenza versus non-influenza on Intermountain Healthcare cases was 0.93 (using IH NLP parser) and 0.93 (using UPMC NLP parser). Case-detection on University of Pittsburgh Medical Center cases was 0.95 (using UPMC NLP parser) and 0.83 (using IH NLP parser). For influenza versus NI-ILI on Intermountain Healthcare cases performance was 0.70 (using IH NLP parser) and 0.76 (using UPMC NLP parser). On University of Pisstburgh Medical Center cases, 0.76 (using UPMC NLP parser) and 0.65 (using IH NLP parser). CONCLUSION: In all but one instance (influenza versus NI-ILI using IH cases), local parsers were more effective at supporting case-detection although performances of non-local parsers were reasonable.

Influenza detection from emergency department reports using natural language processing and Bayesian network classifiers.

OBJECTIVES: To evaluate factors affecting performance of influenza detection, including accuracy of natural language processing (NLP), discriminative ability of Bayesian network (BN) classifiers, and feature selection. METHODS: We derived a testing dataset of 124 influenza patients and 87 non-influenza (shigellosis) patients. To assess NLP finding-extraction performance, we measured the overall accuracy, recall, and precision of Topaz and MedLEE parsers for 31 influenza-related findings against a reference standard established by three physician reviewers. To elucidate the relative contribution of NLP and BN classifier to classification performance, we compared the discriminative ability of nine combinations of finding-extraction methods (expert, Topaz, and MedLEE) and classifiers (one human-parameterized BN and two machine-parameterized BNs). To assess the effects of feature selection, we conducted secondary analyses of discriminative ability using the most influential findings defined by their likelihood ratios. RESULTS: The overall accuracy of Topaz was significantly better than MedLEE (with post-processing) (0.78 vs 0.71, p<0.0001). Classifiers using human-annotated findings were superior to classifiers using Topaz/MedLEE-extracted findings (average area under the receiver operating characteristic (AUROC): 0.75 vs 0.68, p=0.0113), and machine-parameterized classifiers were superior to the human-parameterized classifier (average AUROC: 0.73 vs 0.66, p=0.0059). The classifiers using the 17 'most influential' findings were more accurate than classifiers using all 31 subject-matter expert-identified findings (average AUROC: 0.76>0.70, p<0.05). CONCLUSIONS: Using a three-component evaluation method we demonstrated how one could elucidate the relative contributions of components under an integrated framework. To improve classification performance, this study encourages researchers to improve NLP accuracy, use a machine-parameterized classifier, and apply feature selection methods.

PaTH: towards a learning health system in the Mid-Atlantic region.

The PaTH (University of Pittsburgh/UPMC, Penn State College of Medicine, Temple University Hospital, and Johns Hopkins University) clinical data research network initiative is a collaborative effort among four academic health centers in the Mid-Atlantic region. PaTH will provide robust infrastructure to conduct research, explore clinical outcomes, link with biospecimens, and improve methods for sharing and analyzing data across our diverse populations. Our disease foci are idiopathic pulmonary fibrosis, atrial fibrillation, and obesity. The four network sites have extensive experience in using data from electronic health records and have devised robust methods for patient outreach and recruitment. The network will adopt best practices by using the open-source data-sharing tool, Informatics for Integrating Biology and the Bedside (i2b2), at each site to enhance data sharing using centrally defined common data elements, and will use the Shared Health Research Information Network (SHRINE) for distributed queries across the network.