Automated Calculator for the Pediatric Sequential Organ Failure Assessment Score: Development and External Validation in a Single-Center 7-Year Cohort, 2015-2021.

OBJECTIVES: The pediatric Sequential Organ Failure Assessment (pSOFA) score summarizes severity of organ dysfunction and can be used to predict in-hospital mortality. Manual calculation of the pSOFA score is time-consuming and prone to human error. An automated method that is open-source, flexible, and scalable for calculating the pSOFA score directly from electronic health record data is desirable. DESIGN: Single-center, retrospective cohort study. SETTING: Quaternary 40-bed PICU. PATIENTS: All patients admitted to the PICU between 2015 and 2021 with ICU stay of at least 24 hours. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We used 77 records to evaluate the automated score. The automated algorithm had an overall accuracy of 97%. The algorithm calculated the respiratory component of two cases incorrectly. An expert human annotator had an initial accuracy of 75% at the patient level and 95% at the component level. An untrained human annotator with general clinical research experience had an overall accuracy of 16% and component-wise accuracy of 67%. Weighted kappa for agreement between the automated method and the expert annotator's initial score was 0.92 (95% CI, 0.88-0.95), and between the untrained human annotator and the automated score was 0.50 (95% CI, 0.36-0.61). Data from 9146 patients (in-hospital mortality 3.6%) were included to validate externally the discriminability of the automated pSOFA score. The admission-day pSOFA score had an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.82). CONCLUSIONS: The developed automated algorithm calculates pSOFA score with high accuracy and is more accurate than a trained expert rater and nontrained data abstracter. pSOFA's performance for predicting in-hospital mortality was lower in our cohort than it was for the originally derived score.

Dynamic Prediction of Mortality Using Longitudinally Measured Pediatric Sequential Organ Failure Assessment Scores: A Joint Modeling Approach.

OBJECTIVES: The pediatric Sequential Organ Failure Assessment (pSOFA) score was designed to track illness severity and predict mortality in critically ill children. Most commonly, pSOFA at a point in time is used to assess a static patient condition. However, this approach has a significant drawback because it fails to consider any changes in a patients' condition during their PICU stay and, especially, their response to initial critical care treatment. We aimed to evaluate the performance of longitudinal pSOFA scores for predicting mortality. DESIGN: Single-center, retrospective cohort study. SETTING: Quaternary 40-bed PICU. PATIENTS: All patients admitted to the PICU between 2015 and 2021 with at least 24 hours of ICU stay. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We calculated daily pSOFA scores up to 30 days, or until death or discharge from the PICU, if earlier. We used the joint longitudinal and time-to-event data model for the dynamic prediction of 30-day in-hospital mortality. The dataset, which included 9146 patients with a 30-day in-hospital mortality of 2.6%, was divided randomly into training (75%) and validation (25%) subsets, and subjected to 40 repeated stratified cross-validations. We used dynamic area under the curve (AUC) to evaluate the discriminative performance of the model. Compared with the admission-day pSOFA score, AUC for predicting mortality between days 5 and 30 was improved on average by 6.4% (95% CI, 6.3-6.6%) using longitudinal pSOFA scores from the first 3 days and 9.2% (95% CI, 9.0-9.5%) using scores from the first 5 days. CONCLUSIONS: Compared with admission-day pSOFA score, longitudinal pSOFA scores improved the accuracy of mortality prediction in PICU patients at a single center. The pSOFA score has the potential to be used dynamically for the evaluation of patient conditions.

Biomarker Assessment of a High-Risk, Data-Driven Pediatric Sepsis Phenotype Characterized by Persistent Hypoxemia, Encephalopathy, and Shock.

OBJECTIVES: Identification of children with sepsis-associated multiple organ dysfunction syndrome (MODS) at risk for poor outcomes remains a challenge. We sought to the determine reproducibility of the data-driven "persistent hypoxemia, encephalopathy, and shock" (PHES) phenotype and determine its association with inflammatory and endothelial biomarkers, as well as biomarker-based pediatric risk strata. DESIGN: We retrained and validated a random forest classifier using organ dysfunction subscores in the 2012-2018 electronic health record (EHR) dataset used to derive the PHES phenotype. We used this classifier to assign phenotype membership in a test set consisting of prospectively (2003-2023) enrolled pediatric septic shock patients. We compared profiles of the PERSEVERE family of biomarkers among those with and without the PHES phenotype and determined the association with established biomarker-based mortality and MODS risk strata. SETTING: Twenty-five PICUs across the United States. PATIENTS: EHR data from 15,246 critically ill patients with sepsis-associated MODS split into derivation and validation sets and 1,270 pediatric septic shock patients in the test set of whom 615 had complete biomarker data. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The area under the receiver operator characteristic curve of the modified classifier to predict PHES phenotype membership was 0.91 (95% CI, 0.90-0.92) in the EHR validation set. In the test set, PHES phenotype membership was associated with both increased adjusted odds of complicated course (adjusted odds ratio [aOR] 4.1; 95% CI, 3.2-5.4) and 28-day mortality (aOR of 4.8; 95% CI, 3.11-7.25) after controlling for age, severity of illness, and immunocompromised status. Patients belonging to the PHES phenotype were characterized by greater degree of systemic inflammation and endothelial activation, and were more likely to be stratified as high risk based on PERSEVERE biomarkers predictive of death and persistent MODS. CONCLUSIONS: The PHES trajectory-based phenotype is reproducible, independently associated with poor clinical outcomes, and overlapped with higher risk strata based on prospectively validated biomarker approaches.

External validation and biomarker assessment of a high-risk, data-driven pediatric sepsis phenotype characterized by persistent hypoxemia, encephalopathy, and shock.

Objective: Identification of children with sepsis-associated multiple organ dysfunction syndrome (MODS) at risk for poor outcomes remains a challenge. Data-driven phenotyping approaches that leverage electronic health record (EHR) data hold promise given the widespread availability of EHRs. We sought to externally validate the data-driven 'persistent hypoxemia, encephalopathy, and shock' (PHES) phenotype and determine its association with inflammatory and endothelial biomarkers, as well as biomarker-based pediatric risk-strata. Design: We trained and validated a random forest classifier using organ dysfunction subscores in the EHR dataset used to derive the PHES phenotype. We used the classifier to assign phenotype membership in a test set consisting of prospectively enrolled pediatric septic shock patients. We compared biomarker profiles of those with and without the PHES phenotype and determined the association with established biomarker-based mortality and MODS risk-strata. Setting: 25 pediatric intensive care units (PICU) across the U.S. Patients: EHR data from 15,246 critically ill patients sepsis-associated MODS and 1,270 pediatric septic shock patients in the test cohort of whom 615 had biomarker data. Interventions: None. Measurements and Main Results: The area under the receiver operator characteristic curve (AUROC) of the new classifier to predict PHES phenotype membership was 0.91(95%CI, 0.90-0.92) in the EHR validation set. In the test set, patients with the PHES phenotype were independently associated with both increased odds of complicated course (adjusted odds ratio [aOR] of 4.1, 95%CI: 3.2-5.4) and 28-day mortality (aOR of 4.8, 95%CI: 3.11-7.25) after controlling for age, severity of illness, and immuno-compromised status. Patients belonging to the PHES phenotype were characterized by greater degree of systemic inflammation and endothelial activation, and overlapped with high risk-strata based on PERSEVERE biomarkers predictive of death and persistent MODS. Conclusions: The PHES trajectory-based phenotype is reproducible, independently associated with poor clinical outcomes, and overlap with higher risk-strata based on validated biomarker approaches.

Scalable relevance ranking algorithm via semantic similarity assessment improves efficiency of medical chart review.

OBJECTIVE: Accurately assigning phenotype information to individual patients via computational phenotyping using Electronic Health Records (EHRs) has been seen as the first step towards enabling EHRs for precision medicine research. Chart review labels annotated by clinical experts, also known as "gold standard" labels, are essential for the development and validation of computational phenotyping algorithms. However, given the complexity of EHR systems, the process of chart review is both labor intensive and time consuming. We propose a fully automated algorithm, referred to as pGUESS, to rank EHR notes according to their relevance to a given phenotype. By identifying the most relevant notes, pGUESS can greatly improve the efficiency and accuracy of chart reviews. METHOD: pGUESS uses prior guided semantic similarity to measure the informativeness of a clinical note to a given phenotype. We first select candidate clinical concepts from a pool of comprehensive medical concepts using public knowledge sources and then derive the semantic embedding vector (SEV) for a reference article (SEVref) and each note (SEVnote). The algorithm scores the relevance of a note as the cosine similarity between SEVnote and SEVref. RESULTS: The algorithm was validated against four sets of 200 notes that were manually annotated by clinical experts to assess their informativeness to one of three disease phenotypes. pGUESS algorithm substantially outperforms existing unsupervised approaches for classifying the relevance status with respect to both accuracy and scalability across phenotypes. Averaging over the three phenotypes, the rank correlation between the algorithm ranking and gold standard label was 0.64 for pGUESS, but only 0.47 and 0.35 for the next two best performing algorithms. pGUESS is also much more computationally scalable compared to existing algorithms. CONCLUSION: pGUESS algorithm can substantially reduce the burden of chart review and holds potential in improving the efficiency and accuracy of human annotation.

Adverse drug event rates in pediatric pulmonary hypertension: a comparison of real-world data sources.

OBJECTIVE: Real-world data (RWD) are increasingly used for pharmacoepidemiology and regulatory innovation. Our objective was to compare adverse drug event (ADE) rates determined from two RWD sources, electronic health records and administrative claims data, among children treated with drugs for pulmonary hypertension. MATERIALS AND METHODS: Textual mentions of medications and signs/symptoms that may represent ADEs were identified in clinical notes using natural language processing. Diagnostic codes for the same signs/symptoms were identified in our electronic data warehouse for the patients with textual evidence of taking pulmonary hypertension-targeted drugs. We compared rates of ADEs identified in clinical notes to those identified from diagnostic code data. In addition, we compared putative ADE rates from clinical notes to those from a healthcare claims dataset from a large, national insurer. RESULTS: Analysis of clinical notes identified up to 7-fold higher ADE rates than those ascertained from diagnostic codes. However, certain ADEs (eg, hearing loss) were more often identified in diagnostic code data. Similar results were found when ADE rates ascertained from clinical notes and national claims data were compared. DISCUSSION: While administrative claims and clinical notes are both increasingly used for RWD-based pharmacovigilance, ADE rates substantially differ depending on data source. CONCLUSION: Pharmacovigilance based on RWD may lead to discrepant results depending on the data source analyzed. Further work is needed to confirm the validity of identified ADEs, to distinguish them from disease effects, and to understand tradeoffs in sensitivity and specificity between data sources.

Learning a Comorbidity-Driven Taxonomy of Pediatric Pulmonary Hypertension.

RATIONALE: Pediatric pulmonary hypertension (PH) is a heterogeneous condition with varying natural history and therapeutic response. Precise classification of PH subtypes is, therefore, crucial for individualizing care. However, gaps remain in our understanding of the spectrum of PH in children. OBJECTIVE: We seek to study the manifestations of PH in children and to assess the feasibility of applying a network-based approach to discern disease subtypes from comorbidity data recorded in longitudinal data sets. METHODS AND RESULTS: A retrospective cohort study comprising 6 943 263 children (<18 years of age) enrolled in a commercial health insurance plan in the United States, between January 2010 and May 2013. A total of 1583 (0.02%) children met the criteria for PH. We identified comorbidities significantly associated with PH compared with the general population of children without PH. A Bayesian comorbidity network was constructed to model the interdependencies of these comorbidities, and network-clustering analysis was applied to derive disease subtypes comprising subgraphs of highly connected comorbid conditions. A total of 186 comorbidities were found to be significantly associated with PH. Network analysis of comorbidity patterns captured most of the major PH subtypes with known pathological basis defined by the World Health Organization and Panama classifications. The analysis further identified many subtypes documented in only a few case studies, including rare subtypes associated with several well-described genetic syndromes. CONCLUSIONS: Application of network science to model comorbidity patterns recorded in longitudinal data sets can facilitate the discovery of disease subtypes. Our analysis relearned established subtypes, thus validating the approach, and identified rare subtypes that are difficult to discern through clinical observations, providing impetus for deeper investigation of the disease subtypes that will enrich current disease classifications.

A quantitative analysis of optimal treatment capacity for perinatal asphyxia.

OBJECTIVE: In centers electing to offer therapeutic hypothermia for treating hypoxic-ischemic encephalopathy (HIE), determining the optimal number of cooling devices is not straightforward. The authors used computer-based modeling to determine the level of service as a function of local HIE caseload and number of cooling devices available. METHODS: The authors used discrete event simulation to create a model that varied the number of HIE cases and number of cooling devices available. Outcomes of interest were percentage of HIE-affected infants not cooled, number of infants not cooled, and percentage of time that all cooling devices were in use. RESULTS: With 1 cooling device, even the smallest perinatal center did not achieve a cooling rate of 99% of eligible infants. In contrast, 2 devices ensured 99% service in centers treating as many as 20 infants annually. In centers averaging no more than 1 HIE infant monthly, the addition of a third cooling device did not result in a substantial reduction in the number of infants who would not be cooled. CONCLUSION: Centers electing to offer therapeutic hypothermia with only a single cooling device are at significant risk of being unable to provide treatment to eligible infants, whereas 2 devices appear to suffice for most institutions treating as many as 20 annual HIE cases. Three devices would rarely be needed given current caseloads seen at individual institutions. The quantitative nature of this analysis allows decision makers to determine the number of devices necessary to ensure adequate availability of therapeutic hypothermia given the HIE caseload of a particular institution.

Network analysis of team structure in the neonatal intensive care unit.

OBJECTIVE: The goal was to examine nursing team structure and its relationship with family satisfaction. METHODS: We used electronic health records to create patient-based, 1-mode networks of nursing handoffs. In these networks, nurses were represented as nodes and handoffs as edges. For each patient, we calculated network statistics including team size and diameter, network centrality index, proportion of newcomers to care teams according to day of hospitalization, and a novel measure of the average number of shifts between repeat caregivers, which was meant to quantify nursing continuity. We assessed parental satisfaction by using a standardized survey. RESULTS: Team size increased with increasing length of stay. At 2 weeks of age, 50% of shifts were staffed by a newcomer nurse who had not previously cared for the index patient. The patterns of newcomers to teams did not differ according to birth weight. When the population was dichotomized according to median mean repeat caregiver interval value, increased reports of problems with nursing care were seen with less-consistent staffing by familiar nurses. This relationship persisted after controlling for factors including birth weight, length of stay, and team size. CONCLUSIONS: Family perceptions of nursing care quality are more strongly associated with team structure and the sequence of nursing participation than with team size. Objective measures of health care team structure and function can be examined by applying network analytic techniques to information contained in electronic health records.

CoolSim: using industrial modeling techniques to examine the impact of selective head cooling in a model of perinatal regionalization.

OBJECTIVE: A selective head-cooling device for the treatment of moderate to severe hypoxic-ischemic encephalopathy has been approved by the Food and Drug Administration for use in the United States. To reflect the complexity of health care delivery at the systems level, we used the industrial modeling technique of discrete event simulation to analyze the impact of various deployment strategies for selective head cooling and its partner technology, amplitude-integrated electroencephalography. METHODS: We modeled the course through the perinatal system of all births in Massachusetts over a 1-year period. Cohort and care characteristics were drawn from existing databases. Results of a recently published trial were used to estimate the effects of selective head cooling. One thousand cohort replications were conducted to assess uncertainty. Several policy alternatives were examined, including no use of selective head cooling and scenarios that altered the number and placement of selective head-cooling and amplitude-integrated electroencephalography units throughout the state. Patient-level outcome and cost data were assessed. RESULTS: For all scenarios tested, the use of amplitude-integrated electroencephalography/selective head cooling resulted in better outcomes at lower cost. However, substantial differences in transfer rates, failure-to-cool rates, and total costs were seen across scenarios. Optimal decision-making regarding the number and placement of devices led to a 16% improvement in cost savings and a 10-fold decrease in failure-to-cool rates, compared with other deployment scenarios. These results were insensitive to significant changes in model inputs. CONCLUSIONS: On the basis of currently available data, the package of amplitude-integrated electroencephalography and selective head cooling seems to be an economically desirable intervention. Quantifiable techniques to assess system-wide technology performance provide a powerful approach to informing decisions regarding the structure and function of health care systems.