Application of Participatory Ergonomics to the Dissemination of a Quality Improvement Program for Optimizing Blood Culture Use.

BACKGROUND: Blood cultures are overused in pediatric ICUs (PICUs), which may lead to unnecessary antibiotic use and antibiotic resistance. Using a participatory ergonomics (PE) approach, the authors disseminated a quality improvement (QI) program for optimizing blood culture use in PICUs to a national 14-hospital collaborative. The objective of this study was to evaluate the dissemination process and its impact on blood culture reduction. METHODS: The PE approach emphasized three key principles (stakeholder participation, application of human factors and ergonomics knowledge and tools, and cross-site collaboration) with a six-step dissemination process. Data on interactions between sites and the coordinating team and site experiences with the dissemination process were collected using site diaries and semiannual surveys with local QI teams, respectively, and correlated with the site-specific change in blood culture rates. RESULTS: Overall, participating sites were able to successfully implement the program and reduced their blood culture rates from 149.4 blood cultures per 1,000 patient-days/month before implementation to 100.5 blood cultures per 1,000 patient-days/month after implementation, corresponding to a 32.7% relative reduction (p < 0.001). Variations in the dissemination process, as well as in local interventions and implementation strategies, were observed across sites. Site-specific changes in blood culture rates were weakly negatively correlated with the number of preintervention interactions with the coordinating team (p = 0.057) but not correlated with their experiences with the six domains of the dissemination process or their interventions. CONCLUSIONS: The authors applied a PE approach to disseminate a QI program for optimizing PICU blood culture use to a multisite collaborative. Working with local stakeholders, participating sites tailored their interventions and implementation processes and achieved the goal of reducing blood culture use.

A machine-learning approach for decision support and risk stratification of pediatric perioperative patients based on the APRICOT dataset.

BACKGROUND: Pediatric anesthesia has evolved to a high level of patient safety, yet a small chance remains for serious perioperative complications, even in those traditionally considered at low risk. In practice, prediction of at-risk patients currently relies on the American Society of Anesthesiologists Physical Status (ASA-PS) score, despite reported inconsistencies with this method. AIMS: The goal of this study was to develop predictive models that can classify children as low risk for anesthesia at the time of surgical booking and after anesthetic assessment on the procedure day. METHODS: Our dataset was derived from APRICOT, a prospective observational cohort study conducted by 261 European institutions in 2014 and 2015. We included only the first procedure, ASA-PS classification I to III, and perioperative adverse events not classified as drug errors, reducing the total number of records to 30 325 with an adverse event rate of 4.43%. From this dataset, a stratified train:test split of 70:30 was used to develop predictive machine learning algorithms that could identify children in ASA-PS class I to III at low risk for severe perioperative critical events that included respiratory, cardiac, allergic, and neurological complications. RESULTS: Our selected models achieved accuracies of >0.9, areas under the receiver operating curve of 0.6-0.7, and negative predictive values >95%. Gradient boosting models were the best performing for both the booking phase and the day-of-surgery phase. CONCLUSIONS: This work demonstrates that prediction of patients at low risk of critical PAEs can be made on an individual, rather than population-based, level by using machine learning. Our approach yielded two models that accommodate wide clinical variability and, with further development, are potentially generalizable to many surgical centers.

Predicting respiratory decompensation in mechanically ventilated adult ICU patients.

Introduction: Mechanical ventilation is a life-saving treatment in the Intensive Care Unit (ICU), but often causes patients to be at risk of further respiratory complication. We created a statistical model utilizing electronic health record and physiologic vitals data to predict the Center for Disease Control and Prevention (CDC) defined Ventilator Associated Complications (VACs). Further, we evaluated the effect of data temporal resolution and feature generation method choice on the accuracy of such a constructed model. Methods: We constructed a random forest model to predict occurrence of VACs using health records and chart events from adult patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database. We trained the machine learning models on two patient populations of 1921 and 464 based on low and high frequency data availability. Model features were generated using both basic statistical summaries and tsfresh, a python library that generates a large number of derived time-series features. Classification to determine whether a patient will experience VAC one hour after 35 h of ventilation was performed using a random forest classifier. Two different sample spaces conditioned on five varying feature extraction techniques were evaluated to identify the most optimal selection of features resulting in the best VAC discrimination. Each dataset was assessed using K-folds cross-validation (k = 10), giving average area under the receiver operating characteristic curves (AUROCs) and accuracies. Results: After feature selection, hyperparameter tuning, and feature extraction, the best performing model used automatically generated features on high frequency data and achieved an average AUROC of 0.83 ± 0.11 and an average accuracy of 0.69 ± 0.10. Discussion: Results show the potential viability of predicting VACs using machine learning, and indicate that higher-resolution data and the larger feature set generated by tsfresh yield better AUROCs compared to lower-resolution data and manual statistical features.

A pilot study to predict cardiac arrest in the pediatric intensive care unit.

BACKGROUND: Cardiac arrest is a leading cause of mortality prior to discharge for children admitted to the pediatric intensive care unit. To address this problem, we used machine learning to predict cardiac arrest up to three hours in advance. METHODS: Our data consists of 240 Hz ECG waveform data, 0.5 Hz physiological time series data, medications, and demographics from 1,145 patients in the pediatric intensive care unit at the Johns Hopkins Hospital, 15 of whom experienced a cardiac arrest. The data were divided into training, validating, and testing sets, and features were generated every five minutes. 23 heart rate variability (HRV) metrics were determined from ECG waveforms. 96 summary statistics were calculated for 12 vital signs, such as respiratory rate and blood pressure. Medications were classified into 42 therapeutic drug classes. Binary features were generated to indicate the administration of these different drugs. Next, six machine learning models were evaluated: logistic regression, support vector machine, random forest, XGBoost, LightGBM, and a soft voting ensemble. RESULTS: XGBoost performed the best, with 0.971 auROC, 0.797 auPRC, 99.5% sensitivity, and 69.6% specificity on an independent test set. CONCLUSION: We have created high-performing models that identify signatures of in-hospital cardiac arrest (IHCA) that may not be evident to clinicians. These signatures include a combination of heart rate variability metrics, vital signs data, and therapeutic drug classes. These machine learning models can predict IHCA up to three hours prior to onset with high performance, allowing clinicians to intervene earlier, improving patient outcomes.

Association of Diagnostic Stewardship for Blood Cultures in Critically Ill Children With Culture Rates, Antibiotic Use, and Patient Outcomes: Results of the Bright STAR Collaborative.

Importance: Blood culture overuse in the pediatric intensive care unit (PICU) can lead to unnecessary antibiotic use and contribute to antibiotic resistance. Optimizing blood culture practices through diagnostic stewardship may reduce unnecessary blood cultures and antibiotics. Objective: To evaluate the association of a 14-site multidisciplinary PICU blood culture collaborative with culture rates, antibiotic use, and patient outcomes. Design, Setting, and Participants: This prospective quality improvement (QI) collaborative involved 14 PICUs across the United States from 2017 to 2020 for the Bright STAR (Testing Stewardship for Antibiotic Reduction) collaborative. Data were collected from each participating PICU and from the Children's Hospital Association Pediatric Health Information System for prespecified primary and secondary outcomes. Exposures: A local QI program focusing on blood culture practices in the PICU (facilitated by a larger QI collaborative). Main Outcomes and Measures: The primary outcome was blood culture rates (per 1000 patient-days/mo). Secondary outcomes included broad-spectrum antibiotic use (total days of therapy and new initiations of broad-spectrum antibiotics ≥3 days after PICU admission) and PICU rates of central line-associated bloodstream infection (CLABSI), Clostridioides difficile infection, mortality, readmission, length of stay, sepsis, and severe sepsis/septic shock. Results: Across the 14 PICUs, the blood culture rate was 149.4 per 1000 patient-days/mo preimplementation and 100.5 per 1000 patient-days/mo postimplementation, for a 33% relative reduction (95% CI, 26%-39%). Comparing the periods before and after implementation, the rate of broad-spectrum antibiotic use decreased from 506 days to 440 days per 1000 patient-days/mo, respectively, a 13% relative reduction (95% CI, 7%-19%). The broad-spectrum antibiotic initiation rate decreased from 58.1 to 53.6 initiations/1000 patient-days/mo, an 8% relative reduction (95% CI, 4%-11%). Rates of CLABSI decreased from 1.8 to 1.1 per 1000 central venous line days/mo, a 36% relative reduction (95% CI, 20%-49%). Mortality, length of stay, readmission, sepsis, and severe sepsis/septic shock were similar before and after implementation. Conclusions and Relevance: Multidisciplinary diagnostic stewardship interventions can reduce blood culture and antibiotic use in the PICU. Future work will determine optimal strategies for wider-scale dissemination of diagnostic stewardship in this setting while monitoring patient safety and balancing measures.

Examining Diurnal Differences in Multidisciplinary Care Teams at a Pediatric Trauma Center Using Electronic Health Record Data: Social Network Analysis.

BACKGROUND: The care of pediatric trauma patients is delivered by multidisciplinary care teams with high fluidity that may vary in composition and organization depending on the time of day. OBJECTIVE: This study aims to identify and describe diurnal variations in multidisciplinary care teams taking care of pediatric trauma patients using social network analysis on electronic health record (EHR) data. METHODS: Metadata of clinical activities were extracted from the EHR and processed into an event log, which was divided into 6 different event logs based on shift (day or night) and location (emergency department, pediatric intensive care unit, and floor). Social networks were constructed from each event log by creating an edge among the functional roles captured within a similar time interval during a shift. Overlapping communities were identified from the social networks. Day and night network structures for each care location were compared and validated via comparison with secondary analysis of qualitatively derived care team data, obtained through semistructured interviews; and member-checking interviews with clinicians. RESULTS: There were 413 encounters in the 1-year study period, with 65.9% (272/413) and 34.1% (141/413) beginning during day and night shifts, respectively. A single community was identified at all locations during the day and in the pediatric intensive care unit at night, whereas multiple communities corresponding to individual specialty services were identified in the emergency department and on the floor at night. Members of the trauma service belonged to all communities, suggesting that they were responsible for care coordination. Health care professionals found the networks to be largely accurate representations of the composition of the care teams and the interactions among them. CONCLUSIONS: Social network analysis was successfully used on EHR data to identify and describe diurnal differences in the composition and organization of multidisciplinary care teams at a pediatric trauma center.

A Research Agenda for Diagnostic Excellence in Critical Care Medicine.

Diagnosing critically ill patients in the intensive care unit is difficult. As a result, diagnostic errors in the intensive care unit are common and have been shown to cause harm. Research to improve diagnosis in critical care medicine has accelerated in past years. However, much work remains to fully elucidate the diagnostic process in critical care. To achieve diagnostic excellence, interdisciplinary research is needed, adopting a balanced strategy of continued biomedical discovery while addressing the complex care delivery systems underpinning the diagnosis of critical illness.

Predicting Flow Rate Escalation for Pediatric Patients on High Flow Nasal Cannula Using Machine Learning.

Background: High flow nasal cannula (HFNC) is commonly used as non-invasive respiratory support in critically ill children. There are limited data to inform consensus on optimal device parameters, determinants of successful patient response, and indications for escalation of support. Clinical scores, such as the respiratory rate-oxygenation (ROX) index, have been described as a means to predict HFNC non-response, but are limited to evaluating for escalations to invasive mechanical ventilation (MV). In the presence of apparent HFNC non-response, a clinician may choose to increase the HFNC flow rate to hypothetically prevent further respiratory deterioration, transition to an alternative non-invasive interface, or intubation for MV. To date, no models have been assessed to predict subsequent escalations of HFNC flow rates after HFNC initiation. Objective: To evaluate the abilities of tree-based machine learning algorithms to predict HFNC flow rate escalations. Methods: We performed a retrospective, cohort study assessing children admitted for acute respiratory failure under 24 months of age placed on HFNC in the Johns Hopkins Children's Center pediatric intensive care unit from January 2019 through January 2020. We excluded encounters with gaps in recorded clinical data, encounters in which MV treatment occurred prior to HFNC, and cases electively intubated in the operating room. The primary study outcome was discriminatory capacity of generated machine learning algorithms to predict HFNC flow rate escalations as compared to each other and ROX indices using area under the receiver operating characteristic (AUROC) analyses. In an exploratory fashion, model feature importance rankings were assessed by comparing Shapley values. Results: Our gradient boosting model with a time window of 8 h and lead time of 1 h before HFNC flow rate escalation achieved an AUROC with a 95% confidence interval of 0.810 ± 0.003. In comparison, the ROX index achieved an AUROC of 0.525 ± 0.000. Conclusion: In this single-center, retrospective cohort study assessing children under 24 months of age receiving HFNC for acute respiratory failure, tree-based machine learning models outperformed the ROX index in predicting subsequent flow rate escalations. Further validation studies are needed to ensure generalizability for bedside application.

Early Prediction of Multiple Organ Dysfunction in the Pediatric Intensive Care Unit.

Objective: The objective of the study is to build models for early prediction of risk for developing multiple organ dysfunction (MOD) in pediatric intensive care unit (PICU) patients. Design: The design of the study is a retrospective observational cohort study. Setting: The setting of the study is at a single academic PICU at the Johns Hopkins Hospital, Baltimore, MD. Patients: The patients included in the study were <18 years of age admitted to the PICU between July 2014 and October 2015. Measurements and main results: Organ dysfunction labels were generated every minute from preceding 24-h time windows using the International Pediatric Sepsis Consensus Conference (IPSCC) and Proulx et al. MOD criteria. Early MOD prediction models were built using four machine learning methods: random forest, XGBoost, GLMBoost, and Lasso-GLM. An optimal threshold learned from training data was used to detect high-risk alert events (HRAs). The early prediction models from all methods achieved an area under the receiver operating characteristics curve ≥0.91 for both IPSCC and Proulx criteria. The best performance in terms of maximum F1-score was achieved with random forest (sensitivity: 0.72, positive predictive value: 0.70, F1-score: 0.71) and XGBoost (sensitivity: 0.8, positive predictive value: 0.81, F1-score: 0.81) for IPSCC and Proulx criteria, respectively. The median early warning time was 22.7 h for random forest and 37 h for XGBoost models for IPSCC and Proulx criteria, respectively. Applying spectral clustering on risk-score trajectories over 24 h following early warning provided a high-risk group with ≥0.93 positive predictive value. Conclusions: Early predictions from risk-based patient monitoring could provide more than 22 h of lead time for MOD onset, with ≥0.93 positive predictive value for a high-risk group identified pre-MOD.

Culture Ordering for Patients with New-onset Fever: A Survey of Pediatric Intensive Care Unit Clinician Practices.

INTRODUCTION: Accurate assessment of infection in critically ill patients is vital to their care. Both indiscretion and under-utilization of diagnostic microbiology testing can contribute to inappropriate antibiotic administration or delays in diagnosis. However, indiscretion in diagnostic microbiology cultures may also lead to unnecessary tests that, if false-positive, would incur additional costs and unhelpful evaluations. This quality improvement project objective was to assess pediatric intensive care unit (PICU) clinicians' attitudes and practices around the microbiology work-up for patients with new-onset fever. METHODS: We developed and conducted a self-administered electronic survey of PICU clinicians at a single institution. The survey included 7 common clinical vignettes of PICU patients with new-onset fever and asked participants whether they would obtain central line blood cultures, peripheral blood cultures, respiratory aspirate cultures, cerebrospinal fluid cultures, urine cultures, and/or urinalyses. RESULTS: Forty-seven of 54 clinicians (87%) completed the survey. Diagnostic specimen ordering practices were notably heterogeneous. Respondents unanimously favored a decision-support algorithm to guide culture specimen ordering practices for PICU patients with fever (100%, N = 47). A majority (91.5%, N = 43) indicated that a decision-support algorithm would be a means to align PICU and consulting care teams when ordering culture specimens for patients with fever. CONCLUSION: This survey revealed variability of diagnostic specimen ordering practices for patients with new fever, supporting an opportunity to standardize practices. Clinicians favored a decision-support tool and thought that it would help align patient management between clinical team members. The results will be used to inform future diagnostic stewardship efforts.

Prediction of Impending Septic Shock in Children With Sepsis.

OBJECTIVES: Sepsis and septic shock are leading causes of in-hospital mortality. Timely treatment is crucial in improving patient outcome, yet treatment delays remain common. Early prediction of those patients with sepsis who will progress to its most severe form, septic shock, can increase the actionable window for interventions. We aim to extend a time-evolving risk score, previously developed in adult patients, to predict pediatric sepsis patients who are likely to develop septic shock before its onset, and to determine whether or not these risk scores stratify into groups with distinct temporal evolution once this prediction is made. DESIGN: Retrospective cohort study. SETTING: Academic medical center from July 1, 2016, to December 11, 2020. PATIENTS: Six-thousand one-hundred sixty-one patients under 18 admitted to the Johns Hopkins Hospital PICU. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We trained risk models to predict impending transition into septic shock and compute time-evolving risk scores representative of a patient's probability of developing septic shock. We obtain early prediction performance of 0.90 area under the receiver operating curve, 43% overall positive predictive value, patient-specific positive predictive value as high as 62%, and an 8.9-hour median early warning time using Sepsis-3 labels based on age-adjusted Sequential Organ Failure Assessment score. Using spectral clustering, we stratified pediatric sepsis patients into two clusters differing in septic shock prevalence, mortality, and proportion of patients adequately fluid resuscitated. CONCLUSIONS: We demonstrate the applicability of our methodology for early prediction and stratification for risk of septic shock in pediatric sepsis patients. Through analyses of risk score evolution over time, we corroborate our past finding of an abrupt transition preceding onset of septic shock in children and are able to stratify pediatric sepsis patients using their risk score trajectories into low and high-risk categories.

Diagnostic Errors in Pediatric Critical Care: A Systematic Review.

OBJECTIVES: To summarize the literature on prevalence, impact, and contributing factors related to diagnostic error in the PICU. DATA SOURCES: Search of PubMed, EMBASE, and the Cochrane Library up to December 2019. STUDY SELECTION: Studies on diagnostic error and the diagnostic process in pediatric critical care were included. Non-English studies with no translation, case reports/series, studies providing no information on diagnostic error, studies focused on non-PICU populations, and studies focused on a single condition/disease or a single diagnostic test/tool were excluded. DATA EXTRACTION: Data on research design, objectives, study sample, and results pertaining to the prevalence, impact, and factors associated with diagnostic error were abstracted from each study. DATA SYNTHESIS: Using independent tiered review, 396 abstracts were screened, and 17 studies (14 full-text, 3 abstracts) were ultimately included. Fifteen of 17 studies (88%) had an observational research design. Autopsy studies (autopsy rates were 20-47%) showed a 10-23% rate of missed major diagnoses; 5-16% of autopsy-discovered diagnostic errors had a potential adverse impact on survival and would have changed management. Retrospective record reviews reported varying rates of diagnostic error from 8% in a general PICU population to 12% among unexpected critical admissions and 21-25% of patients discussed at PICU morbidity and mortality conferences. Cardiovascular, infectious, congenital, and neurologic conditions were most commonly misdiagnosed. Systems factors (40-67%), cognitive factors (20-3%), and both systems and cognitive factors (40%) were associated with diagnostic error. Limited information was available on the impact of misdiagnosis. CONCLUSIONS: Knowledge of diagnostic errors in the PICU is limited. Future work to understand diagnostic errors should involve a balanced focus between studying the diagnosis of individual diseases and uncovering common system- and process-related determinants of diagnostic error.

Consensus Recommendations for Blood Culture Use in Critically Ill Children Using a Modified Delphi Approach.

OBJECTIVES: Blood cultures are fundamental in evaluating for sepsis, but excessive cultures can lead to false-positive results and unnecessary antibiotics. Our objective was to create consensus recommendations focusing on when to safely avoid blood cultures in PICU patients. DESIGN: A panel of 29 multidisciplinary experts engaged in a two-part modified Delphi process. Round 1 consisted of a literature summary and an electronic survey sent to invited participants. In the survey, participants rated a series of recommendations about when to avoid blood cultures on five-point Likert scale. Consensus was achieved for the recommendation(s) if 75% of respondents chose a score of 4 or 5, and these were included in the final recommendations. Any recommendations that did not meet these a priori criteria for consensus were discussed during the in-person expert panel review (Round 2). Round 2 was facilitated by an independent expert in consensus methodology. After a review of the survey results, comments from round 1, and group discussion, the panelists voted on these recommendations in real-time. SETTING: Experts' institutions; in-person discussion in Baltimore, MD. SUBJECTS: Experts in pediatric critical care, infectious diseases, nephrology, oncology, and laboratory medicine. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Of the 27 original recommendations, 18 met criteria for achieving consensus in Round 1; some were modified for clarity or condensed from multiple into single recommendations during Round 2. The remaining nine recommendations were discussed and modified until consensus was achieved during Round 2, which had 26 real-time voting participants. The final document contains 19 recommendations. CONCLUSIONS: Using a modified Delphi process, we created consensus recommendations on when to avoid blood cultures and prevent overuse in the PICU. These recommendations are a critical step in disseminating diagnostic stewardship on a wider scale in critically ill children.