Clinical Decision Support Principles for Quality Improvement and Research.

Pediatric hospitalists frequently interact with clinical decision support (CDS) tools in patient care and use these tools for quality improvement or research. In this method/ology paper, we provide an introduction and practical approach to developing and evaluating CDS tools within the electronic health record. First, we define CDS and describe the types of CDS interventions that exist. We then outline a stepwise approach to CDS development, which begins with defining the problem and understanding the system. We present a framework for metric development and then describe tools that can be used for CDS design (eg, 5 Rights of CDS, "10 commandments," usability heuristics, human-centered design) and testing (eg, validation, simulation, usability testing). We review approaches to evaluating CDS tools, which range from randomized studies to traditional quality improvement methods. Lastly, we discuss practical considerations for implementing CDS, including the assessment of a project team's skills and an organization's information technology resources.

Between-hospital variation in clinical decision support availability for common inpatient pediatric conditions: Results of a national Pediatric Research in Inpatient Settings (PRIS) Network survey.

Implementing pediatric-focused clinical decision support (CDS) into hospital electronic health records can lead to improvements in patient care and accelerate quality improvement and research initiatives. However, its design, development, and implementation can be a time-consuming and costly endeavor that may not be feasible for all hospital settings. In this cross-sectional study, we surveyed Pediatric Research in Inpatient Settings (PRIS) Network hospitals about the availability of CDS tools to gain an understanding of the functionality available across 8 common inpatient pediatric diagnoses. Among the conditions, asthma had the most extensive CDS availability, while mood disorders had the least. Overall, freestanding children's hospitals had the greatest breadth in CDS coverage across conditions and depth in CDS types within conditions. Future initiatives should examine the relationship between CDS availability and clinical outcomes as well as its relationship with hospitals' performance executing multicenter informatics projects, quality improvement collaboratives, and implementation science strategies.

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit.

BACKGROUND: Late recognition of in-hospital deterioration is a source of preventable harm. Emergency transfers (ET), when hospitalized patients require intensive care unit (ICU) interventions within 1 h of ICU transfer, are a proximal measure of late recognition associated with increased mortality and length of stay (LOS). OBJECTIVE: To apply diagnostic process improvement frameworks to identify missed opportunities for improvement in diagnosis (MOID) in ETs and evaluate their association with outcomes. DESIGN, SETTINGS, AND PARTICIPANTS: A single-center retrospective cohort study of ETs, January 2015 to June 2019. ET criteria include intubation, vasopressor initiation, or ≥ $\ge \phantom{\rule{}{0ex}}$ 60 mL/kg fluid resuscitation 1 h before to 1 h after ICU transfer. The primary exposure was the presence of MOID, determined using SaferDx. Cases were screened by an ICU and non-ICU physician. Final determinations were made by an interdisciplinary group. Diagnostic process improvement opportunities were identified. MAIN OUTCOME AND MEASURES: Primary outcomes were in-hospital mortality and posttransfer LOS, analyzed by multivariable regression adjusting for age, service, deterioration category, and pretransfer LOS. RESULTS: MOID was identified in 37 of 129 ETs (29%, 95% confidence interval [CI] 21%-37%). Cases with MOID differed in originating service, but not demographically. Recognizing the urgency of an identified condition was the most common diagnostic process opportunity. ET cases with MOID had higher odds of mortality (odds ratio 5.5; 95% CI 1.5-20.6; p = .01) and longer posttransfer LOS (rate ratio 1.7; 95% CI 1.1-2.6; p = .02). CONCLUSION: MOID are common in ETs and are associated with increased mortality risk and posttransfer LOS. Diagnostic improvement strategies should be leveraged to support earlier recognition of clinical deterioration.

Classification of Health Information Technology Safety Events in a Pediatric Tertiary Care Hospital.

OBJECTIVE: State agencies have developed reporting systems of safety events that include events related to health information technology (HIT). These data come from hospital reporting systems where staff submit safety reports and nurses, in the role of safety managers, review, and code events. Safety managers may have varying degrees of experience with identifying events related to HIT. Our objective was to review events potentially involving HIT and compare those with what was reported to the state. METHODS: We performed a structured review of 1 year of safety events from an academic pediatric healthcare system. We reviewed the free-text description of each event and applied a classification scheme derived from the AHRQ Health IT Hazard Manager and compared the results with events reported to the state as involving HIT. RESULTS: Of 33,218 safety events for a 1-year period, 1247 included key words related to HIT and/or were indicated by safety managers as involving HIT. Of the 1247 events, the structured review identified 769 as involving HIT. In comparison, safety managers only identified 194 of the 769 events (25%) as involving HIT. Most events, 353 (46%), not identified by safety managers were documentation issues. Of the 1247 events, the structured review identified 478 as not involving HIT while safety managers identified and reported 81 of these 478 events (17%) as involving HIT. CONCLUSIONS: The current process of reporting safety events lacks standardization in identifying health technology contributions to safety events, which may minimize the effectiveness of safety initiatives.

Clinician Perspectives on Specifications for Metrics to Inform Pediatric Alarm Management.

Background: Ongoing management of monitor alarms is important for reducing alarm fatigue among clinicians (e.g., nurses, physicians). Strategies to enhance clinician engagement in active alarm management in pediatric acute care have not been well explored. Access to alarm summary metrics may enhance clinician engagement. Objective: To lay the foundation for intervention development, we sought to identify functional specifications for formulating, packaging, and delivering alarm metrics to clinicians. Methods: Our team of clinician scientists and human factors engineers conducted focus groups with clinicians from medical-surgical inpatient units in a children's hospital. We inductively coded transcripts, developed codes into themes, and grouped themes into "current state" and "future state." Results: We conducted five focus groups with 13 clinicians (eight registered nurses and five doctors of medicine). In the current state, information exchanged among team members about alarm burden is initiated by nurses on an ad hoc basis. For a future state, clinicians identified ways in which alarm metrics could help them manage alarms and described specific information, such as alarm trends, benchmarks, and contextual data, that would support decision-making. Conclusion: We developed four recommendations for future strategies to enhance clinicians' active management of patient alarms: (1) formulate alarm metrics for clinicians by categorizing alarm rates by type and summarizing alarm trends over time, (2) package alarm metrics with contextual patient data to facilitate clinicians' sensemaking, (3) deliver alarm metrics in a forum that facilitates interprofessional discussion, and (4) provide clinician education to establish a shared mental model about alarm fatigue and evidence-based alarm-reduction strategies.

Clinical decision support with a comprehensive in-EHR patient tracking system improves genetic testing follow up.

OBJECTIVE: We sought to develop and evaluate an electronic health record (EHR) genetic testing tracking system to address the barriers and limitations of existing spreadsheet-based workarounds. MATERIALS AND METHODS: We evaluated the spreadsheet-based system using mixed effects logistic regression to identify factors associated with delayed follow up. These factors informed the design of an EHR-integrated genetic testing tracking system. After deployment, we assessed the system in 2 ways. We analyzed EHR access logs and note data to assess patient outcomes and performed semistructured interviews with users to identify impact of the system on work. RESULTS: We found that patient-reported race was a significant predictor of documented genetic testing follow up, indicating a possible inequity in care. We implemented a CDS system including a patient data capture form and management dashboard to facilitate important care tasks. The system significantly sped review of results and significantly increased documentation of follow-up recommendations. Interviews with key system users identified a range of sociotechnical factors (ie, tools, tasks, collaboration) that contribute to safer and more efficient care. DISCUSSION: Our new tracking system ended decades of workarounds for identifying and communicating test results and improved clinical workflows. Interview participants related that the system decreased cognitive and time burden which allowed them to focus on direct patient interaction. CONCLUSION: By assembling a multidisciplinary team, we designed a novel patient tracking system that improves genetic testing follow up. Similar approaches may be effective in other clinical settings.

Measuring Training Disruptions Using an Informatics Based Tool.

OBJECTIVE: Training disruptions, such as planned curricular adjustments or unplanned global pandemics, impact residency training in ways that are difficult to quantify. Informatics-based medical education tools can help measure these impacts. We tested the ability of a software platform driven by electronic health record data to quantify anticipated changes in trainee clinical experiences during the COVID-19 pandemic. METHODS: We previously developed and validated the Trainee Individualized Learning System (TRAILS) to identify pediatric resident clinical experiences (i.e. shifts, resident provider-patient interactions (rPPIs), and diagnoses). We used TRAILS to perform a year-over-year analysis comparing pediatrics residents at a large academic children's hospital during March 15-June 15 in 2018 (Control #1), 2019 (Control #2), and 2020 (Exposure). RESULTS: Residents in the exposure cohort had fewer shifts than those in both control cohorts (P < .05). rPPIs decreased an average of 43% across all PGY levels, with interns experiencing a 78% decrease in Continuity Clinic. Patient continuity decreased from 23% to 11%. rPPIs with common clinic and emergency department diagnoses decreased substantially during the exposure period. CONCLUSIONS: Informatics tools like TRAILS may help program directors understand the impact of training disruptions on resident clinical experiences and target interventions to learners' needs and development.

Sustainable deimplementation of continuous pulse oximetry monitoring in children hospitalized with bronchiolitis: study protocol for the Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial.

BACKGROUND: Methods of sustaining the deimplementation of overused medical practices (i.e., practices not supported by evidence) are understudied. In pediatric hospital medicine, continuous pulse oximetry monitoring of children with the common viral respiratory illness bronchiolitis is recommended only under specific circumstances. Three national guidelines discourage its use for children who are not receiving supplemental oxygen, but guideline-discordant practice (i.e., overuse) remains prevalent. A 6-hospital pilot of educational outreach with audit and feedback resulted in immediate reductions in overuse; however, the best strategies to optimize sustainment of deimplementation success are unknown. METHODS: The Eliminating Monitor Overuse (EMO) trial will compare two deimplementation strategies in a hybrid type III effectiveness-deimplementation trial. This longitudinal cluster-randomized design will be conducted in Pediatric Research in Inpatient Settings (PRIS) Network hospitals and will include baseline measurement, active deimplementation, and sustainment phases. After a baseline measurement period, 16-19 hospitals will be randomized to a deimplementation strategy that targets unlearning (educational outreach with audit and feedback), and the other 16-19 will be randomized to a strategy that targets unlearning and substitution (adding an EHR-integrated clinical pathway decision support tool). The primary outcome is the sustainment of deimplementation in bronchiolitis patients who are not receiving any supplemental oxygen, analyzed as a longitudinal difference-in-differences comparison of overuse rates across study arms. Secondary outcomes include equity of deimplementation and the fidelity to, and cost of, each deimplementation strategy. To understand how the deimplementation strategies work, we will test hypothesized mechanisms of routinization (clinicians developing new routines supporting practice change) and institutionalization (embedding of practice change into existing organizational systems). DISCUSSION: The EMO trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, costs, and likelihood of sustained practice change using rigorously designed deimplementation strategies. The trial will also advance care for a high-incidence, costly pediatric lung disease. TRIAL REGISTRATION: ClinicalTrials.gov, NCT05132322 . Registered on November 10, 2021.

Diagnostic Reasoning of Resident Physicians in the Age of Clinical Pathways.

Background: Development of skills in diagnostic reasoning is paramount to the transition from novice to expert clinicians. Efforts to standardize approaches to diagnosis and treatment using clinical pathways are increasingly common. The effects of implementing pathways into systems of care during diagnostic education and practice among pediatric residents are not well described. Objective: To characterize pediatric residents' perceptions of the tradeoffs between clinical pathway use and diagnostic reasoning. Methods: We conducted a qualitative study from May to December 2019. Senior pediatric residents from a high-volume general pediatric inpatient service at an academic hospital participated in semi-structured interviews. We utilized a basic interpretive qualitative approach informed by a dual process diagnostic reasoning framework. Results: Nine residents recruited via email were interviewed. Residents reported using pathways when admitting patients and during teaching rounds. All residents described using pathways primarily as management tools for patients with a predetermined diagnosis, rather than as aids in formulating a diagnosis. As such, pathways primed residents to circumvent crucial steps of deliberate diagnostic reasoning. However, residents relied on bedside assessment to identify when patients are "not quite fitting the mold" of the current pathway diagnosis, facilitating recalibration of the diagnostic process. Conclusions: This study identifies important educational implications at the intersection of residents' cognitive diagnostic processes and use of clinical pathways. We highlight potential challenges clinical pathways pose for skill development in diagnostic reasoning by pediatric residents. We suggest opportunities for educators to leverage clinical pathways as a framework for development of these skills.

Using Data-Driven Machine Learning to Predict Unplanned ICU Transfers with Critical Deterioration from Electronic Health Records.

OBJECTIVE: We aimed to develop a data-driven machine learning model for predicting critical deterioration events from routinely collected EHR data in hospitalized children. MATERIALS: This retrospective cohort study included all pediatric inpatients hospitalized on a medical or surgical ward between 2014-2018 at a quaternary children's hospital. METHODS: We developed a large data-driven approach and evaluated three machine learning models to predict pediatric critical deterioration events. We evaluated the models using a nested, stratified 10-fold cross-validation. The evaluation metrics included C-statistic, sensitivity, and positive predictive value. We also compared the machine learning models with patients identified as high-risk Watchers by bedside clinicians. RESULTS: The study included 57,233 inpatient admissions from 34,976 unique patients. 3,943 variables were identified from the EHR data. The XGBoost model performed best (C-statistic=0.951, CI: 0.946 ∼ 0.956). CONCLUSIONS: Our data-driven machine learning models accurately predicted patient deterioration. Future sociotechnical analysis will inform deployment within the clinical setting.

Characteristics of Emergency Room and Hospital Encounters Resulting From Consumer Home Monitors.

BACKGROUND AND OBJECTIVES: Consumer home monitors (CHM), which measure vital signs, are popular products marketed to detect airway obstruction and arrhythmia. Yet, they lack evidence of infant death prevention, demonstrate suboptimal accuracy, and may result in false alarms that prompt unnecessary acute care visits. To better understand the hospital utilization and costs of CHM, we characterized emergency department (ED) and hospital encounters associated with CHM use at a children's hospital. METHODS: We used structured query language to search the free text of all ED and admission notes between January 2013 and December 2019 to identify clinical documentation discussing CHM use. Two physicians independently reviewed the presence of CHM use and categorized encounter characteristics. RESULTS: Evidence of CHM use contributed to the presentation of 36 encounters in a sample of over 300 000 encounters, with nearly half occurring in 2019. The leading discharge diagnoses were viral infection (13, 36%), gastroesophageal reflux (8, 22%) and false positive alarm (6, 17%). Median encounter duration was 20 hours (interquartile range: 3 hours to 2 days; max 10.5 days) and median cost of encounters was $2188 (interquartile range: $255 to $7632; max $84 928). CONCLUSIONS: Although the annual rate of CHM-related encounters was low and did not indicate a major public health burden, for individual families who present to the ED or hospital for concerns related to CHMs, there may be important adverse financial and emotional consequences.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts.

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

Clinical Decision Support in the PICU: Implications for Design and Evaluation.

OBJECTIVES: To assess the current landscape of clinical decision support (CDS) tools in PICUs in order to identify priority areas of focus in this field. DESIGN: International, quantitative, cross-sectional survey. SETTING: Role-specific, web-based survey administered in November and December 2020. SUBJECTS: Medical directors, bedside nurses, attending physicians, and residents/advanced practice providers at Pediatric Acute Lung Injury and Sepsis Network-affiliated PICUs. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The survey was completed by 109 respondents from 45 institutions, primarily attending physicians from university-affiliated PICUs in the United States. The most commonly used CDS tools were people-based resources (93% used always or most of the time) and laboratory result highlighting (86%), with order sets, order-based alerts, and other electronic CDS tools also used frequently. The most important goal providers endorsed for CDS tools were a proven impact on patient safety and an evidence base for their use. Negative perceptions of CDS included concerns about diminished critical thinking and the burden of intrusive processes on providers. Routine assessment of existing CDS was rare, with infrequent reported use of observation to assess CDS impact on workflows or measures of individual alert burden. CONCLUSIONS: Although providers share some consensus over CDS utility, we identified specific priority areas of research focus. Consensus across practitioners exists around the importance of evidence-based CDS tools having a proven impact on patient safety. Despite broad presence of CDS tools in PICUs, practitioners continue to view them as intrusive and with concern for diminished critical thinking. Deimplementing ineffective CDS may mitigate this burden, though postimplementation evaluation of CDS is rare.

Leveraging EHR Data to Evaluate the Association of Late Recognition of Deterioration With Outcomes.

OBJECTIVES: Emergency transfers (ETs), deterioration events with late recognition requiring ICU interventions within 1 hour of transfer, are associated with adverse outcomes. We leveraged electronic health record (EHR) data to assess the association between ETs and outcomes. We also evaluated the association between intervention timing (urgency) and outcomes. METHODS: We conducted a propensity-score-matched study of hospitalized children requiring ICU transfer between 2015 and 2019 at a single institution. The primary exposure was ET, automatically classified using Epic Clarity Data stored in our enterprise data warehouse endotracheal tube in lines/drains/airway flowsheet, vasopressor in medication administration record, and/or ≥60 ml/kg intravenous fluids in intake/output flowsheets recorded within 1 hour of transfer. Urgent intervention was defined as interventions within 12 hours of transfer. RESULTS: Of 2037 index transfers, 129 (6.3%) met ET criteria. In the propensity-score-matched cohort (127 ET, 374 matched controls), ET was associated with higher in-hospital mortality (13% vs 6.1%; odds ratio, 2.47; 95% confidence interval [95% CI], 1.24-4.9, P = .01), longer ICU length of stay (subdistribution hazard ratio of ICU discharge 0.74; 95% CI, 0.61-0.91, P < .01), and longer posttransfer length of stay (SHR of hospital discharge 0.71; 95% CI, 0.56-0.90, P < .01). Increased intervention urgency was associated with increased mortality risk: 4.1% no intervention, 6.4% urgent intervention, and 10% emergent intervention. CONCLUSIONS: An EHR measure of deterioration with late recognition is associated with increased mortality and length of stay. Mortality risk increased with intervention urgency. Leveraging EHR automation facilitates generalizability, multicenter collaboratives, and metric consistency.

The Alarm Burden of Excess Continuous Pulse Oximetry Monitoring Among Patients With Bronchiolitis.

Guidelines discourage continuous pulse oximetry monitoring of hospitalized infants with bronchiolitis who are not receiving supplemental oxygen. Excess monitoring is theorized to contribute to increased alarm burden, but this burden has not been quantified. We evaluated admissions of 201 children (aged 0-24 months) with bronchiolitis. We categorized time ≥60 minutes following discontinuation of supplemental oxygen as "continuously monitored (guideline-discordant)," "intermittently measured (guideline-concordant)," or "unable to classify." Across 4402 classifiable hours, 77% (11,101) of alarms occurred during periods of guideline-discordant monitoring. Patients experienced a median of 35 alarms (interquartile range [IQR], 10-81) during guideline-discordant, continuously monitored time, representing a rate of 6.7 alarms per hour (IQR, 2.1-12.3). In comparison, the median hourly alarm rate during periods of guideline-concordant intermittent measurement was 0.5 alarms per hour (IQR, 0.1-0.8). Reducing guideline-discordant monitoring in bronchiolitis patients would reduce nurse alarm burden.

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics.

BACKGROUND: Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden. OBJECTIVE: (1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics. MATERIALS AND METHODS: We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric. RESULTS: Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden. CONCLUSION: Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.

EHR-Integrated Monitor Data to Measure Pulse Oximetry Use in Bronchiolitis.

BACKGROUND AND OBJECTIVES: Continuous pulse oximetry (oxygen saturation [Spo2]) monitoring in hospitalized children with bronchiolitis not requiring supplemental oxygen is discouraged by national guidelines, but determining monitoring status accurately requires in-person observation. Our objective was to determine if electronic health record (EHR) data can accurately estimate the extent of actual Spo2 monitoring use in bronchiolitis. METHODS: This repeated cross-sectional study included infants aged 8 weeks through 23 months hospitalized with bronchiolitis. In the validation phase at 3 children's hospitals, we calculated the test characteristics of the Spo2 monitor data streamed into the EHR each minute when monitoring was active compared with in-person observation of Spo2 monitoring use. In the application phase at 1 children's hospital, we identified periods when supplemental oxygen was administered using EHR flowsheet documentation and calculated the duration of Spo2 monitoring that occurred in the absence of supplemental oxygen. RESULTS: Among 668 infants at 3 hospitals (validation phase), EHR-integrated Spo2 data from the same minute as in-person observation had a sensitivity of 90%, specificity of 98%, positive predictive value of 88%, and negative predictive value of 98% for actual Spo2 monitoring use. Using EHR-integrated data in a sample of 317 infants at 1 hospital (application phase), infants were monitored in the absence of oxygen supplementation for a median 4.1 hours (interquartile range 1.4-9.4 hours). Those who received supplemental oxygen experienced a median 5.6 hours (interquartile range 3.0-10.6 hours) of monitoring after oxygen was stopped. CONCLUSIONS: EHR-integrated monitor data are a valid measure of actual Spo2 monitoring use that may help hospitals more efficiently identify opportunities to deimplement guideline-inconsistent use.

Physiologic Monitor Alarm Burden and Nurses' Subjective Workload in a Children's Hospital.

BACKGROUND AND OBJECTIVES: Physiologic monitor alarms occur at high rates in children's hospitals; ≤1% are actionable. The burden of alarms has implications for patient safety and is challenging to measure directly. Nurse workload, measured by using a version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) validated among nurses, is a useful indicator of work burden that has been associated with patient outcomes. A recent study revealed that 5-point increases in the NASA-TLX score were associated with a 22% increased risk in missed nursing care. Our objective was to measure the relationship between alarm count and nurse workload by using the NASA-TLX. METHODS: We conducted a repeated cross-sectional study of pediatric nurses in a tertiary care children's hospital to measure the association between NASA-TLX workload evaluations (using the nurse-validated scale) and alarm count in the 2 hours preceding NASA-TLX administration. Using a multivariable mixed-effects regression accounting for nurse-level clustering, we modeled the adjusted association of alarm count with workload. RESULTS: The NASA-TLX score was assessed in 26 nurses during 394 nursing shifts over a 2-month period. In adjusted regression models, experiencing >40 alarms in the preceding 2 hours was associated with a 5.5 point increase (95% confidence interval 5.2 to 5.7; P < .001) in subjective workload. CONCLUSION: Alarm count in the preceding 2 hours is associated with a significant increase in subjective nurse workload that exceeds the threshold associated with increased risk of missed nursing care and potential patient harm.

Analysis: Protocol for a New Method to Measure Physiologic Monitor Alarm Responsiveness.

Evaluating the clinical impacts of healthcare alarm management systems plays a critical role in assessing newly implemented monitoring technology, exposing latent threats to patient safety, and identifying opportunities for system improvement. We describe a novel, accurate, rapidly implementable, and readily reproducible in situ simulation approach to measure alarm response times and rates without the challenges and expense of video analysis. An interprofessional team consisting of biomedical engineers, human factors engineers, information technology specialists, nurses, physicians, facilitators from the hospital's simulation center, clinical informaticians, and hospital administrative leadership worked with three units at a pediatric hospital to design and conduct the simulations. Existing hospital technology was used to transmit a simulated, unambiguously critical alarm that appeared to originate from an actual patient to the nurse's mobile device, and discreet observers measured responses. Simulation observational data can be used to design and evaluate quality improvement efforts to address alarm responsiveness and to benchmark performance of different alarm communication systems.