Effect of Automated Real-Time Feedback on Early-Sepsis Care: A Pragmatic Clinical Trial.

OBJECTIVES: To determine if a real-time monitoring system with automated clinician alerts improves 3-hour sepsis bundle adherence. DESIGN: Prospective, pragmatic clinical trial. Allocation alternated every 7 days. SETTING: Quaternary hospital from December 1, 2020 to November 30, 2021. PATIENTS: Adult emergency department or inpatients meeting objective sepsis criteria triggered an electronic medical record (EMR)-embedded best practice advisory. Enrollment occurred when clinicians acknowledged the advisory indicating they felt sepsis was likely. INTERVENTION: Real-time automated EMR monitoring identified suspected sepsis patients with incomplete bundle measures within 1-hour of completion deadlines and generated reminder pages. Clinicians responsible for intervention group patients received reminder pages; no pages were sent for controls. The primary analysis cohort was the subset of enrolled patients at risk of bundle nonadherent care that had reminder pages generated. MEASUREMENTS AND MAIN RESULTS: The primary outcome was orders for all 3-hour bundle elements within guideline time limits. Secondary outcomes included guideline-adherent delivery of all 3-hour bundle elements, 28-day mortality, antibiotic discontinuation within 48-hours, and pathogen recovery from any culture within 7 days of time-zero. Among 3,269 enrolled patients, 1,377 had reminder pages generated and were included in the primary analysis. There were 670 (48.7%) at-risk patients randomized to paging alerts and 707 (51.3%) to control. Bundle-adherent orders were placed for 198 intervention patients (29.6%) versus 149 (21.1%) controls (difference: 8.5%; 95% CI, 3.9-13.1%; p = 0.0003). Bundle-adherent care was delivered for 152 (22.7%) intervention versus 121 (17.1%) control patients (difference: 5.6%; 95% CI, 1.4-9.8%; p = 0.0095). Mortality was similar between groups (8.4% vs 8.3%), as were early antibiotic discontinuation (35.1% vs 33.4%) and pan-culture negativity (69.0% vs 68.2%). CONCLUSIONS: Real-time monitoring and paging alerts significantly increased orders for and delivery of guideline-adherent care for suspected sepsis patients at risk of 3-hour bundle nonadherence. The trial was underpowered to determine whether adherence affected mortality. Despite enrolling patients with clinically suspected sepsis, early antibiotic discontinuation and pan-culture negativity were common, highlighting challenges in identifying appropriate patients for sepsis bundle application.

Result Push Notifications Improve Time to Emergency Department Disposition: A Pragmatic Observational Study.

STUDY OBJECTIVE: Emergency department (ED) crowding has multiple causative factors, including delayed patient throughput. Patient care efficiency may be improved by addressing delays in decisionmaking following diagnostic testing results. We examined the influence of sending subscribed result push notifications to ED clinicians' smartphones on reducing the time to disposition decision. RESULTS: All ED patient visits between October 2022 and October 2023 with a laboratory or imaging result during the ED visit and a disposition within 6 hours of the last result were included. We identified whether the last resulted study before the ED disposition decision had a subscribed push notification by the clinician who dispositioned the patient. The primary outcome was the time between the last study result and the first disposition decision. Generalized estimating equation analysis was used to control for variables including patient demographics, clinical factors, and discharging clinician. RESULTS: The final study population included 237,872 encounters. The median patient age was 50 years, and 55.6% of patients were women. During the study period, 27.1% of clinicians used push notifications at least once. Of unique orders, 1.5% had a subscribed result push notification, including 0.9% of laboratory orders and 4.7% of imaging orders. The time between last result to disposition decision was 18 minutes (95% confidence interval [CI] 15 to 21) faster when a push notification was requested. CONCLUSION: Elective push notification of test results was associated with reduced time between the last laboratory or imaging result and ED disposition decision. Further study is needed to determine its effect on overall ED throughput.

Variability in Emergency Department Procedure Rates and Distributions in a Regional Health System: A Cross-Sectional Observational Study.

STUDY OBJECTIVE: Procedural competency is essential to the practice of emergency medicine. However, there are limited data quantifying emergency department procedural volumes to inform the work of educators and credentialing bodies. In this study, we characterize procedural scope and volume in a regional health care system and compare rates between practice settings and over time. METHODS: Cross-sectional data were acquired from electronic medical records of a regional health care system from March 2017 through February 2022. Nonspecific entries, esoteric procedures, and nonprocedural clinical skills were excluded. Procedural rates were compared: (1) between academic and community hospitals, (2) across study years, and (3) across seasons. Analyses were repeated for pediatric encounters, and with study year 4 removed to assess the influence of the first year of the coronavirus disease 2019 pandemic on results. RESULTS: There were 131,976 instances of 40 qualifying procedures in 1,979,935 unique visits across 9 EDs. Several high-acuity procedures had similar rates in academic and community settings, including cardiac pacing, cricothyrotomy, and lateral canthotomy. Year-over-year procedural rates were stable or increasing for most procedures, with a notable exception of lumbar puncture. Most procedures did not have significant seasonal variation, and most findings were stable when study year 4 was removed from the analysis. CONCLUSION: All procedures were performed in all settings and rates of several emergent procedures were similar in both settings, underscoring the importance of broad procedural competence for all emergency physicians. Educators and credentialing organizations can use these data to inform decisions regarding curriculum design and certification requirements.

Prospective External Validation of a Commercial Model Predicting the Likelihood of Inpatient Admission From the Emergency Department.

STUDY OBJECTIVE: Early notification of admissions from the emergency department (ED) may allow hospitals to plan for inpatient bed demand. This study aimed to assess Epic's ED Likelihood to Occupy an Inpatient Bed predictive model and its application in improving hospital bed planning workflows. METHODS: All ED adult (18 years and older) visits from September 2021 to August 2022 at a large regional health care system were included. The primary outcome was inpatient admission. The predictive model is a random forest algorithm that uses demographic and clinical features. The model was implemented prospectively, with scores generated every 15 minutes. The area under the receiver operator curves (AUROC) and precision-recall curves (AUPRC) were calculated using the maximum score prior to the outcome and for each prediction independently. Test characteristics and lead time were calculated over a range of model score thresholds. RESULTS: Over 11 months, 329,194 encounters were evaluated, with an incidence of inpatient admission of 25.4%. The encounter-level AUROC was 0.849 (95% confidence interval [CI], 0.848 to 0.851), and the AUPRC was 0.643 (95% CI, 0.640 to 0.647). With a prediction horizon of 6 hours, the AUROC was 0.758 (95% CI, 0.758 to 0.759,) and the AUPRC was 0.470 (95% CI, 0.469 to 0.471). At a predictive model threshold of 40, the sensitivity was 0.49, the positive predictive value was 0.65, and the median lead-time warning was 127 minutes before the inpatient bed request. CONCLUSION: The Epic ED Likelihood to Occupy an Inpatient Bed model may improve hospital bed planning workflows. Further study is needed to determine its operational effect.

Implementation of an Electronic Alert to Improve Timeliness of Second Dose Antibiotics for Patients With Suspected Serious Infections in the Emergency Department: A Quasi-Randomized Controlled Trial.

STUDY OBJECTIVE: Delays in the second dose of antibiotics in the emergency department (ED) are associated with increased morbidity and mortality in patients with serious infections. We analyzed the influence of clinical decision support to prevent delays in second doses of broad-spectrum antibiotics in the ED. METHODS: We allocated adult patients who received cefepime or piperacillin/tazobactam in 9 EDs within an integrated health care system to an electronic alert that reminded ED clinicians to reorder antibiotics at the appropriate interval vs usual care. The primary outcome was a median delay in antibiotic administration. Secondary outcomes were rates of intensive care unit (ICU) admission, hospital mortality, and hospital length of stay. We included a post hoc secondary outcome of frequency of major delay (>25% of expected interval for second antibiotic dose). RESULTS: A total of 1,113 ED patients treated with cefepime or piperacillin/tazobactam were enrolled in the study, of whom 420 remained under ED care when their second dose was due and were included in the final analysis. The clinical decision support tool was associated with reduced antibiotic delays (median difference 35 minutes, 95% confidence interval [CI], 5 to 65). There were no differences in ICU transfers, inpatient mortality, or hospital length of stay. The clinical decision support tool was associated with decreased probability of major delay (absolute risk reduction 13%, 95% CI, 6 to 20). CONCLUSIONS: The implementation of a clinical decision support alert reminding clinicians to reorder second doses of antibiotics was associated with a reduction in the length and frequency of antibiotic delays in the ED. There was no effect on the rates of ICU transfers, inpatient mortality, or hospital length of stay.

COVID-19 Seroprevalence in ED Health Care Professionals Study: A Cross-Sectional Study.

INTRODUCTION: ED health care professionals are at the frontline of evaluation and management of patients with acute, and often undifferentiated, illness. During the initial phase of the SARS-CoV-2 outbreak, there were concerns that ED health care professionals may have been at increased risk of exposure to SARS-CoV-2 due to difficulty in early identification of patients. This study assessed the seroprevalence of SARS-CoV-2 antibodies among ED health care professionals without confirmed history of COVID-19 infection at a quaternary academic medical center. METHODS: This study used a cross-sectional design. An ED health care professional was deemed eligible if they had worked at least 4 shifts in the adult emergency department from April 1, 2020, through May 31, 2020, were asymptomatic on the day of blood draw, and were not known to have had prior documented COVID-19 infection. The study period was December 17, 2020, to January 27, 2021. Eligible participants completed a questionnaire and had a blood sample drawn. Samples were run on the Roche Cobas Elecsys Anti-SARS-CoV-2 antibody assay. RESULTS: Of 103 health care professionals (16 attending physicians, 4 emergency residents, 16 advanced practice professionals, and 67 full-time emergency nurses), only 3 (2.9%; exact 95% CI, 0.6%-8.3%) were seropositive for SARS-CoV-2 antibodies. DISCUSSION: At this quaternary academic medical center, among those who volunteered to take an antibody test, there was a low seroprevalence of SARS-CoV-2 antibodies among ED clinicians who were asymptomatic at the time of blood draw and not known to have had prior COVID-19 infection.

Surveillance for Healthcare-Associated Infections: Hospital-Onset Adult Sepsis Events Versus Current Reportable Conditions.

BACKGROUND: US hospitals are required by the Centers for Medicare and Medicaid Services to publicly report central line-associated bloodstream infections (CLABSIs), catheter-associated urinary tract infections (CAUTIs), Clostridioidesdiffficile, methicillin-resistant Staphylococcus aureus bacteremia, and selected surgical site infections for benchmarking and pay-for-performance programs. It is unclear, however, to what extent these conditions capture the full breadth of serious healthcare-associated infections (HAIs). The Centers for Disease Control and Prevention's (CDC's) hospital-onset Adult Sepsis Event (HO-ASE) definition could facilitate more comprehensive and efficient surveillance for serious HAIs, but the overlap between HO-ASE and currently reportable HAIs is unknown. METHODS: We retrospectively assessed the overlap between HO-ASEs and reportable HAIs among adults hospitalized between June 2015-June 2018 in 3 hospitals. Medical record reviews were conducted for 110 randomly selected HO-ASE cases to determine clinical correlates. RESULTS: Among 282 441 hospitalized patients, 2301 (0.8%) met HO-ASE criteria and 1260 (0.4%) had reportable HAIs. In-hospital mortality rates were higher with HO-ASEs than reportable HAIs (28.6% vs 12.9%). Mortality rates for HO-ASE missed by reportable HAIs were substantially higher than mortality rates for reportable HAIs missed by HO-ASE (28.1% vs 6.3%). Reportable HAIs were only present in 334/2301 (14.5%) HO-ASEs, most commonly CLABSIs (6.0% of HO-ASEs), C. difficile (5.0%), and CAUTIs (3.0%). On medical record review, most HO-ASEs were caused by pneumonia (39.1%, of which only 34.9% were ventilator-associated), bloodstream infections (17.4%, of which only 10.5% were central line-associated), non-C. difficile intra-abdominal infections (14.5%), urinary infections (7.3%, of which 87.5% were catheter-associated), and skin/soft tissue infections (6.4%). CONCLUSIONS: CDC's HO-ASE definition detects many serious nosocomial infections missed by currently reportable HAIs. HO-ASE surveillance could increase the efficiency and clinical significance of surveillance while identifying new targets for prevention.

Coronavirus Disease 2019 (COVID-19) Diagnostic Clinical Decision Support: A Pre-Post Implementation Study of CORAL (COvid Risk cALculator).

BACKGROUND: Isolation of hospitalized persons under investigation (PUIs) for coronavirus disease 2019 (COVID-19) reduces nosocomial transmission risk. Efficient evaluation of PUIs is needed to preserve scarce healthcare resources. We describe the development, implementation, and outcomes of an inpatient diagnostic algorithm and clinical decision support system (CDSS) to evaluate PUIs. METHODS: We conducted a pre-post study of CORAL (COvid Risk cALculator), a CDSS that guides frontline clinicians through a risk-stratified COVID-19 diagnostic workup, removes transmission-based precautions when workup is complete and negative, and triages complex cases to infectious diseases (ID) physician review. Before CORAL, ID physicians reviewed all PUI records to guide workup and precautions. After CORAL, frontline clinicians evaluated PUIs directly using CORAL. We compared pre- and post-CORAL frequency of repeated severe acute respiratory syndrome coronavirus 2 nucleic acid amplification tests (NAATs), time from NAAT result to PUI status discontinuation, total duration of PUI status, and ID physician work hours, using linear and logistic regression, adjusted for COVID-19 incidence. RESULTS: Fewer PUIs underwent repeated testing after an initial negative NAAT after CORAL than before CORAL (54% vs 67%, respectively; adjusted odd ratio, 0.53 [95% confidence interval, .44-.63]; P < .01). CORAL significantly reduced average time to PUI status discontinuation (adjusted difference [standard error], -7.4 [0.8] hours per patient), total duration of PUI status (-19.5 [1.9] hours per patient), and average ID physician work-hours (-57.4 [2.0] hours per day) (all P < .01). No patients had a positive NAAT result within 7 days after discontinuation of precautions via CORAL. CONCLUSIONS: CORAL is an efficient and effective CDSS to guide frontline clinicians through the diagnostic evaluation of PUIs and safe discontinuation of precautions.

Clinical Decision Support Reduces Unnecessary Tetanus Vaccinations in the Emergency Department.

STUDY OBJECTIVE: Tetanus is the most common vaccination given in the emergency department; yet, administrations of tetanus vaccine boosters in the ED may not comply with the US Centers for Disease Control and Prevention's recommended vaccination schedule. We implemented a clinical decision support alert in the electronic health record that warned providers when ordering a tetanus vaccine if a prior one had been given within 10 years and studied its efficacy to reduce potentially unnecessary vaccines in the ED. METHODS: This was a retrospective, quasi-experimental, 1-group, pretest-posttest study in 3 hospital EDs in Boston, MA. We studied adult patients for whom tetanus vaccines were ordered despite a history of vaccination within the prior 10 years. We compared the number of potentially unnecessary tetanus vaccine administrations in a baseline phase (when the clinical decision support alert was not visible) versus an intervention phase. RESULTS: Of eligible patients, 22.1% (95% confidence interval [CI] 21.8% to 22.4%) had prior tetanus vaccines within 5 years, 12.8% (95% CI 12.5% to 13.0%) within 5 to 10 years, 3.8% (95% CI 3.6% to 3.9%) more than 10 years ago, and 61.3% (95% CI 60.9% to 61.7%) had no prior tetanus vaccination documentation. Of 60,983 encounters, 337 met the inclusion criteria. A tetanus vaccination was administered in 91% (95% CI 87% to 96%) of encounters in the baseline phase, compared to 55% (95% CI 47% to 62%) during the intervention. The absolute risk reduction was 36.7% (95% CI 28.0% to 45.4%), and the number of encounters needed to alert to avoid 1 potentially unnecessary tetanus vaccine (number needed to treat) was 2.7 (95% CI 2.2% to 3.6%). For patients with tetanus vaccines within the prior 5 years, the absolute risk reduction was 47.9% (95% CI 35.5 % to 60.3%) and the number needed to treat was 2.1 (95% CI 1.7% to 2.8%). CONCLUSION: A clinical decision support alert that warns ED clinicians that a patient may have an up-to-date tetanus vaccination status reduces potentially unnecessary vaccinations.

Sedation - Effects of disorders of abuse on therapeutic efficacy (SEDATE): A retrospective cohort study.

BACKGROUND: The impact of alcohol or opioid use disorders on medication dosing for procedural sedation in the emergency department (ED) is unclear, as most of the literature is from gastrointestinal endoscopy. Exploring how these patient factors affect sedative and analgesic medications may inform more nuanced sedation strategies in the emergency department. METHODS: This was a retrospective chart-review cohort study across five EDs from 2015 to 2020. Included were adult patients who underwent procedural sedation in the ED, categorized into three a priori groups: alcohol use disorder (AUD), opioid use disorder (OUD), and individuals with neither (non-SUD). Wilcoxon test was used to compare the time-averaged dose of agents between groups. Logistic regression was used to model multi-agent sedations. The propofol time-averaged dose was the primary outcome. Secondary outcomes included other agents, sedation duration, and switching to other agents. RESULTS: 2725 sedations were included in the analysis. 59 patients had a history of AUD, and 40 had a history of OUD. Time-averaged doses of medications did not differ significantly between AUD and non-SUD patients. Likewise, patients with OUD did not receive different doses of medications compared to non-SUD. The propofol doses for non-SUD, AUD, and OUD were 0.033 IQR 0.04; 0.042 IQR 0.05; and 0.058 IQR 0.04 mg/kg*min, respectively. Sedation duration was not different across groups. Having AUD or OUD is not associated with increased odds of requiring multiple sedative agents. CONCLUSION: Although sedation in patients with AUD or OUD may be associated with significant case bias, these patient factors did not significantly alter outcomes compared to the general population. This study suggests there is no evidence to proactively adjust medication strategy in ED patients with AUD or OUD.

Limited English Proficiency and Sepsis Mortality by Race and Ethnicity.

Importance: Patients with limited English proficiency (LEP) face multiple barriers and are at risk for worse health outcomes compared with patients with English proficiency (EP). In sepsis, a major cause of mortality in the US, the association of LEP with health outcomes is not widely explored. Objective: To assess the association between LEP and inpatient mortality among patients with sepsis and test the hypothesis that LEP would be associated with higher mortality rates. Design, Setting, and Participants: This retrospective cohort study of hospitalized patients with sepsis included those who met the Centers for Disease Control and Prevention's sepsis criteria, received antibiotics within 24 hours, and were admitted through the emergency department. Data were collected from the electronic medical records of a large New England tertiary care center from January 1, 2016, to December 31, 2019. Data were analyzed from January 8, 2021, to March 2, 2023. Exposures: Limited English proficiency, gathered via self-reported language preference in electronic medical records. Main Outcomes and Measures: The primary outcome was inpatient mortality. The analysis used multivariable generalized estimating equation models with propensity score adjustment and analysis of covariance to analyze the association between LEP and inpatient mortality due to sepsis. Results: A total of 2709 patients met the inclusion criteria, with a mean (SD) age of 65.0 (16.2) years; 1523 (56.2%) were men and 327 (12.1%) had LEP. Nine patients (0.3%) were American Indian or Alaska Native, 101 (3.7%) were Asian, 314 (11.6%) were Black, 226 (8.3%) were Hispanic, 38 (1.4%) were Native Hawaiian or Other Pacific Islander or of other race or ethnicity, 1968 (72.6%) were White, and 6 (0.2%) were multiracial. Unadjusted mortality included 466 of 2382 patients with EP (19.6%) and 69 of 327 with LEP (21.1%). No significant difference was found in mortality odds for the LEP compared with EP groups (odds ratio [OR], 1.12 [95% CI, 0.88-1.42]). When stratified by race and ethnicity, odds of inpatient mortality for patients with LEP were significantly higher among the non-Hispanic White subgroup (OR, 1.76 [95% CI, 1.41-2.21]). This significant difference was also present in adjusted analyses (adjusted OR, 1.56 [95% CI, 1.02-2.39]). No significant differences were found in inpatient mortality between LEP and EP in the racial and ethnic minority subgroup (OR, 0.99 [95% CI, 0.63-1.58]; adjusted OR, 0.91 [95% CI, 0.56-1.48]). Conclusions and Relevance: In a large diverse academic medical center, LEP had no significant association overall with sepsis mortality. In a subgroup analysis, LEP was associated with increased mortality among individuals identifying as non-Hispanic White. This finding highlights a potential language-based inequity in sepsis care. Further studies are needed to understand drivers of this inequity, how it may manifest in other diverse health systems, and to inform equitable care models for patients with LEP.

Second Time's the Charm? Assessing the Sensitivity and Yield of Inpatient Diagnostic Algorithms for Pulmonary Tuberculosis in a Low-Prevalence Setting.

Background: For persons with suspected pulmonary tuberculosis, the guidelines of the Centers for Disease Control and Prevention recommend collecting 3 respiratory specimens 8 to 24 hours apart for acid-fast bacilli (AFB) smear and culture, in addition to 1 nucleic acid amplification test (NAAT). However, data supporting this approach are limited. Our objective was to estimate the performance of 1, 2, or 3 AFB smears with or without NAATs to detect pulmonary tuberculosis in a low-prevalence setting. Methods: We conducted a retrospective study of hospitalized persons at 8 Massachusetts acute care facilities who underwent mycobacterial culture on 1 or more respiratory specimens between July 2016 and December 2022. We evaluated percentage positivity and yield on serial AFB smears and NAATs among people with growth of Mycobacterium tuberculosis on mycobacterial cultures. Results: Among 104 participants with culture-confirmed pulmonary tuberculosis, the first AFB smear was positive in 41 cases (39%). A second AFB smear was positive in 11 (22%) of the 49 cases in which it was performed. No third AFB smears were positive following 2 initial negative smears. Of 52 smear-negative cases, 36 had a NAAT performed, leading to 23 additional diagnoses. Overall sensitivity to detect tuberculosis prior to culture positivity was higher in any strategy involving 1 or 2 NAATs (74%-79%), even without AFB smears, as compared with 3 smears alone (60%). Conclusions: Tuberculosis diagnostic testing with 2 AFB smears offered the same yield as 3 AFB smears while potentially reducing laboratory burden and duration of airborne infection isolation. Use of 1 or 2 NAATs increased sensitivity to detect culture-positive pulmonary tuberculosis when added to AFB smear-based diagnostic testing alone.

Separating the rash from the chaff: novel clinical decision support deployed during the mpox outbreak.

A clinical decision support system, EvalMpox, was developed to apply person under investigation (PUI) criteria for patients presenting with rash and to recommend testing for PUIs. Of 668 patients evaluated, an EvalMpox recommendation for testing had a positive predictive value of 35% and a negative predictive value of 99% for a positive mpox test.

Use of Clinical Decision Support to Improve the Laboratory Evaluation of Monoclonal Gammopathies.

OBJECTIVES: There is considerable variation in ordering practices for the initial laboratory evaluation of monoclonal gammopathies (MGs) despite clear society guidelines to include serum free light chain (sFLC) testing. We assessed the ability of a clinical decision support (CDS) alert to improve guideline compliance and analyzed its clinical impact. METHODS: We designed and deployed a targeted CDS alert to educate and prompt providers to order an sFLC assay when ordering serum protein electrophoresis (SPEP) testing. RESULTS: The alert was highly effective at increasing the co-ordering of SPEP and sFLC testing. Preimplementation, 62.8% of all SPEP evaluations included sFLC testing, while nearly 90% of evaluations included an sFLC assay postimplementation. In patients with no prior sFLC testing, analysis of sFLC orders prompted by the alert led to the determination that 28.9% (800/2,769) of these patients had an abnormal κ/λ ratio. In 452 of these patients, the sFLC assay provided the only laboratory evidence of a monoclonal protein. Moreover, within this population, there were numerous instances of new diagnoses of multiple myeloma and other MGs. CONCLUSIONS: The CDS alert increased compliance with society guidelines and improved the diagnostic evaluation of patients with suspected MGs.

Clinical decision support improves blood culture collection before intravenous antibiotic administration in the emergency department.

OBJECTIVE: Surviving Sepsis guidelines recommend blood cultures before administration of intravenous (IV) antibiotics for patients with sepsis or moderate to high risk of bacteremia. Clinical decision support (CDS) that reminds emergency department (ED) providers to obtain blood cultures when ordering IV antibiotics may lead to improvements in this process measure. METHODS: This was a multicenter causal impact analysis comparing timely blood culture collections prior to IV antibiotics for adult ED patients 1 year before and after a CDS intervention implementation in the electronic health record. A Bayesian structured time-series model compared daily timely blood cultures collected compared to a forecasted synthetic control. Mixed effects models evaluated the impact of the intervention controlling for confounders. RESULTS: The analysis included 54 538 patients over 2 years. In the baseline phase, 46.1% had blood cultures prior to IV antibiotics, compared to 58.8% after the intervention. Causal impact analysis determined an absolute increase of 13.1% (95% CI 10.4-15.7%) of timely blood culture collections overall, although the difference in patients with a sepsis diagnosis or who met CDC Adult Sepsis Event criteria was not significant, absolute difference 8.0% (95% CI -0.2 to 15.8). Blood culture positivity increased in the intervention phase, and contamination rates were similar in both study phases. DISCUSSION: CDS improved blood culture collection before IV antibiotics in the ED, without increasing overutilization. CONCLUSION: A simple CDS alert increased timely blood culture collections in ED patients for whom concern for infection was high enough to warrant IV antibiotics.

Real-Time User Feedback to Support Clinical Decision Support System Improvement.

OBJECTIVES: To improve clinical decision support (CDS) by allowing users to provide real-time feedback when they interact with CDS tools and by creating processes for responding to and acting on this feedback. METHODS: Two organizations implemented similar real-time feedback tools and processes in their electronic health record and gathered data over a 30-month period. At both sites, users could provide feedback by using Likert feedback links embedded in all end-user facing alerts, with results stored outside the electronic health record, and provide feedback as a comment when they overrode an alert. Both systems are monitored daily by clinical informatics teams. RESULTS: The two sites received 2,639 Likert feedback comments and 623,270 override comments over a 30-month period. Through four case studies, we describe our use of end-user feedback to rapidly respond to build errors, as well as identifying inaccurate knowledge management, user-interface issues, and unique workflows. CONCLUSION: Feedback on CDS tools can be solicited in multiple ways, and it contains valuable and actionable suggestions to improve CDS alerts. Additionally, end users appreciate knowing their feedback is being received and may also make other suggestions to improve the electronic health record. Incorporation of end-user feedback into CDS monitoring, evaluation, and remediation is a way to improve CDS.

Early identification of patients admitted to hospital for covid-19 at risk of clinical deterioration: model development and multisite external validation study.

OBJECTIVE: To create and validate a simple and transferable machine learning model from electronic health record data to accurately predict clinical deterioration in patients with covid-19 across institutions, through use of a novel paradigm for model development and code sharing. DESIGN: Retrospective cohort study. SETTING: One US hospital during 2015-21 was used for model training and internal validation. External validation was conducted on patients admitted to hospital with covid-19 at 12 other US medical centers during 2020-21. PARTICIPANTS: 33 119 adults (≥18 years) admitted to hospital with respiratory distress or covid-19. MAIN OUTCOME MEASURES: An ensemble of linear models was trained on the development cohort to predict a composite outcome of clinical deterioration within the first five days of hospital admission, defined as in-hospital mortality or any of three treatments indicating severe illness: mechanical ventilation, heated high flow nasal cannula, or intravenous vasopressors. The model was based on nine clinical and personal characteristic variables selected from 2686 variables available in the electronic health record. Internal and external validation performance was measured using the area under the receiver operating characteristic curve (AUROC) and the expected calibration error-the difference between predicted risk and actual risk. Potential bed day savings were estimated by calculating how many bed days hospitals could save per patient if low risk patients identified by the model were discharged early. RESULTS: 9291 covid-19 related hospital admissions at 13 medical centers were used for model validation, of which 1510 (16.3%) were related to the primary outcome. When the model was applied to the internal validation cohort, it achieved an AUROC of 0.80 (95% confidence interval 0.77 to 0.84) and an expected calibration error of 0.01 (95% confidence interval 0.00 to 0.02). Performance was consistent when validated in the 12 external medical centers (AUROC range 0.77-0.84), across subgroups of sex, age, race, and ethnicity (AUROC range 0.78-0.84), and across quarters (AUROC range 0.73-0.83). Using the model to triage low risk patients could potentially save up to 7.8 bed days per patient resulting from early discharge. CONCLUSION: A model to predict clinical deterioration was developed rapidly in response to the covid-19 pandemic at a single hospital, was applied externally without the sharing of data, and performed well across multiple medical centers, patient subgroups, and time periods, showing its potential as a tool for use in optimizing healthcare resources.

Algorithmic Detection of Boolean Logic Errors in Clinical Decision Support Statements.

OBJECTIVE: Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logic errors in CDS statements. METHODS: Nine health care organizations extracted Boolean logic statements from their Epic electronic health record (EHR). We developed an open-source software tool, which implemented the Espresso logic minimization algorithm, to identify three classes of logic errors. RESULTS: Participating organizations submitted 260,698 logic statements, of which 44,890 were minimized by Espresso. We found errors in 209 of them. Every participating organization had at least two errors, and all organizations reported that they would act on the feedback. DISCUSSION: An automated algorithm can readily detect specific categories of Boolean CDS logic errors. These errors represent a minority of CDS errors, but very likely require correction to avoid patient safety issues. This process found only a few errors at each site, but the problem appears to be widespread, affecting all participating organizations. CONCLUSION: Both CDS implementers and EHR vendors should consider implementing similar algorithms as part of the CDS authoring process to reduce the number of errors in their CDS interventions.

Iron studies and transferrin, a source of test ordering confusion highly amenable to clinical decision support.

INTRODUCTION: An important cause of laboratory test misordering and overutilization is clinician confusion between tests with similar sounding names or similar indications. We identified an area of test ordering confusion with iron studies that involves total iron binding capacity (TIBC), transferrin, and transferrin saturation. We observed concurrent ordering of direct transferrin along with TIBC at many hospitals within our health system and suspected this was unnecessary. METHODS: We extracted patient test results for transferrin, TIBC and other biomarkers. Using these data, we evaluated both patterns of test utilization and test result concordance. We implemented a clinical decision support (CDS) alert to discourage unnecessary orders for direct transferrin. RESULTS: Using linear regression, we were able to predict transferrin from either TIBC alone or TIBC with other analytes with a high degree of accuracy, demonstrating that in most cases, direct transferrin in combination with TIBC provides little if any additional diagnostic information beyond TIBC alone. The CDS alert proved highly effective in reducing transferrin test utilization at four different hospitals. CONCLUSIONS: Concurrent ordering of direct transferrin and TIBC should usually be avoided. Removal of transferrin or TIBC from the test menu or implementation of CDS may improve utilization of these tests.