Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings.

OBJECTIVES: To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings. MATERIALS AND METHODS: Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale. RESULTS: For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision. CONCLUSION: GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format.

Importance: By law, patients have immediate access to discharge notes in their medical records. Technical language and abbreviations make notes difficult to read and understand for a typical patient. Large language models (LLMs [eg, GPT-4]) have the potential to transform these notes into patient-friendly language and format. Objective: To determine whether an LLM can transform discharge summaries into a format that is more readable and understandable. Design, Setting, and Participants: This cross-sectional study evaluated a sample of the discharge summaries of adult patients discharged from the General Internal Medicine service at NYU (New York University) Langone Health from June 1 to 30, 2023. Patients discharged as deceased were excluded. All discharge summaries were processed by the LLM between July 26 and August 5, 2023. Interventions: A secure Health Insurance Portability and Accountability Act-compliant platform, Microsoft Azure OpenAI, was used to transform these discharge summaries into a patient-friendly format between July 26 and August 5, 2023. Main Outcomes and Measures: Outcomes included readability as measured by Flesch-Kincaid Grade Level and understandability using Patient Education Materials Assessment Tool (PEMAT) scores. Readability and understandability of the original discharge summaries were compared with the transformed, patient-friendly discharge summaries created through the LLM. As balancing metrics, accuracy and completeness of the patient-friendly version were measured. Results: Discharge summaries of 50 patients (31 female [62.0%] and 19 male [38.0%]) were included. The median patient age was 65.5 (IQR, 59.0-77.5) years. Mean (SD) Flesch-Kincaid Grade Level was significantly lower in the patient-friendly discharge summaries (6.2 [0.5] vs 11.0 [1.5]; P < .001). PEMAT understandability scores were significantly higher for patient-friendly discharge summaries (81% vs 13%; P < .001). Two physicians reviewed each patient-friendly discharge summary for accuracy on a 6-point scale, with 54 of 100 reviews (54.0%) giving the best possible rating of 6. Summaries were rated entirely complete in 56 reviews (56.0%). Eighteen reviews noted safety concerns, mostly involving omissions, but also several inaccurate statements (termed hallucinations). Conclusions and Relevance: The findings of this cross-sectional study of 50 discharge summaries suggest that LLMs can be used to translate discharge summaries into patient-friendly language and formats that are significantly more readable and understandable than discharge summaries as they appear in electronic health records. However, implementation will require improvements in accuracy, completeness, and safety. Given the safety concerns, initial implementation will require physician review.

Electronic Health Record Messaging Patterns of Health Care Professionals in Inpatient Medicine.

This cross-sectional study examines secure messaging patterns among health care professionals in the internal medicine inpatient setting.

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: Prospective Improvement Study.

BACKGROUND: The introduction of electronic workflows has allowed for the flow of raw uncontextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete clinically significant "signals." There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating. OBJECTIVE: This study aims to describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation. METHODS: This is a multihospital nonrandomized prospective improvement study conducted on the inpatient general internal medicine service across 3 hospital campuses at the New York University Langone Health System. A group of physician leaders representing each campus met biweekly for 6 months. The output of these meetings included (1) a conceptualization of the note bloat problem as a dysfunction in information flow, (2) a set of guiding principles for organizational documentation improvement, (3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations, and (4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience was measured using the System Usability Scale (SUS). Primary outcome measures after go-live include template utilization rate and note length in characters. RESULTS: In usability testing among 22 medicine providers, the new progress note template averaged a usability score of 90.6 out of 100 on the SUS. A total of 77% (17/22) of providers strongly agreed that the new template was easy to use, and 64% (14/22) strongly agreed that they would like to use the template frequently. In the 3 months after template implementation, general internal medicine providers wrote 67% (51,431/76,647) of all inpatient notes with the new templates. During this period, the organization saw a 46% (2768/6191), 47% (3505/7819), and 32% (3427/11,226) reduction in note length for general medicine progress notes, consults, and history and physical notes, respectively, when compared to a baseline measurement period prior to interventions. CONCLUSIONS: A bundled intervention that included the deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency, and patient outcomes.

Evaluating the Effect of a COVID-19 Predictive Model to Facilitate Discharge: A Randomized Controlled Trial.

BACKGROUND: We previously developed and validated a predictive model to help clinicians identify hospitalized adults with coronavirus disease 2019 (COVID-19) who may be ready for discharge given their low risk of adverse events. Whether this algorithm can prompt more timely discharge for stable patients in practice is unknown. OBJECTIVES: The aim of the study is to estimate the effect of displaying risk scores on length of stay (LOS). METHODS: We integrated model output into the electronic health record (EHR) at four hospitals in one health system by displaying a green/orange/red score indicating low/moderate/high-risk in a patient list column and a larger COVID-19 summary report visible for each patient. Display of the score was pseudo-randomized 1:1 into intervention and control arms using a patient identifier passed to the model execution code. Intervention effect was assessed by comparing LOS between intervention and control groups. Adverse safety outcomes of death, hospice, and re-presentation were tested separately and as a composite indicator. We tracked adoption and sustained use through daily counts of score displays. RESULTS: Enrolling 1,010 patients from May 15, 2020 to December 7, 2020, the trial found no detectable difference in LOS. The intervention had no impact on safety indicators of death, hospice or re-presentation after discharge. The scores were displayed consistently throughout the study period but the study lacks a causally linked process measure of provider actions based on the score. Secondary analysis revealed complex dynamics in LOS temporally, by primary symptom, and hospital location. CONCLUSION: An AI-based COVID-19 risk score displayed passively to clinicians during routine care of hospitalized adults with COVID-19 was safe but had no detectable impact on LOS. Health technology challenges such as insufficient adoption, nonuniform use, and provider trust compounded with temporal factors of the COVID-19 pandemic may have contributed to the null result. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04570488.

Quantitative and Qualitative Evaluation of Provider Use of a Novel Machine Learning Model for Favorable Outcome Prediction.

Predictive models may be particularly beneficial to clinicians when they face uncertainty and seek to develop a mental model of disease progression, but we know little about the post-implementation effects of predictive models on clinicians' experience of their work. Combining survey and interview methods, we found that providers using a predictive algorithm reported being significantly less uncertain and better able to anticipate, plan and prepare for patient discharge than non-users. The tool helped hospitalists form and develop confidence in their mental models of a novel disease (Covid-19). Yet providers' attention to the predictive tool declined as their confidence in their own mental models grew. Predictive algorithms that not only offer data but also provide feedback on decisions, thus supporting providers' motivation for continuous learning, hold promise for more sustained provider attention and cognition augmentation.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials.

BACKGROUND: Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools. OBJECTIVE: This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care. METHODS: A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior. RESULTS: To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images. CONCLUSIONS: These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS. TRIAL REGISTRATION: Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

Giving Your Electronic Health Record a Checkup After COVID-19: A Practical Framework for Reviewing Clinical Decision Support in Light of the Telemedicine Expansion.

BACKGROUND: The transformation of health care during COVID-19, with the rapid expansion of telemedicine visits, presents new challenges to chronic care and preventive health providers. Clinical decision support (CDS) is critically important to chronic care providers, and CDS malfunction is common during times of change. It is essential to regularly reassess an organization's ambulatory CDS program to maintain care quality. This is especially true after an immense change, like the COVID-19 telemedicine expansion. OBJECTIVE: Our objective is to reassess the ambulatory CDS program at a large academic medical center in light of telemedicine's expansion in response to the COVID-19 pandemic. METHODS: Our clinical informatics team devised a practical framework for an intrapandemic ambulatory CDS assessment focused on the impact of the telemedicine expansion. This assessment began with a quantitative analysis comparing CDS alert performance in the context of in-person and telemedicine visits. Board-certified physician informaticists then completed a formal workflow review of alerts with inferior performance in telemedicine visits. Informaticists then reported on themes and optimization opportunities through the existing CDS governance structure. RESULTS: Our assessment revealed that 10 of our top 40 alerts by volume were not firing as expected in telemedicine visits. In 3 of the top 5 alerts, providers were significantly less likely to take action in telemedicine when compared to office visits. Cumulatively, alerts in telemedicine encounters had an action taken rate of 5.3% (3257/64,938) compared to 8.3% (19,427/233,636) for office visits. Observations from a clinical informaticist workflow review included the following: (1) Telemedicine visits have different workflows than office visits. Some alerts developed for the office were not appearing at the optimal time in the telemedicine workflow. (2) Missing clinical data is a common reason for the decreased alert firing seen in telemedicine visits. (3) Remote patient monitoring and patient-reported clinical data entered through the portal could replace data collection usually completed in the office by a medical assistant or registered nurse. CONCLUSIONS: In a large academic medical center at the pandemic epicenter, an intrapandemic ambulatory CDS assessment revealed clinically significant CDS malfunctions that highlight the importance of reassessing ambulatory CDS performance after the telemedicine expansion.

Treating COVID-19 With Hydroxychloroquine (TEACH): A Multicenter, Double-Blind Randomized Controlled Trial in Hospitalized Patients.

BACKGROUND: Effective therapies to combat coronavirus 2019 (COVID-19) are urgently needed. Hydroxychloroquine (HCQ) has in vitro antiviral activity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but the clinical benefit of HCQ in treating COVID-19 is unclear. Randomized controlled trials are needed to determine the safety and efficacy of HCQ for the treatment of hospitalized patients with COVID-19. METHODS: We conducted a multicenter, double-blind randomized clinical trial of HCQ among patients hospitalized with laboratory-confirmed COVID-19. Subjects were randomized in a 1:1 ratio to HCQ or placebo for 5 days and followed for 30 days. The primary efficacy outcome was a severe disease progression composite end point (death, intensive care unit admission, mechanical ventilation, extracorporeal membrane oxygenation, and/or vasopressor use) at day 14. RESULTS: A total of 128 patients were included in the intention-to-treat analysis. Baseline demographic, clinical, and laboratory characteristics were similar between the HCQ (n = 67) and placebo (n = 61) arms. At day 14, 11 (16.4%) subjects assigned to HCQ and 6 (9.8%) subjects assigned to placebo met the severe disease progression end point, but this did not achieve statistical significance (P = .350). There were no significant differences in COVID-19 clinical scores, number of oxygen-free days, SARS-CoV-2 clearance, or adverse events between HCQ and placebo. HCQ was associated with a slight increase in mean corrected QT interval, an increased D-dimer, and a trend toward an increased length of stay. CONCLUSIONS: In hospitalized patients with COVID-19, our data suggest that HCQ does not prevent severe outcomes or improve clinical scores. However, our conclusions are limited by a relatively small sample size, and larger randomized controlled trials or pooled analyses are needed.

A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients.

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

The Financial and Clinical Impact of an Electronic Health Record Integrated Pathway in Elective Colon Surgery.

BACKGROUND: Enhanced Recovery after Surgery (ERAS) pathways have been shown to reduce length of stay, but there have been limited evaluations of novel electronic health record (EHR)-based pathways. Compliance with ERAS in real-world settings has been problematic. OBJECTIVE: This article evaluates a novel ERAS electronic pathway (E-Pathway) activity integrated with the EHR for patients undergoing elective colorectal surgery. METHODS: We performed a retrospective cohort study of surgical patients age ≥ 18 years hospitalized from March 1, 2013 to August 31, 2016. The primary cohort consisted of patients admitted for elective colon surgery. We also studied a control group of patients undergoing other elective procedures. The E-Pathway was implemented on March 2, 2015. The primary outcome was variable costs per case. Secondary outcomes were observed to expected length of stay and 30-day readmissions. RESULTS: We included 823 (470 and 353 in the pre- and postintervention, respectively) colon surgery patients and 3,415 (1,819 and 1,596 in the pre- and postintervention) surgical control patients in the study. Among the colon surgery cohort, there was statistically significant (p = 0.040) decrease in costs of 1.28% (95% confidence interval [CI] 0.06-2.48%) per surgical encounter per month over the 18-month postintervention period, amounting to a total savings of $2,730 per patient at the 1-year postintervention period. The surgical control group had a nonsignificant (p = 0.231) decrease in monthly costs of 0.57% (95% CI 1.51 to - 0.37%) postintervention. For the 30-day readmission rates, there were no statistically significant changes in either cohort. CONCLUSION: Our study is the first to report on the reduced costs after implementation of a novel sophisticated E-Pathway for ERAS. E-Pathways can be a powerful vehicle to support ERAS adoption.

Design and implementation of a clinical decision support tool for primary palliative Care for Emergency Medicine (PRIM-ER).

BACKGROUND: The emergency department is a critical juncture in the trajectory of care of patients with serious, life-limiting illness. Implementation of a clinical decision support (CDS) tool automates identification of older adults who may benefit from palliative care instead of relying upon providers to identify such patients, thus improving quality of care by assisting providers with adhering to guidelines. The Primary Palliative Care for Emergency Medicine (PRIM-ER) study aims to optimize the use of the electronic health record by creating a CDS tool to identify high risk patients most likely to benefit from primary palliative care and provide point-of-care clinical recommendations. METHODS: A clinical decision support tool entitled Emergency Department Supportive Care Clinical Decision Support (Support-ED) was developed as part of an institutionally-sponsored value based medicine initiative at the Ronald O. Perelman Department of Emergency Medicine at NYU Langone Health. A multidisciplinary approach was used to develop Support-ED including: a scoping review of ED palliative care screening tools; launch of a workgroup to identify patient screening criteria and appropriate referral services; initial design and usability testing via the standard System Usability Scale questionnaire, education of the ED workforce on the Support-ED background, purpose and use, and; creation of a dashboard for monitoring and feedback. RESULTS: The scoping review identified the Palliative Care and Rapid Emergency Screening (P-CaRES) survey as a validated instrument in which to adapt and apply for the creation of the CDS tool. The multidisciplinary workshops identified two primary objectives of the CDS: to identify patients with indicators of serious life limiting illness, and to assist with referrals to services such as palliative care or social work. Additionally, the iterative design process yielded three specific patient scenarios that trigger a clinical alert to fire, including: 1) when an advance care planning document was present, 2) when a patient had a previous disposition to hospice, and 3) when historical and/or current clinical data points identify a serious life-limiting illness without an advance care planning document present. Monitoring and feedback indicated a need for several modifications to improve CDS functionality. CONCLUSIONS: CDS can be an effective tool in the implementation of primary palliative care quality improvement best practices. Health systems should thoughtfully consider tailoring their CDSs in order to adapt to their unique workflows and environments. The findings of this research can assist health systems in effectively integrating a primary palliative care CDS system seamlessly into their processes of care. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03424109. Registered 6 February 2018, Grant Number: AT009844-01.

Interrupting providers with clinical decision support to improve care for heart failure.

BACKGROUND: Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions. OBJECTIVES: To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure. METHODS: Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS. RESULTS: Of 958 hospitalizations, interruptive alert hospitalizations had higher rates of discharge utilization of ACE inhibitors or ARBs than non-interruptive alert hospitalizations (79.6% vs. 74.2%, p = 0.05). Utilization was higher for interruptive alert versus non-interruptive alert hospitalizations which were principally for causes other than heart failure (79.8% vs. 73.4%; p = 0.05) but no difference was observed among hospitalizations with a principal heart failure diagnosis (85.9% vs.81.7%; p = 0.49). As compared to non-interruptive hospitalizations, interruptive alert hospitalizations were more likely to have had: an alert with any response (40.6% vs. 13.1%, p < 0.001), contraindications reported (33.1% vs 11.3%, p < 0.001), and an ACE inhibitor ordered within twelve hours of the alert (17.6% vs 10.3%, p < 0.01). The response rate for the interruptive alert was 1.7%, and a median (25th, 75th percentile) of 14 (5,32) alerts were triggered per hospitalization. CONCLUSIONS: A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration. CLINICALTRIALS. GOV IDENTIFIER: NCT02858674.

Interruptive Versus Noninterruptive Clinical Decision Support: Usability Study.

BACKGROUND: Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. OBJECTIVE: This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. METHODS: We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to "think aloud" while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. RESULTS: A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. CONCLUSIONS: Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay.

Objective: The purpose of this study was to determine whether an electronic health record-based sepsis alert system could improve quality of care and clinical outcomes for patients with sepsis. Materials and Methods: We performed a patient-level interrupted time series study of emergency department patients with severe sepsis or septic shock between January 2013 and April 2015. The intervention, introduced in February 2014, was a system of interruptive sepsis alerts triggered by abnormal vital signs or laboratory results. Primary outcomes were length of stay (LOS) and in-hospital mortality; other outcomes included time to first lactate and blood cultures prior to antibiotics. We also assessed sensitivity, positive predictive value (PPV), and clinician response to the alerts. Results: Mean LOS for patients with sepsis decreased from 10.1 to 8.6 days (P < .001) following alert introduction. In adjusted time series analysis, the intervention was associated with a decreased LOS of 16% (95% CI, 5%-25%; P = .007, with significance of α = 0.006) and no change thereafter (0%; 95% CI, -2%, 2%). The sepsis alert system had no effect on mortality or other clinical or process measures. The intervention had a sensitivity of 80.4% and a PPV of 14.6%. Discussion: Alerting based on simple laboratory and vital sign criteria was insufficient to improve sepsis outcomes. Alert fatigue due to the low PPV is likely the primary contributor to these results. Conclusion: A more sophisticated algorithm for sepsis identification is needed to improve outcomes.

System Changes to Implement the Joint Commission Tobacco Treatment (TOB) Performance Measures for Improving the Treatment of Tobacco Use Among Hospitalized Patients.

BACKGROUND: In 2012 The Joint Commission implemented new Tobacco Treatment (TOB) performance measures for hospitals. A study evaluated the impact of a hospital-based electronic health record (EHR) intervention on adherence to the revised TOB measures. METHODS: The study was conducted in two acute care hospitals in New York City. Data abstracted from the EHR were analyzed retrospectively from 4,871 smokers discharged between December 2012 and March 2015 to evaluate the impact of two interventions: an order set to prompt clinicians to prescribe pharmacotherapy and a nurse-delivered counseling module that automatically populated the nursing care plan for all smokers. The study estimated the relative odds of a patient being prescribed medication and/or receiving smoking cessation counseling in the intervention period compared to the baseline time period. RESULTS: There was a modest increase in medication orders (odds ratio [OR], 1.35). In contrast, rates of counseling increased 10-fold (OR, 10.54). Patients admitted through surgery were less likely to receive both counseling and medication compared with the medicine service. CONCLUSION: Hospitalization presents an important opportunity to engage smokers in treatment for primary and secondary prevention of tobacco-related illnesses. EHRs can be leveraged to facilitate integration of TOB measure requirements into routine inpatient care; however, the smaller effect on prescribing patterns suggests limitations in this approach alone in changing clinician behavior to meet this measure. The success of the nurse-focused EHR-driven intervention suggests an effective tool for integrating the cessation counseling component of the new measures and the importance of nursing's role in achieving the Joint Commission measure targets.

Impact of an Intervention to Improve Weekend Hospital Care at an Academic Medical Center: An Observational Study.

BACKGROUND: Hospital care on weekends has been associated with delays in care, reduced quality, and poor clinical outcomes. OBJECTIVE: The purpose of this study was to evaluate the impact of a weekend hospital intervention on processes of care and clinical outcomes. The multifaceted intervention included expanded weekend diagnostic services, improved weekend discharge processes, and increased physician and care management services on weekends. DESIGN AND PATIENTS: This was an interrupted time series observational study of adult non-obstetric patients hospitalized at a single academic medical center between January 2011 and January 2014. The study included 18 months prior to and 19 months following the implementation of the intervention. Data were analyzed using segmented regression analysis with adjustment for confounders. MAIN MEASURES: The primary outcome was average length of stay. Secondary outcomes included percent of patients discharged on weekends, 30-day readmission rate, and in-hospital mortality rate. KEY RESULTS: The study included 57,163 hospitalizations. Following implementation of the intervention, average length of stay decreased by 13 % (95 % CI 10-15 %) and continued to decrease by 1 % (95 % CI 1-2 %) per month as compared to the underlying time trend. The proportion of weekend discharges increased by 12 % (95 % CI 2-22 %) at the time of the intervention and continued to increase by 2 % (95 % CI 1-3 %) per month thereafter. The intervention had no impact on readmissions or mortality. During the post-implementation period, the hospital was evacuated and closed for 2 months due to damage from Hurricane Sandy, and a new hospital-wide electronic health record was introduced. The contributions of these events to our findings are not known. We observed a lower inpatient census and found differences in patient characteristics, including higher rates of Medicaid insurance and comorbidities, in the post-Hurricane Sandy period as compared to the pre-Sandy period. CONCLUSIONS: The intervention was associated with a reduction in length of stay and an increase in weekend discharges. Our longitudinal study also illuminated the challenges of evaluating the effectiveness of a large-scale intervention in a real-world hospital setting.

Electronic health record use, intensity of hospital care, and patient outcomes.

OBJECTIVE: Previous studies have suggested that weekend hospital care is inferior to weekday care and that this difference may be related to diminished care intensity. The purpose of this study was to determine whether a metric for measuring intensity of hospital care based on use of the electronic health record was associated with patient-level outcomes. METHODS: We performed a cohort study of hospitalizations at an academic medical center. Intensity of care was defined as the hourly number of provider accessions of the electronic health record, termed "electronic health record interactions." Hospitalizations were categorized on the basis of the mean difference in electronic health record interactions between the first Friday and the first Saturday of hospitalization. We used regression models to determine the association of these categories with patient outcomes after adjusting for covariates. RESULTS: Electronic health record interactions decreased from Friday to Saturday in 77% of the 9051 hospitalizations included in the study. Compared with hospitalizations with no change in Friday to Saturday electronic health record interactions, the relative lengths of stay for hospitalizations with a small, moderate, and large decrease in electronic health record interactions were 1.05 (95% confidence interval [CI], 1.00-1.10), 1.11 (95% CI, 1.05-1.17), and 1.25 (95% CI, 1.15-1.35), respectively. Although a large decrease in electronic health record interactions was associated with in-hospital mortality, these findings were not significant after risk adjustment (odds ratio 1.74, 95% CI, 0.93-3.25). CONCLUSIONS: Intensity of inpatient care, measured by electronic health record interactions, significantly diminished from Friday to Saturday, and this decrease was associated with length of stay. Hospitals should consider monitoring and correcting temporal fluctuations in care intensity.

Monitoring the pulse of hospital activity: electronic health record utilization as a measure of care intensity.

BACKGROUND: Hospital care on weekends has been associated with reduced quality and poor clinical outcomes, suggesting that decreases in overall intensity of care may have important clinical effects. We describe a new measure of hospital intensity of care based on utilization of the electronic health record (EHR). METHODS: We measured global intensity of care at our academic medical center by monitoring the use of the EHR in 2011. Our primary measure, termed EHR interactions, was the number of accessions of a patient's electronic record by a clinician, adjusted for hospital census, per unit of time. Our secondary measure was percent of total available central processing unit (CPU) power used to access EHR servers at a given time. RESULTS: EHR interactions were lower on weekend days as compared to weekdays at every hour (P < 0.0001), and the daytime peak in intensity noted each weekday was blunted on weekends. The relative rate and 95% confidence interval (CI) of census-adjusted record accessions per patient on weekdays compared with weekends were: 1.76 (95% CI: 1.74-1.77), 1.52 (95% CI: 1.50-1.55), and 1.14 (95% CI: 1.12-1.17) for day, morning/evening, and night hours, respectively. Percent CPU usage correlated closely with EHR interactions (r = 0.90). CONCLUSIONS: EHR usage is a valid and easily reproducible measure of intensity of care in the hospital. Using this measure we identified large, hour-specific differences between weekend and weekday intensity. EHR interactions may serve as a useful measure for tracking and improving temporal variations in care that are common, and potentially deleterious, in hospital systems.

The impact of the heparin-induced thrombocytopenia (HIT) computerized alert on provider behaviors and patient outcomes.

OBJECTIVE: The aim of this study was to measure the effect of an electronic heparin-induced thrombocytopenia (HIT) alert on provider ordering behaviors and on patient outcomes. MATERIALS AND METHODS: A pop-up alert was created for providers when an individual's platelet values had decreased by 50% or to <100,000/mm(3) in the setting of recent heparin exposure. The authors retrospectively compared inpatients admitted between January 24, 2008 and August 24, 2008 to a control group admitted 1 year prior to the HIT alert. The primary outcome was a change in HIT antibody testing. Secondary outcomes included an assessment of incidence of HIT antibody positivity, percentage of patients started on a direct thrombin inhibitor (DTI), length of stay and overall mortality. RESULTS: There were 1006 and 1081 patients in the control and intervention groups, respectively. There was a 33% relative increase in HIT antibody test orders (p=0.01), and 33% more of these tests were ordered the first day after the criteria were met when a pop-up alert was given (p=0.03). Heparin was discontinued in 25% more patients in the alerted group (p=0.01), and more direct thrombin inhibitors were ordered for them (p=0.03). The number who tested HIT antibody-positive did not differ, however, between the two groups (p=0.99). The length of stay and mortality were similar in both groups. CONCLUSIONS: The HIT alert significantly impacted provider behaviors. However, the alert did not result in more cases of HIT being detected or an improvement in overall mortality. Our findings do not support implementation of a computerized HIT alert.