Prognostic clinical decision support for pneumonia in the emergency department: A randomized trial.

BACKGROUND: Hospitalization rates for childhood pneumonia vary widely. Risk-based clinical decision support (CDS) interventions may reduce unwarranted variation. METHODS: We conducted a pragmatic randomized trial in two US pediatric emergency departments (EDs) comparing electronic health record (EHR)-integrated prognostic CDS versus usual care for promoting appropriate ED disposition in children (<18 years) with pneumonia. Encounters were randomized 1:1 to usual care versus custom CDS featuring a validated pneumonia severity score predicting risk for severe in-hospital outcomes. Clinicians retained full decision-making authority. The primary outcome was inappropriate ED disposition, defined as early transition to lower- or higher-level care. Safety and implementation outcomes were also evaluated. RESULTS: The study enrolled 536 encounters (269 usual care and 267 CDS). Baseline characteristics were similar across arms. Inappropriate disposition occurred in 3% of usual care encounters and 2% of CDS encounters (adjusted odds ratio: 0.99, 95% confidence interval: [0.32, 2.95]). Length of stay was also similar and adverse safety outcomes were uncommon in both arms. The tool's custom user interface and content were viewed as strengths by surveyed clinicians (>70% satisfied). Implementation barriers include intrinsic (e.g., reaching the right person at the right time) and extrinsic factors (i.e., global pandemic). CONCLUSIONS: EHR-based prognostic CDS did not improve ED disposition decisions for children with pneumonia. Although the intervention's content was favorably received, low subject accrual and workflow integration problems likely limited effectiveness. Clinical Trials Registration: NCT06033079.

User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial.

OBJECTIVES: To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia. METHODS: Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on Fast Healthcare Interoperability Resources (FHIR) web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and postdeployment summative evaluation. RESULTS: Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitated enrollment, randomization, model visualization, data capture, and reporting for trial purposes. CONCLUSION: The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Inpatient nurses' preferences and decisions with risk information visualization.

OBJECTIVE: We examined the influence of 4 different risk information formats on inpatient nurses' preferences and decisions with an acute clinical deterioration decision-support system. MATERIALS AND METHODS: We conducted a comparative usability evaluation in which participants provided responses to multiple user interface options in a simulated setting. We collected qualitative data using think aloud methods. We collected quantitative data by asking participants which action they would perform after each time point in 3 different patient scenarios. RESULTS: More participants (n = 6) preferred the probability format over relative risk ratios (n = 2), absolute differences (n = 2), and number of persons out of 100 (n = 0). Participants liked average lines, having a trend graph to supplement the risk estimate, and consistent colors between trend graphs and possible actions. Participants did not like too much text information or the presence of confidence intervals. From a decision-making perspective, use of the probability format was associated with greater concordance in actions taken by participants compared to the other 3 risk information formats. DISCUSSION: By focusing on nurses' preferences and decisions with several risk information display formats and collecting both qualitative and quantitative data, we have provided meaningful insights for the design of clinical decision-support systems containing complex quantitative information. CONCLUSION: This study adds to our knowledge of presenting risk information to nurses within clinical decision-support systems. We encourage those developing risk-based systems for inpatient nurses to consider expressing risk in a probability format and include a graph (with average line) to display the patient's recent trends.

Empowering patients to address diabetes care gaps: formative usability testing of a novel patient portal intervention.

Objective: The aim of this study was to design and assess the formative usability of a novel patient portal intervention designed to empower patients with diabetes to initiate orders for diabetes-related monitoring and preventive services. Materials and Methods: We used a user-centered Design Sprint methodology to create our intervention prototype and assess its usability with 3 rounds of iterative testing. Participants (5/round) were presented with the prototype and asked to perform common, standardized tasks using think-aloud procedures. A facilitator rated task performance using a scale: (1) completed with ease, (2) completed with difficulty, and (3) failed. Participants completed the System Usability Scale (SUS) scored 0-worst to 100-best. All testing occurred remotely via Zoom. Results: We identified 3 main categories of usability issues: distrust about the automated system, content concerns, and layout difficulties. Changes included improving clarity about the ordering process and simplifying language; however, design constraints inherent to the electronic health record system limited our ability to respond to all usability issues (eg, could not modify fixed elements in layout). Percent of tasks completed with ease across each round were 67%, 60%, and 80%, respectively. Average SUS scores were 87, 74, and 93, respectively. Across rounds, participants found the intervention valuable and appreciated the concept of patient-initiated ordering. Conclusions: Through iterative user-centered design and testing, we improved the usability of the patient portal intervention. A tool that empowers patients to initiate orders for disease-specific services as part of their existing patient portal account has potential to enhance the completion of recommended health services and improve clinical outcomes.

Antibiotic clinical decision support for pneumonia in the ED: A randomized trial.

BACKGROUND: Electronic health record-based clinical decision support (CDS) is a promising antibiotic stewardship strategy. Few studies have evaluated the effectiveness of antibiotic CDS in the pediatric emergency department (ED). OBJECTIVE: To compare the effectiveness of antibiotic CDS vs. usual care for promoting guideline-concordant antibiotic prescribing for pneumonia in the pediatric ED. DESIGN: Pragmatic randomized clinical trial. SETTING AND PARTICIPANTS: Encounters for children (6 months-18 years) with pneumonia presenting to two tertiary care children s hospital EDs in the United States. INTERVENTION: CDS or usual care was randomly assigned during 4-week periods within each site. The CDS intervention provided antibiotic recommendations tailored to each encounter and in accordance with national guidelines. MAIN OUTCOME AND MEASURES: The primary outcome was exclusive guideline-concordant antibiotic prescribing within the first 24 h of care. Safety outcomes included time to first antibiotic order, encounter length of stay, delayed intensive care, and 3- and 7-day revisits. RESULTS: 1027 encounters were included, encompassing 478 randomized to usual care and 549 to CDS. Exclusive guideline-concordant prescribing did not differ at 24 h (CDS, 51.7% vs. usual care, 53.3%; odds ratio [OR] 0.94 [95% confidence interval [CI]: 0.73, 1.20]). In pre-specified stratified analyses, CDS was associated with guideline-concordant prescribing among encounters discharged from the ED (74.9% vs. 66.0%; OR 1.53 [95% CI: 1.01, 2.33]), but not among hospitalized encounters. Mean time to first antibiotic was shorter in the CDS group (3.0 vs 3.4 h; p = .024). There were no differences in safety outcomes. CONCLUSIONS: Effectiveness of ED-based antibiotic CDS was greatest among those discharged from the ED. Longitudinal interventions designed to target both ED and inpatient clinicians and to address common implementation challenges may enhance the effectiveness of CDS as a stewardship tool.

Descriptive Usability Study of CirrODS: Clinical Decision and Workflow Support Tool for Management of Patients With Cirrhosis.

BACKGROUND: There are gaps in delivering evidence-based care for patients with chronic liver disease and cirrhosis. OBJECTIVE: Our objective was to use interactive user-centered design methods to develop the Cirrhosis Order Set and Clinical Decision Support (CirrODS) tool in order to improve clinical decision-making and workflow. METHODS: Two work groups were convened with clinicians, user experience designers, human factors and health services researchers, and information technologists to create user interface designs. CirrODS prototypes underwent several rounds of formative design. Physicians (n=20) at three hospitals were provided with clinical scenarios of patients with cirrhosis, and the admission orders made with and without the CirrODS tool were compared. The physicians rated their experience using CirrODS and provided comments, which we coded into categories and themes. We assessed the safety, usability, and quality of CirrODS using qualitative and quantitative methods. RESULTS: We created an interactive CirrODS prototype that displays an alert when existing electronic data indicate a patient is at risk for cirrhosis. The tool consists of two primary frames, presenting relevant patient data and allowing recommended evidence-based tests and treatments to be ordered and categorized. Physicians viewed the tool positively and suggested that it would be most useful at the time of admission. When using the tool, the clinicians placed fewer orders than they placed when not using the tool, but more of the orders placed were considered to be high priority when the tool was used than when it was not used. The physicians' ratings of CirrODS indicated above average usability. CONCLUSIONS: We developed a novel Web-based combined clinical decision-making and workflow support tool to alert and assist clinicians caring for patients with cirrhosis. Further studies are underway to assess the impact on quality of care for patients with cirrhosis in actual practice.

Electronically Mediated Time-out Initiative to Reduce the Incidence of Wrong Surgery: An Interventional Observational Study.

BACKGROUND: "Wrong surgery" is defined as wrong site, wrong operation, or wrong patient, with estimated incidence up to 1 per 5,000 cases. Responding to national attention on wrong surgery, our objective was to create a care redesign intervention to minimize the rate of wrong surgery. METHODS: The authors created an electronic system using existing intraoperative electronic documentation to present a time-out checklist on large in-room displays. Time-out was dynamically interposed as a forced-function documentation step between "patient-in-operating room" and "incision." Time to complete documentation was obtained from audit logs. The authors measured the postimplementation wrong surgery rate and used Bayesian methods to compare the pre- and postimplementation rates at our institution. Previous probabilities were selected using wrong surgery rate estimates from the observed performance reported in the literature to generate previous probabilities (4.24 wrong surgeries per 100,000 cases). RESULTS: No documentation times exceeded 5 min; 97% of documentation tasks were completed within 2 min. The authors performed 243,939 operations over 5 yr using the system, with zero wrong surgeries, compared with 253,838 operations over 6 yr with two wrong surgeries before implementation. Bayesian analysis suggests an 84% probability that the postimplementation wrong rate is lower than baseline. However, given the rarity of wrong surgery in our sample, there is substantial uncertainty. The total system-development cost was $34,000, roughly half the published cost of one weighted median settlement for wrong surgery. CONCLUSION: Implementation of a forced-completion electronically mediated time-out process before incision is feasible, but it is unclear whether true performance improvements occur.