Clinical Decision Support Principles for Quality Improvement and Research.

Pediatric hospitalists frequently interact with clinical decision support (CDS) tools in patient care and use these tools for quality improvement or research. In this method/ology paper, we provide an introduction and practical approach to developing and evaluating CDS tools within the electronic health record. First, we define CDS and describe the types of CDS interventions that exist. We then outline a stepwise approach to CDS development, which begins with defining the problem and understanding the system. We present a framework for metric development and then describe tools that can be used for CDS design (eg, 5 Rights of CDS, "10 commandments," usability heuristics, human-centered design) and testing (eg, validation, simulation, usability testing). We review approaches to evaluating CDS tools, which range from randomized studies to traditional quality improvement methods. Lastly, we discuss practical considerations for implementing CDS, including the assessment of a project team's skills and an organization's information technology resources.

Classification of Health Information Technology Safety Events in a Pediatric Tertiary Care Hospital.

OBJECTIVE: State agencies have developed reporting systems of safety events that include events related to health information technology (HIT). These data come from hospital reporting systems where staff submit safety reports and nurses, in the role of safety managers, review, and code events. Safety managers may have varying degrees of experience with identifying events related to HIT. Our objective was to review events potentially involving HIT and compare those with what was reported to the state. METHODS: We performed a structured review of 1 year of safety events from an academic pediatric healthcare system. We reviewed the free-text description of each event and applied a classification scheme derived from the AHRQ Health IT Hazard Manager and compared the results with events reported to the state as involving HIT. RESULTS: Of 33,218 safety events for a 1-year period, 1247 included key words related to HIT and/or were indicated by safety managers as involving HIT. Of the 1247 events, the structured review identified 769 as involving HIT. In comparison, safety managers only identified 194 of the 769 events (25%) as involving HIT. Most events, 353 (46%), not identified by safety managers were documentation issues. Of the 1247 events, the structured review identified 478 as not involving HIT while safety managers identified and reported 81 of these 478 events (17%) as involving HIT. CONCLUSIONS: The current process of reporting safety events lacks standardization in identifying health technology contributions to safety events, which may minimize the effectiveness of safety initiatives.

Clinical decision support with a comprehensive in-EHR patient tracking system improves genetic testing follow up.

OBJECTIVE: We sought to develop and evaluate an electronic health record (EHR) genetic testing tracking system to address the barriers and limitations of existing spreadsheet-based workarounds. MATERIALS AND METHODS: We evaluated the spreadsheet-based system using mixed effects logistic regression to identify factors associated with delayed follow up. These factors informed the design of an EHR-integrated genetic testing tracking system. After deployment, we assessed the system in 2 ways. We analyzed EHR access logs and note data to assess patient outcomes and performed semistructured interviews with users to identify impact of the system on work. RESULTS: We found that patient-reported race was a significant predictor of documented genetic testing follow up, indicating a possible inequity in care. We implemented a CDS system including a patient data capture form and management dashboard to facilitate important care tasks. The system significantly sped review of results and significantly increased documentation of follow-up recommendations. Interviews with key system users identified a range of sociotechnical factors (ie, tools, tasks, collaboration) that contribute to safer and more efficient care. DISCUSSION: Our new tracking system ended decades of workarounds for identifying and communicating test results and improved clinical workflows. Interview participants related that the system decreased cognitive and time burden which allowed them to focus on direct patient interaction. CONCLUSION: By assembling a multidisciplinary team, we designed a novel patient tracking system that improves genetic testing follow up. Similar approaches may be effective in other clinical settings.

Reducing Alert Burden in Electronic Health Records: State of the Art Recommendations from Four Health Systems.

BACKGROUND: Electronic health record (EHR) alert fatigue, while widely recognized as a concern nationally, lacks a corresponding comprehensive mitigation plan. OBJECTIVES: The goal of this manuscript is to provide practical guidance to clinical informaticists and other health care leaders who are considering creating a program to manage EHR alerts. METHODS: This manuscript synthesizes several approaches and recommendations for better alert management derived from four U.S. health care institutions that presented their experiences and recommendations at the American Medical Informatics Association 2019 Clinical Informatics Conference in Atlanta, Georgia, United States. The assembled health care institution leaders represent academic, pediatric, community, and specialized care domains. We describe governance and management, structural concepts and components, and human-computer interactions with alerts, and make recommendations regarding these domains based on our experience supplemented with literature review. This paper focuses on alerts that impact bedside clinicians. RESULTS: The manuscript addresses the range of considerations relevant to alert management including a summary of the background literature about alerts, alert governance, alert metrics, starting an alert management program, approaches to evaluating alerts prior to deployment, and optimization of existing alerts. The manuscript includes examples of alert optimization successes at two of the represented institutions. In addition, we review limitations on the ability to evaluate alerts in the current state and identify opportunities for further scholarship. CONCLUSION: Ultimately, alert management programs must strive to meet common goals of improving patient care, while at the same time decreasing the alert burden on clinicians. In so doing, organizations have an opportunity to promote the wellness of patients, clinicians, and EHRs themselves.

Performance of a Clinical Decision Support Tool to Identify PICU Patients at High Risk for Clinical Deterioration.

OBJECTIVES: To evaluate the translation of a paper high-risk checklist for PICU patients at risk of clinical deterioration to an automated clinical decision support tool. DESIGN: Retrospective, observational cohort study of an automated clinical decision support tool, the PICU Warning Tool, adapted from a paper checklist to predict clinical deterioration events in PICU patients within 24 hours. SETTING: Two quaternary care medical-surgical PICUs-The Children's Hospital of Philadelphia and Cincinnati Children's Hospital Medical Center. PATIENTS: The study included all patients admitted from July 1, 2014, to June 30, 2015, the year prior to the initiation of any focused situational awareness work at either institution. INTERVENTIONS: We replicated the predictions of the real-time PICU Warning Tool by retrospectively querying the institutional data warehouse to identify all patients that would have flagged as high-risk by the PICU Warning Tool for their index deterioration. MEASUREMENTS AND MAIN RESULTS: The primary exposure of interest was determination of high-risk status during PICU admission via the PICU Warning Tool. The primary outcome of interest was clinical deterioration event within 24 hours of a positive screen. The date and time of the deterioration event was used as the index time point. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of the performance of the PICU Warning Tool. There were 6,233 patients evaluated with 233 clinical deterioration events experienced by 154 individual patients. The positive predictive value of the PICU Warning Tool was 7.1% with a number needed to screen of 14 patients for each index clinical deterioration event. The most predictive of the individual criteria were elevated lactic acidosis, high mean airway pressure, and profound acidosis. CONCLUSIONS: Performance of a clinical decision support translation of a paper-based tool showed inferior test characteristics. Improved feasibility of identification of high-risk patients using automated tools must be balanced with performance.

Towards a Maturity Model for Clinical Decision Support Operations.

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Structured override reasons for drug-drug interaction alerts in electronic health records.

OBJECTIVE: The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records. MATERIALS AND METHODS: We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices. RESULTS: Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk." DISCUSSION: We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved. CONCLUSIONS: Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Design and Implementation of a Pediatric ICU Acuity Scoring Tool as Clinical Decision Support.

BACKGROUND AND OBJECTIVE: Pediatric in-hospital cardiac arrest most commonly occurs in the pediatric intensive care unit (PICU) and is frequently preceded by early warning signs of clinical deterioration. In this study, we describe the implementation and evaluation of criteria to identify high-risk patients from a paper-based checklist into a clinical decision support (CDS) tool in the electronic health record (EHR). MATERIALS AND METHODS: The validated paper-based tool was first adapted by PICU clinicians and clinical informaticians and then integrated into clinical workflow following best practices for CDS design. A vendor-based rule engine was utilized. Littenberg's assessment framework helped guide the overall evaluation. Preliminary testing took place in EHR development environments with more rigorous evaluation, testing, and feedback completed in the live production environment. To verify data quality of the CDS rule engine, a retrospective Structured Query Language (SQL) data query was also created. As a process metric, preparedness was measured in pre- and postimplementation surveys. RESULTS: The system was deployed, evaluating approximately 340 unique patients monthly across 4 clinical teams. The verification against retrospective SQL of 15-minute intervals over a 30-day period revealed no missing triggered intervals and demonstrated 99.3% concordance of positive triggers. Preparedness showed improvements across multiple domains to our a priori goal of 90%. CONCLUSION: We describe the successful adaptation and implementation of a real-time CDS tool to identify PICU patients at risk of deterioration. Prospective multicenter evaluation of the tool's effectiveness on clinical outcomes is necessary before broader implementation can be recommended.

Clinical Decision Support Tool for Parental Tobacco Treatment in Hospitalized Children.

OBJECTIVES: To create and evaluate the feasibility, acceptability, and usability of a clinical decision support (CDS) tool within the electronic health record (EHR) to help pediatricians provide smoking cessation counseling and treatment to parents of hospitalized children exposed to secondhand smoke (SHS). METHODS: Mixed method study of first-year pediatric residents on one inpatient unit. Residents received training in smoking cessation counseling, nicotine replacement therapy (NRT) prescribing, and use of a CDS tool to aid in this process. The tool, which alerted when a patient was identified as exposed to SHS based on the history taken on admission or during a prior encounter, had the following capabilities: adding SHS exposure to the patient's problem list; referral to Free Quitline through discharge instructions; and linking to a printable NRT prescription form. We measured feasibility by EHR utilization data. We measured acceptability and usability of the tool by administering questionnaires to residents. RESULTS: From June-August 2015, the alert triggered for 106 patients, and the tool was used for 52 (49%) patients. 41 (39%) patients had SHS exposure added to the problem list, 34 (32%) parents were referred to the Quitline through discharge instructions, and 15 (14%) parents were prescribed NRT. 10 out of 15 (67%) eligible pediatricians used the tool. All clinicians surveyed (9 out of 10) found the tool acceptable and rated its usability good to excellent (average System Usability Scale score was 85 out of 100, 95% CI, 76-93). CONCLUSIONS: A non-interruptive CDS tool to help residents provide smoking cessation counseling in the hospital was feasible, acceptable, and usable. Future work will investigate impacts on patient outcomes.

Safety of Automatic End Dates for Antimicrobial Orders to Facilitate Stewardship.

Following implementation of automatic end dates for antimicrobial orders to facilitate antimicrobial stewardship at a large, academic children's hospital, no differences were observed in patient mortality, length of stay, or readmission rates, even among patients with documented bacteremia. Infect Control Hosp Epidemiol 2016;37:974-978.

Optimization of drug-drug interaction alert rules in a pediatric hospital's electronic health record system using a visual analytics dashboard.

OBJECTIVE: To develop and evaluate an electronic dashboard of hospital-wide electronic health record medication alerts for an alert fatigue reduction quality improvement project. METHODS: We used visual analytics software to develop the dashboard. We collaborated with the hospital-wide Clinical Decision Support committee to perform three interventions successively deactivating clinically irrelevant drug-drug interaction (DDI) alert rules. We analyzed the impact of the interventions on care providers' and pharmacists' alert and override rates using an interrupted time series framework with piecewise regression. RESULTS: We evaluated 2 391 880 medication alerts between January 31, 2011 and January 26, 2014. For pharmacists, the median alert rate prior to the first DDI deactivation was 58.74 alerts/100 orders (IQR 54.98-60.48) and 25.11 alerts/100 orders (IQR 23.45-26.57) following the three interventions (p<0.001). For providers, baseline median alert rate prior to the first round of DDI deactivation was 19.73 alerts/100 orders (IQR 18.66-20.24) and 15.11 alerts/100 orders (IQR 14.44-15.49) following the three interventions (p<0.001). In a subgroup analysis, we observed a decrease in pharmacists' override rates for DDI alerts that were not modified in the system from a median of 93.06 overrides/100 alerts (IQR 91.96-94.33) to 85.68 overrides/100 alerts (IQR 84.29-87.15, p<0.001). The medication serious safety event rate decreased during the study period, and there were no serious safety events reported in association with the deactivated alert rules. CONCLUSIONS: An alert dashboard facilitated safe rapid-cycle reductions in alert burden that were temporally associated with lower pharmacist override rates in a subgroup of DDIs not directly affected by the interventions; meanwhile, the pharmacists' frequency of selecting the 'cancel' option increased. We hypothesize that reducing the alert burden enabled pharmacists to devote more attention to clinically relevant alerts.

Relationships of US youth homicide victims and their offenders, 1976-1999.

BACKGROUND: Homicide is the second leading cause of death in children aged 0 to 19 years. Tailoring violence prevention programs to high-risk individuals requires understanding victim-offender relationships. OBJECTIVE: To elucidate differences in the relationships between homicide victims aged 0 to 19 years and their offenders. DESIGN: Cross-sectional study using the Uniform Crime Reports: Supplemental Homicide Reports, 1976-1999. SETTING AND PARTICIPANTS: The Supplemental Homicide Reports contain incident-level information about criminal homicides, including location and victim and offender characteristics. National coverage is approximately 92%; 70 258 victims were studied. MAIN OUTCOME MEASURES: Differences in the relationships of homicide victims and offenders based on sex, age, population of homicide location, and weapon. RESULTS: Odds ratios and 95% confidence intervals (CIs) are reported. Using strangers as the referent group, murdered girls were 3.6 (95% CI, 3.4-3.9) times more likely to have been killed by family members and 21.3 (95% CI, 18.5-24.4) times more likely to have been killed by intimate partners than murdered boys. Victims younger than 10 years were 33 (95% CI, 30.4-36.1) times more likely than victims older than 10 years to have been killed by a family member and 2.4 (95% CI, 2.2-2.6) times more likely to have been killed by someone else known to them. Stranger homicides occurred in areas with approximately 145 000 more residents (P<.01). Handguns were more likely to be used during homicides committed by strangers (P<.01). These associations remained after adjusting for potential confounders. CONCLUSIONS: Future violence prevention programs may have more effect when targeted to specific sex and age groups. Elucidation of the origins of sex differences, focus on evidence-based child abuse prevention efforts, and enforcement of current gun control laws may help reduce the number of homicides among children.