User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial.

OBJECTIVE: To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia. METHODS: Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on FHIR web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and post-deployment summative evaluation. RESULTS: Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitatd enrollment, randomization, model visualization, data capture, and reporting for trial purposes. CONCLUSIONS: The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Antibiotic clinical decision support for pneumonia in the ED: A randomized trial.

BACKGROUND: Electronic health record-based clinical decision support (CDS) is a promising antibiotic stewardship strategy. Few studies have evaluated the effectiveness of antibiotic CDS in the pediatric emergency department (ED). OBJECTIVE: To compare the effectiveness of antibiotic CDS vs. usual care for promoting guideline-concordant antibiotic prescribing for pneumonia in the pediatric ED. DESIGN: Pragmatic randomized clinical trial. SETTING AND PARTICIPANTS: Encounters for children (6 months-18 years) with pneumonia presenting to two tertiary care children s hospital EDs in the United States. INTERVENTION: CDS or usual care was randomly assigned during 4-week periods within each site. The CDS intervention provided antibiotic recommendations tailored to each encounter and in accordance with national guidelines. MAIN OUTCOME AND MEASURES: The primary outcome was exclusive guideline-concordant antibiotic prescribing within the first 24 h of care. Safety outcomes included time to first antibiotic order, encounter length of stay, delayed intensive care, and 3- and 7-day revisits. RESULTS: 1027 encounters were included, encompassing 478 randomized to usual care and 549 to CDS. Exclusive guideline-concordant prescribing did not differ at 24 h (CDS, 51.7% vs. usual care, 53.3%; odds ratio [OR] 0.94 [95% confidence interval [CI]: 0.73, 1.20]). In pre-specified stratified analyses, CDS was associated with guideline-concordant prescribing among encounters discharged from the ED (74.9% vs. 66.0%; OR 1.53 [95% CI: 1.01, 2.33]), but not among hospitalized encounters. Mean time to first antibiotic was shorter in the CDS group (3.0 vs 3.4 h; p = .024). There were no differences in safety outcomes. CONCLUSIONS: Effectiveness of ED-based antibiotic CDS was greatest among those discharged from the ED. Longitudinal interventions designed to target both ED and inpatient clinicians and to address common implementation challenges may enhance the effectiveness of CDS as a stewardship tool.

Clinical decision support automates care gap detection among primary care patients with nonalcoholic fatty liver disease.

BACKGROUND: Although guidelines recommend primary care-driven management of NAFLD, workflow constraints hinder feasibility. Leveraging electronic health records to risk stratify patients proposes a scalable, workflow-integrated strategy. MATERIALS AND METHODS: We prospectively evaluated an electronic health record-embedded clinical decision support system's ability to risk stratify patients with NAFLD and detect gaps in care. Patients missing annual laboratory testing to calculate Fibrosis-4 Score (FIB-4) or those missing necessary linkage to further care were considered to have a gap in care. Linkage to care was defined as either referral for elastography-based testing or for consultation in hepatology clinic depending on clinical and biochemical characteristics. RESULTS: Patients with NAFLD often lacked annual screening labs within primary care settings (1129/2154; 52%). Linkage to care was low in all categories, with <3% of patients with abnormal FIB-4 undergoing further evaluation. DISCUSSION: Significant care gaps exist within primary care for screening and risk stratification of patients with NAFLD and can be efficiently addressed using electronic health record functionality.

A multi-site randomized trial of a clinical decision support intervention to improve problem list completeness.

OBJECTIVE: To improve problem list documentation and care quality. MATERIALS AND METHODS: We developed algorithms to infer clinical problems a patient has that are not recorded on the coded problem list using structured data in the electronic health record (EHR) for 12 clinically significant heart, lung, and blood diseases. We also developed a clinical decision support (CDS) intervention which suggests adding missing problems to the problem list. We evaluated the intervention at 4 diverse healthcare systems using 3 different EHRs in a randomized trial using 3 predetermined outcome measures: alert acceptance, problem addition, and National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (NCQA HEDIS) clinical quality measures. RESULTS: There were 288 832 opportunities to add a problem in the intervention arm and the problem was added 63 777 times (acceptance rate 22.1%). The intervention arm had 4.6 times as many problems added as the control arm. There were no significant differences in any of the clinical quality measures. DISCUSSION: The CDS intervention was highly effective at improving problem list completeness. However, the improvement in problem list utilization was not associated with improvement in the quality measures. The lack of effect on quality measures suggests that problem list documentation is not directly associated with improvements in quality measured by National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (NCQA HEDIS) quality measures. However, improved problem list accuracy has other benefits, including clinical care, patient comprehension of health conditions, accurate CDS and population health, and for research. CONCLUSION: An EHR-embedded CDS intervention was effective at improving problem list completeness but was not associated with improvement in quality measures.

Impact of Updating Pharmacogenetic Results: Lessons Learned from the PREDICT Program.

Pharmacogenomic (PGx) evidence for selective serotonin reuptake inhibitors (SSRIs) continues to evolve. For sites offering testing, maintaining up-to-date interpretations and implementing new clinical decision support (CDS) driven by existing results creates practical and technical challenges. Vanderbilt University Medical Center initiated panel testing in 2010, added CYP2D6 testing in 2017, and released CDS for SSRIs in 2020. We systematically reinterpreted historic CYP2C19 and CYP2D6 genotypes to update phenotypes to current nomenclature and to launch provider CDS and patient-oriented content for SSRIs. Chart review was conducted to identify and recontact providers caring for patients with current SSRI therapy and new actionable recommendations. A total of 15,619 patients' PGx results were reprocessed. Of the non-deceased patients reprocessed, 21% (n = 3278) resulted in CYP2C19*1/*17 reinterpretations. Among 289 patients with an actionable recommendation and SSRI medication prescription, 31.8% (n = 92) did not necessitate contact of a clinician, while 43.2% (n = 125) resulted in clinician contacted, and for 25% (n = 72) no appropriate clinician was able to be identified. Maintenance of up-to-date interpretations and recommendations for PGx results over the lifetime of a patient requires continuous effort. Reprocessing is a key strategy for maintenance and expansion of PGx content to be periodically considered and implemented.

Algorithmic Detection of Boolean Logic Errors in Clinical Decision Support Statements.

OBJECTIVE: Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logic errors in CDS statements. METHODS: Nine health care organizations extracted Boolean logic statements from their Epic electronic health record (EHR). We developed an open-source software tool, which implemented the Espresso logic minimization algorithm, to identify three classes of logic errors. RESULTS: Participating organizations submitted 260,698 logic statements, of which 44,890 were minimized by Espresso. We found errors in 209 of them. Every participating organization had at least two errors, and all organizations reported that they would act on the feedback. DISCUSSION: An automated algorithm can readily detect specific categories of Boolean CDS logic errors. These errors represent a minority of CDS errors, but very likely require correction to avoid patient safety issues. This process found only a few errors at each site, but the problem appears to be widespread, affecting all participating organizations. CONCLUSION: Both CDS implementers and EHR vendors should consider implementing similar algorithms as part of the CDS authoring process to reduce the number of errors in their CDS interventions.

A Tutorial for Pharmacogenomics Implementation Through End-to-End Clinical Decision Support Based on Ten Years of Experience from PREDICT.

Vanderbilt University Medical Center implemented pharmacogenomics (PGx) testing with the Pharmacogenomic Resource for Enhanced Decisions in Care and Treatment (PREDICT) initiative in 2010. This tutorial reviews the laboratory considerations, technical infrastructure, and programmatic support required to deliver panel-based PGx testing across a large health system with examples and experiences from the first decade of the PREDICT initiative. From the time of inception, automated clinical decision support (CDS) has been a critical capability for delivering PGx results to the point-of-care. Key features of the CDS include human-readable interpretations and clinical guidance that is anticipatory, actionable, and adaptable to changes in the scientific literature. Implementing CDS requires that structured results from the laboratory be encoded in standards-based messages that are securely ingested by electronic health records. Translating results to guidance also requires an informatics infrastructure with multiple components: (1) to manage the interpretation of raw genomic data to "star allele" results to expected phenotype, (2) to define the rules that associate a phenotype with recommended changes to clinical care, and (3) to manage and update the knowledge base. Knowledge base management is key to processing new results with the latest guidelines, and to ensure that historical genomic results can be reinterpreted with revised CDS. We recommend that these components be deployed with institutional authorization, programmatic support, and clinician education to govern the CDS content and policies around delivery.

Is the Climb Worth the View? The Savings/Alert Ratio for Reducing Vitamin D Testing.

BACKGROUND: Despite guideline recommendations, vitamin D testing has increased substantially. Clinical decision support (CDS) presents an opportunity to reduce inappropriate laboratory testing. OBJECTIVES AND METHODS: To reduce inappropriate testing of vitamin D at the Vanderbilt University Medical Center, a CDS assigned providers to receive or not receive an electronic alert each time a 25-hydroxyvitamin D assay was ordered for an adult patient unless the order was associated with a diagnosis in the patient's chart for which vitamin D testing is recommended. The CDS ran for 80 days, collecting data on number of tests, provider information, and basic patient demographics. RESULTS: During the 80 days, providers placed 12,368 orders for 25-hydroxyvitamin D. The intervention group ordered a vitamin D assay and received the alert for potentially inappropriate testing 2,181 times and completed the 25-hydroxyvitamin D order in 89.9% of encounters, while the control group ordered a vitamin D assay (without receiving an alert) 2,032 times and completed the order in 98.1% of encounters, for an absolute reduction of testing of 8% (p < 0.001). CONCLUSION: This CDS reduced vitamin D ordering by utilizing a soft-stop approach. At a charge of $179.00 per test and a cost to the laboratory of $4.20 per test, each display of the alert led to an average reduction of $14.70 in charges and of $0.34 in spending by the laboratory (the savings/alert ratio). By describing the effectiveness of an electronic alert in terms of the savings/alert ratio, the impact of this intervention can be better appreciated and compared with other interventions.

Natural Language Processing and Machine Learning to Enable Clinical Decision Support for Treatment of Pediatric Pneumonia.

Pneumonia is the most frequent cause of infectious disease-related deaths in children worldwide. Clinical decision support (CDS) applications can guide appropriate treatment, but the system must first recognize the appropriate diagnosis. To enable CDS for pediatric pneumonia, we developed an algorithm integrating natural language processing (NLP) and random forest classifiers to identify potential pediatric pneumonia from radiology reports. We deployed the algorithm in the EHR of a large children's hospital using real-time NLP. We describe the development and deployment of the algorithm, and evaluate our approach using 9-months of data gathered while the system was in use. Our model, trained on individual radiology reports, had an AUC of 0.954. The intervention, evaluated on patient encounters that could include multiple radiology reports, achieved a sensitivity, specificity, and positive predictive value of0.899, 0.949, and 0.781, respectively.

Acute kidney injury risk-based screening in pediatric inpatients: a pragmatic randomized trial.

BACKGROUND: Pediatric acute kidney injury (AKI) is common and associated with increased morbidity, mortality, and length of stay. We performed a pragmatic randomized trial testing the hypothesis that AKI risk alerts increase AKI screening. METHODS: All intensive care and ward admissions of children aged 28 days through 21 years without chronic kidney disease from 12/6/2016 to 11/1/2017 were included. The intervention alert displayed if calculated AKI risk was > 50% and no serum creatinine (SCr) was ordered within 24 h. The primary outcome was SCr testing within 48 h of AKI risk > 50%. RESULTS: Among intensive care admissions, 973/1909 (51%) were randomized to the intervention. Among those at risk, more SCr tests were ordered for the intervention group than for controls (418/606, 69% vs. 361/597, 60%, p = 0.002). AKI incidence and severity were the same in intervention and control groups. Among ward admissions, 5492/10997 (50%) were randomized to the intervention, and there were no differences between groups in SCr testing, AKI incidence, or severity of AKI. CONCLUSIONS: Alerts based on real-time prediction of AKI risk increased screening rates in intensive care but not pediatric ward settings. Pragmatic clinical trials provide the opportunity to assess clinical decision support and potentially eliminate ineffective alerts.

Towards a Maturity Model for Clinical Decision Support Operations.

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Creating an immunization content database for knowledge management across clinical systems.

PURPOSE: A initiative at an academic medical center to create a single database of immunization-related content to inform the build and configuration of immunization-related knowledge assets across multiple clinical systems is described. METHODS: Semistructured expert interviews were conducted to ascertain the immunization information needs of the institution's clinical systems. Based on those needs, an immunization domain model constructed with data available from the Centers for Disease Control and Prevention (CDC) website was developed and used to analyze and compare current immunization-related content from CDC data sources with the content of the institution's clinical systems. RESULTS: Five identified clinical systems that used immunization-related content collectively required 22 unique information concepts, 11 of which were obtainable from CDC vaccine code sets. The proportion of vaccines designated by CDC as active products (i.e., currently available administrable vaccines) that were included in the 5 clinical systems ranged from 59% to 95%; in addition, some non-active-status vaccines were listed as active-status products in the various clinical systems. Upon further review, updates to immunization-related content in the 5 clinical systems were implemented. CONCLUSION: Creating a single database for immunization-related content based on CDC data facilitated an explicit and tractable knowledge management process and helped ensure that clinical systems had correct and current content. The immunization domain model created has the potential to assist in the automated detection of updates and relaying those updates to the applicable clinical systems.

Rebuilding the Standing Prescription Renewal Orders.

BACKGROUND: Managing prescription renewal requests is a labor-intensive challenge in ambulatory care. In 2009, Vanderbilt University Medical Center developed clinic-specific standing prescription renewal orders that allowed nurses, under specific conditions, to authorize renewal requests. Formulary and authorization changes made maintaining these documents very challenging. OBJECTIVE: This article aims to review, standardize, and restructure legacy standing prescription renewal orders into a modular, scalable, and easier to manage format for conversion and use in a new electronic health record (EHR). METHODS: We created an enterprise-wide renewal domain model using modular subgroups within the main institutional standing renewal order policy by extracting metadata, medication group names, medication ingredient names, and renewal criteria from approved legacy standing renewal orders. Instance-based matching compared medication groups in a pairwise manner to calculate a similarity score between medication groups. We grouped and standardized medication groups with high similarity by mapping them to medication classes from a medication terminology vendor and filtering them by intended route (e.g., oral, subcutaneous, inhalation). After standardizing the renewal criteria to a short list of reusable criteria, the Pharmacy and Therapeutics (P&T) committee reviewed and approved candidate medication groups and corresponding renewal criteria. RESULTS: Seventy-eight legacy standing prescription renewal orders covered 135 clinics (some applied to multiple clinics). Several standing orders were perfectly congruent, listing identical medications for renewal. We consolidated 870 distinct medication classes to 164 subgroups and assigned renewal criteria. We consolidated 379 distinct legacy renewal criteria to 21 criteria. After approval by the P&T committee, we built subgroups in a structured and consistent format in the new EHR, where they facilitated chart review and standing order adherence by nurses. Additionally, clinicians could search an autogenerated document of the standing order content from the EHR data warehouse. CONCLUSION: We describe a methodology for standardizing and scaling standing prescription renewal orders at an enterprise level while transitioning to a new EHR.

Leveraging Knowledge Representation to Maintain Immunization Clinical Decision Support.

Immunizations are one of the most cost-effective interventions for preventing morbidity and mortality. As vaccines, related clinical knowledge and requirements change, clinical applications must be updated in a timely manner to avoid practicing outdated medicine. We use the Centers for Disease Control and Prevention (CDC) as a source for immunization knowledge for our Clinical Information Systems (CIS). After identifying knowledge management related gaps in the CDC's content and email notification service, we developed and adapted a knowledge management tool chain - called COMET - for facilitating automatic processing of the available immunization content to implement mature knowledge lifecycle management practices locally. The implemented features include error and change tracking, content discovery and analytics, and tracking of dependencies to dependent downstream CISs. We demonstrate the creation of a tool that enables content curators to visualize, track, and implement immunization changes.

Electronic health record-based predictive models for acute kidney injury screening in pediatric inpatients.

BackgroundAcute kidney injury (AKI) is common in pediatric inpatients and is associated with increased morbidity, mortality, and length of stay. Its early identification can reduce severity.MethodsTo create and validate an electronic health record (EHR)-based AKI screening tool, we generated temporally distinct development and validation cohorts using retrospective data from our tertiary care children's hospital, including children aged 28 days through 21 years with sufficient serum creatinine measurements to determine AKI status. AKI was defined as 1.5-fold or 0.3 mg/dl increase in serum creatinine. Age, medication exposures, platelet count, red blood cell distribution width, serum phosphorus, serum transaminases, hypotension (ICU only), and pH (ICU only) were included in AKI risk prediction models.ResultsFor ICU patients, 791/1,332 (59%) of the development cohort and 470/866 (54%) of the validation cohort had AKI. In external validation, the ICU prediction model had a c-statistic=0.74 (95% confidence interval 0.71-0.77). For non-ICU patients, 722/2,337 (31%) of the development cohort and 469/1,474 (32%) of the validation cohort had AKI, and the prediction model had a c-statistic=0.69 (95% confidence interval 0.66-0.72).ConclusionsAKI screening can be performed using EHR data. The AKI screening tool can be incorporated into EHR systems to identify high-risk patients without serum creatinine data, enabling targeted laboratory testing, early AKI identification, and modification of care.

Using Best Practices to Extract, Organize, and Reuse Embedded Decision Support Content Knowledge Rules from Mature Clinical Systems.

Clinical decision support (CDS) knowledge, embedded over time in mature medical systems, presents an interesting and complex opportunity for information organization, maintenance, and reuse. To have a holistic view of all decision support requires an in-depth understanding of each clinical system as well as expert knowledge of the latest evidence. This approach to clinical decision support presents an opportunity to unify and externalize the knowledge within rules-based decision support. Driven by an institutional need to prioritize decision support content for migration to new clinical systems, the Center for Knowledge Management and Health Information Technology teams applied their unique expertise to extract content from individual systems, organize it through a single extensible schema, and present it for discovery and reuse through a newly created Clinical Support Knowledge Acquisition and Archival Tool (CS-KAAT). CS-KAAT can build and maintain the underlying knowledge infrastructure needed by clinical systems.

Use of a computerized C-reactive protein (CRP) based sepsis evaluation in very low birth weight (VLBW) infants: a five-year experience.

BACKGROUND: Serial C-reactive protein (CRP) values may be useful for decision-making regarding duration of antibiotics in neonates. However, established standard of practice for its use in preterm very low birth weight (<1500 g, VLBW) infants are lacking. OBJECTIVE: Evaluate compliance with a CRP-guided computerized decision support (CDS) algorithm and compare characteristics and outcomes of compliant versus non-compliant cases. Measure correlation between CRPs and white blood count (WBC) indices. METHODS: We examined 3 populations: 1) all preterm VLBW infants born at Vanderbilt 2006-2011 - we assessed provider compliance with CDS algorithm and measured relevant outcomes; 2) all patients with positive blood culture results admitted to the Vanderbilt NICU 2006-2012 - we tested the correlation between CRP and WBC results within 7 days of blood culture phlebotomy; 3) 1,000 randomly selected patients out of the 7,062 patients admitted to the NICU 2006-2012 - we correlated time-associated CRP values and absolute neutrophil counts. RESULTS: Of 636 VLBW infants in cohort 1), 569 (89%) received empiric antibiotics for suspected early-onset sepsis. In 409 infants (72%) the CDS algorithm was followed; antibiotics were discontinued ≤48 hours in 311 (55%) with normal serial CRPs and continued in 98 (17%) with positive CRPs, resulting in significant reduction in antibiotic exposure (p<0.001) without increase in complications or subsequent infections. One hundred sixty (28%) were considered non-compliant because antibiotics were continued beyond 48 hours despite negative serial CRPs and blood cultures. Serial CRPs remained negative in 38 (12%) of 308 blood culture-positive infants from cohort 2, but only 4 patients had clinically probable sepsis with single organisms and no immunodeficiency besides extreme prematurity. Leukopenia of any cell type was not linked with CRPs in cohorts 2 and 3. CONCLUSIONS: CDS/CRP-guided antibiotic use is safe and effective in culture-negative VLBW infants. CRP results are not affected by low WBC indices.