Reducing prescribing of antibiotics for acute respiratory infections using a frontline nurse-led EHR-Integrated clinical decision support tool: protocol for a stepped wedge randomized control trial.

BACKGROUND: Overprescribing of antibiotics for acute respiratory infections (ARIs) remains a major issue in outpatient settings. Use of clinical prediction rules (CPRs) can reduce inappropriate antibiotic prescribing but they remain underutilized by physicians and advanced practice providers. A registered nurse (RN)-led model of an electronic health record-integrated CPR (iCPR) for low-acuity ARIs may be an effective alternative to address the barriers to a physician-driven model. METHODS: Following qualitative usability testing, we will conduct a stepped-wedge practice-level cluster randomized controlled trial (RCT) examining the effect of iCPR-guided RN care for low acuity patients with ARI. The primary hypothesis to be tested is: Implementation of RN-led iCPR tools will reduce antibiotic prescribing across diverse primary care settings. Specifically, this study aims to: (1) determine the impact of iCPRs on rapid strep test and chest x-ray ordering and antibiotic prescribing rates when used by RNs; (2) examine resource use patterns and cost-effectiveness of RN visits across diverse clinical settings; (3) determine the impact of iCPR-guided care on patient satisfaction; and (4) ascertain the effect of the intervention on RN and physician burnout. DISCUSSION: This study represents an innovative approach to using an iCPR model led by RNs and specifically designed to address inappropriate antibiotic prescribing. This study has the potential to provide guidance on the effectiveness of delegating care of low-acuity patients with ARIs to RNs to increase use of iCPRs and reduce antibiotic overprescribing for ARIs in outpatient settings. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04255303, Registered February 5 2020, https://clinicaltrials.gov/ct2/show/NCT04255303 .

Integrating Clinical Decision Support Into Electronic Health Record Systems Using a Novel Platform (EvidencePoint): Developmental Study.

BACKGROUND: Through our work, we have demonstrated how clinical decision support (CDS) tools integrated into the electronic health record (EHR) assist providers in adopting evidence-based practices. This requires confronting technical challenges that result from relying on the EHR as the foundation for tool development; for example, the individual CDS tools need to be built independently for each different EHR. OBJECTIVE: The objective of our research was to build and implement an EHR-agnostic platform for integrating CDS tools, which would remove the technical constraints inherent in relying on the EHR as the foundation and enable a single set of CDS tools that can work with any EHR. METHODS: We developed EvidencePoint, a novel, cloud-based, EHR-agnostic CDS platform, and we will describe the development of EvidencePoint and the deployment of its initial CDS tools, which include EHR-integrated applications for clinical use cases such as prediction of hospitalization survival for patients with COVID-19, venous thromboembolism prophylaxis, and pulmonary embolism diagnosis. RESULTS: The results below highlight the adoption of the CDS tools, the International Medical Prevention Registry on Venous Thromboembolism-D-Dimer, the Wells' criteria, and the Northwell COVID-19 Survival (NOCOS), following development, usability testing, and implementation. The International Medical Prevention Registry on Venous Thromboembolism-D-Dimer CDS was used in 5249 patients at the 2 clinical intervention sites. The intervention group tool adoption was 77.8% (4083/5249 possible uses). For the NOCOS tool, which was designed to assist with triaging patients with COVID-19 for hospital admission in the event of constrained hospital resources, the worst-case resourcing scenario never materialized and triaging was never required. As a result, the NOCOS tool was not frequently used, though the EvidencePoint platform's flexibility and customizability enabled the tool to be developed and deployed rapidly under the emergency conditions of the pandemic. Adoption rates for the Wells' criteria tool will be reported in a future publication. CONCLUSIONS: The EvidencePoint system successfully demonstrated that a flexible, user-friendly platform for hosting CDS tools outside of a specific EHR is feasible. The forthcoming results of our outcomes analyses will demonstrate the adoption rate of EvidencePoint tools as well as the impact of behavioral economics "nudges" on the adoption rate. Due to the EHR-agnostic nature of EvidencePoint, the development process for additional forms of CDS will be simpler than traditional and cumbersome IT integration approaches and will benefit from the capabilities provided by the core system of EvidencePoint.

Considerations for using predictive models that include race as an input variable: The case study of lung cancer screening.

Indiscriminate use of predictive models incorporating race can reinforce biases present in source data and lead to an exacerbation of health disparities. In some countries, such as the United States, there is therefore a push to remove race from prediction models; however, there are still many prediction models that use race as an input. Biomedical informaticists who are given the responsibility of using these predictive models in healthcare environments are likely to be faced with questions like how to deal with race covariates in these models. Thus, there is a need for a pragmatic framework to help model users think through how to include race in their chosen model so as to avoid inadvertently exacerbating disparities. In this paper, we use the case study of lung cancer screening to propose a simple framework to guide how model users can approach the use (or non-use) of race inputs in the predictive models they are tasked with leveraging in electronic health records and clinical workflows.

Nudging Health Care Providers' Adoption of Clinical Decision Support: Protocol for the User-Centered Development of a Behavioral Economics-Inspired Electronic Health Record Tool.

BACKGROUND: The improvements in care resulting from clinical decision support (CDS) have been significantly limited by consistently low health care provider adoption. Health care provider attitudes toward CDS, specifically psychological and behavioral barriers, are not typically addressed during any stage of CDS development, although they represent an important barrier to adoption. Emerging evidence has shown the surprising power of using insights from the field of behavioral economics to address psychological and behavioral barriers. Nudges are formal applications of behavioral economics, defined as positive reinforcement and indirect suggestions that have a nonforced effect on decision-making. OBJECTIVE: Our goal is to employ a user-centered design process to develop a CDS tool-the pulmonary embolism (PE) risk calculator-for PE risk stratification in the emergency department that incorporates a behavior theory-informed nudge to address identified behavioral barriers to use. METHODS: All study activities took place at a large academic health system in the New York City metropolitan area. Our study used a user-centered and behavior theory-based approach to achieve the following two aims: (1) use mixed methods to identify health care provider barriers to the use of an active CDS tool for PE risk stratification and (2) develop a new CDS tool-the PE risk calculator-that addresses behavioral barriers to health care providers' adoption of CDS by incorporating nudges into the user interface. These aims were guided by the revised Observational Research Behavioral Information Technology model. A total of 50 clinicians who used the original version of the tool were surveyed with a quantitative instrument that we developed based on a behavior theory framework-the Capability-Opportunity-Motivation-Behavior framework. A semistructured interview guide was developed based on the survey responses. Inductive methods were used to analyze interview session notes and audio recordings from 12 interviews. Revised versions of the tool were developed that incorporated nudges. RESULTS: Functional prototypes were developed by using Axure PRO (Axure Software Solutions) software and usability tested with end users in an iterative agile process (n=10). The tool was redesigned to address 4 identified major barriers to tool use; we included 2 nudges and a default. The 6-month pilot trial for the tool was launched on October 1, 2021. CONCLUSIONS: Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers, along with conducting traditional usability testing, facilitated the development of a tool with greater potential to transform clinical care. The tool will be tested in a prospective pilot trial. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42653.

A framework for digital health equity.

We present a comprehensive Framework for Digital Health Equity, detailing key digital determinants of health (DDoH), to support the work of digital health tool creators in industry, health systems operations, and academia. The rapid digitization of healthcare may widen health disparities if solutions are not developed with these determinants in mind. Our framework builds on the leading health disparities framework, incorporating a digital environment domain. We examine DDoHs at the individual, interpersonal, community, and societal levels, discuss the importance of a root cause, multi-level approach, and offer a pragmatic case study that applies our framework.

Association of race/ethnicity with mortality in patients hospitalized with COVID-19.

OBJECTIVE: To evaluate racial and ethnic differences in mortality among patients hospitalized with coronavirus disease 2019 (COVID-19) after adjusting for baseline characteristics and comorbidities. METHODS: This retrospective cohort study at 13 acute care facilities in the New York City metropolitan area included sequentially hospitalized patients between March 1, 2020, and April 27, 2020. Last day of follow up was July 31, 2020. Patient demographic information, including race/ethnicity and comorbidities, were collected. The primary outcome was in-hospital mortality. RESULTS: A total of 10 869 patients were included in the study (median age, 65 years [interquartile range (IQR) 54-77; range, 18-107 years]; 40.5% female). In adjusted time-to-event analysis, increased age, male sex, insurance type (Medicare and Self-Pay), unknown smoking status, and a higher score on the Charlson Comorbidity Index were significantly associated with higher in-hospital mortality. Adjusted risk of hospital mortality for Black, Asian, Hispanic, multiracial/other, and unknown race/ethnicity patients were similar to risk for White patients. CONCLUSIONS: In a large diverse cohort of patients hospitalized with COVID-19, patients from racial/ethnic minorities experienced similar mortality risk as White patients.

Comparison of Chest Radiograph Impressions for Diagnosing Pneumonia: Accounting for Categories of Language Certainty.

OBJECTIVES: Uncertain language in chest radiograph (CXR) reports for the diagnosis of pneumonia is prevalent. The purpose of this study is to validate an a priori stratification of CXR results for diagnosing pneumonia based on language of certainty. DESIGN: Retrospective chart review. SETTING AND PARTICIPANTS: CXR reports of 2,411 patient visits ≥ 18 years, admitted to medicine, who received a CXR and noncontrast chest CT within 48 hours of emergency department registration at two large academic hospitals (tertiary and quaternary care) were reviewed. METHODS: The CXR and CT report impressions were categorized as negative, uncertain, or positive. Uncertain CXRs were further stratified into four categories based on language modifiers for the degree of pneumonia certainty. Comparison of CXR and CT results were determined using χ2 test; a P value of less than .0033 was considered significant to account for multiple comparisons. RESULTS: CXR reports for the diagnosis of pneumonia revealed the following distribution: 61% negative, 32% uncertain, and 7% positive; CT reports were 55% negative, 22% uncertain, and 23% positive for the diagnosis of pneumonia. There were significant differences between CXR categories compared with CT categories for diagnosis of pneumonia (P < .001). Negative CXR results were not significantly different than the uncertain category with the most uncertain language (P = .030) but were significantly different from all other uncertain categories and positive CXR results (each P < .001). Positive CXR results were not significantly different than the least uncertain category (most certain language) (P = .130) but were significantly different from all other categories (each P < .001). CONCLUSIONS AND IMPLICATIONS: Language used in CXR reports to diagnose pneumonia exists in categories of varying certainty and should be considered when evaluating patients for pneumonia.

The effect of race coefficients on preemptive listing for kidney transplantation.

Background: Race coefficients of estimated glomerular filtration rate (eGFR) formulas may be partially responsible for racial inequality in preemptive listing for kidney transplantation. Methods: We used the Scientific Registry of Transplant Recipients database to evaluate differences in racial distribution of preemptive listing before and after application of the Modification of Diet in Renal Disease (MDRD) and the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) race coefficients to all preemptively listed non-Black kidney transplant candidates (eGFR modulation). Odds of preemptive listing were calculated by race, with Black as the reference before and after eGFR modulation. Variables known to influence preemptive listing were included in the model. Results: Among 385 087 kidney-alone transplant candidates from 1 January 2010 to 2 December 2020, 118 329 (30.7%) candidates were identified as preemptively listed (71.7% White, 19% Black, 7.8% Asian, 0.6% multi-racial, 0.6% Native American and 0.3% Pacific Islander). After eGFR modulation, non-Black patients with an eGFR ≥20 mL/min/1.73 m2 were removed. Compared with Black candidates, the adjusted odds of preemptive listing for White candidates decreased from 2.01 [95% confidence interval (95% CI) 1.78-2.26] before eGFR modulation to 1.18 (95% CI 1.0-1.39; P = 0.046) with the MDRD and 1.37 (95% CI 1.18-1.58) with the CKD-EPI equations after adjusting for race coefficients. Conclusions: Removing race coefficients in GFR estimation formulas may result in a more equitable distribution of Black candidates listed earlier on a preemptive basis.

Automated Pulmonary Embolism Risk Assessment Using the Wells Criteria: Validation Study.

BACKGROUND: Computed tomography pulmonary angiography (CTPA) is frequently used in the emergency department (ED) for the diagnosis of pulmonary embolism (PE), while posing risk for contrast-induced nephropathy and radiation-induced malignancy. OBJECTIVE: We aimed to create an automated process to calculate the Wells score for pulmonary embolism for patients in the ED, which could potentially reduce unnecessary CTPA testing. METHODS: We designed an automated process using electronic health records data elements, including using a combinatorial keyword search method to query free-text fields, and calculated automated Wells scores for a sample of all adult ED encounters that resulted in a CTPA study for PE at 2 tertiary care hospitals in New York, over a 2-month period. To validate the automated process, the scores were compared to those derived from a 2-clinician chart review. RESULTS: A total of 202 ED encounters resulted in a completed CTPA to form the retrospective study cohort. Patients classified as "PE likely" by the automated process (126/202, 62%) had a PE prevalence of 15.9%, whereas those classified as "PE unlikely" (76/202, 38%; Wells score >4) had a PE prevalence of 7.9%. With respect to classification of the patient as "PE likely," the automated process achieved an accuracy of 92.1% when compared with the chart review, with sensitivity, specificity, positive predictive value, and negative predictive value of 93%, 90.5%, 94.4%, and 88.2%, respectively. CONCLUSIONS: This was a successful development and validation of an automated process using electronic health records data elements, including free-text fields, to classify risk for PE in ED visits.

Barriers to the Use of Clinical Decision Support for the Evaluation of Pulmonary Embolism: Qualitative Interview Study.

BACKGROUND: Clinicians often disregard potentially beneficial clinical decision support (CDS). OBJECTIVE: In this study, we sought to explore the psychological and behavioral barriers to the use of a CDS tool. METHODS: We conducted a qualitative study involving emergency medicine physicians and physician assistants. A semistructured interview guide was created based on the Capability, Opportunity, and Motivation-Behavior model. Interviews focused on the barriers to the use of a CDS tool built based on Wells' criteria for pulmonary embolism to assist clinicians in establishing pretest probability of pulmonary embolism before imaging. RESULTS: Interviews were conducted with 12 clinicians. Six barriers were identified, including (1) Bayesian reasoning, (2) fear of missing a pulmonary embolism, (3) time pressure or cognitive load, (4) gestalt includes Wells' criteria, (5) missed risk factors, and (6) social pressure. CONCLUSIONS: Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers will be paramount in developing CDS that can meet its potential to transform clinical care.

In-Hospital 30-Day Survival Among Young Adults With Coronavirus Disease 2019: A Cohort Study.

BACKGROUND: Our objective was to characterize young adult patients hospitalized with coronavirus disease 2019 (COVID-19) and identify predictors of survival at 30 days. METHODS: This retrospective cohort study took place at 12 acute care hospitals in the New York City area. Patients aged 18-39 hospitalized with confirmed COVID-19 between March 1 and April 27, 2020 were included in the study. Demographic, clinical, and outcome data were extracted from electronic health record reports. RESULTS: A total of 1013 patients were included in the study (median age, 33 years; interquartile range [IQR], 28-36; 52% female). At the study end point, 940 (92.8%) patients were discharged alive, 18 (1.8%) remained hospitalized, 5 (0.5%) were transferred to another acute care facility, and 50 (4.9%) died. The most common comorbidities in hospitalized young adult patients were obesity (51.2%), diabetes mellitus (14.8%), and hypertension (13%). Multivariable analysis revealed that obesity (adjusted hazard ratio [aHR], 2.71; 95% confidence interval [CI], 1.28-5.73; P = .002) and Charlson comorbidity index score (aHR, 1.20; 95% CI, 1.07-1.35; P = .002) were independent predictors of in-hospital 30-day mortality. CONCLUSIONS: Obesity was identified as the strongest negative predictor of 30-day in-hospital survival in young adults with COVID-19.

External validation of the IMPROVE-DD risk assessment model for venous thromboembolism among inpatients with COVID-19.

There is a need to discriminate which COVID-19 inpatients are at higher risk for venous thromboembolism (VTE) to inform prophylaxis strategies. The IMPROVE-DD VTE risk assessment model (RAM) has previously demonstrated good discrimination in non-COVID populations. We aimed to externally validate the IMPROVE-DD VTE RAM in medical patients hospitalized with COVID-19. This retrospective cohort study evaluated the IMPROVE-DD VTE RAM in adult patients with COVID-19 admitted to one of thirteen Northwell Health hospitals in the New York metropolitan area between March 1, 2020 and April 27, 2020. VTE was defined as new-onset symptomatic deep venous thrombosis or pulmonary embolism. To assess the predictive value of the RAM, the receiver operating characteristic (ROC) curve was plotted and the area under the curve (AUC) was calculated. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Of 9407 patients who met study criteria, 274 patients developed VTE with a prevalence of 2.91%. The VTE rate was 0.41% for IMPROVE-DD score 0-1 (low risk), 1.21% for score 2-3 (moderate risk), and 5.30% for score ≥ 4 (high risk). Approximately 45.7% of patients were classified as high VTE risk, 33.3% moderate risk, and 21.0% low risk. Discrimination of low versus moderate-high VTE risk demonstrated sensitivity 0.971, specificity 0.215, PPV 0.036, and NPV 0.996. ROC AUC was 0.703. In this external validation study, the IMPROVE-DD VTE RAM demonstrated very good discrimination to identify hospitalized COVID-19 patients at low, moderate, and high VTE risk.

The Association of Structural Inequities and Race With Out-of-Hospital Sudden Death During the COVID-19 Pandemic.

[Figure: see text].

Dissemination of child abuse clinical decision support: Moving beyond a single electronic health record.

BACKGROUND: Child maltreatment is a leading cause of pediatric morbidity and mortality. We previously reported on development and implementation of a child abuse clinical decision support system (CA-CDSS) in the Cerner electronic health record (EHR). Our objective was to develop a CA-CDSS in two different EHRs. METHODS: Using the CA-CDSS in Cerner as a template, CA-CDSSs were developed for use in four hospitals in the Northwell Health system who use Allscripts and two hospitals in the University of Wisconsin health system who use Epic. Each system had a combination of triggers, alerts and child abuse-specific order sets. Usability evaluation was done prior to launch of the CA-CDSS. RESULTS: Over an 18-month period, a CA-CDSS was embedded into Epic and Allscripts at two hospital systems. The CA-CDSSs vary significantly from each other in terms of the type of triggers which were able to be used, the type of alert, the ability of the alert to link directly to child abuse-specific order sets and the order sets themselves. CONCLUSIONS: Dissemination of CA-CDSS from one EHR into the EHR in other health care systems is possible but time-consuming and needs to be adapted to the strengths and limitations of the specific EHR. Site-specific usability evaluation, buy-in of multiple stakeholder groups and significant information technology support are needed. These barriers limit scalability and widespread dissemination of CA-CDSS.

How Should Clinicians' Performance Be Assessed When Health Care Organizations Implement Behavioral Architecture That Generates Negative Consequences?

Behavioral interventions have been shown to have powerful effects on human behavior both outside of and within the context of health care. As organizations increasingly adopt behavioral architecture, care must be taken to consider its potential negative consequences. An evidenced-based approach is best, whereby interventions that might have a significant deleterious effect on patients' health outcomes are first tested and rigorously evaluated before being systematically rolled out. In the case of clinical decision support, brief and thorough instructions should be provided for use. Physician performance when using these systems is best measured relatively, in the context of peers with similar training. Responsibility for errors must be shared with clinical team members and system designers.

Impact of Clinical Decision Support on Antibiotic Prescribing for Acute Respiratory Infections: a Cluster Randomized Implementation Trial.

BACKGROUND: Clinical decision support (CDS) is a promising tool for reducing antibiotic prescribing for acute respiratory infections (ARIs). OBJECTIVE: To assess the impact of previously effective CDS on antibiotic-prescribing rates for ARIs when adapted and implemented in diverse primary care settings. DESIGN: Cluster randomized clinical trial (RCT) implementing a CDS tool designed to guide evidence-based evaluation and treatment of streptococcal pharyngitis and pneumonia. SETTING: Two large academic health system primary care networks with a mix of providers. PARTICIPANTS: All primary care practices within each health system were invited. All providers within participating clinic were considered a participant. Practices were randomized selection to a control or intervention group. INTERVENTIONS: Intervention practice providers had access to an integrated clinical prediction rule (iCPR) system designed to determine the risk of bacterial infection from reason for visit of sore throat, cough, or upper respiratory infection and guide evidence-based evaluation and treatment. MAIN OUTCOME(S): Change in overall antibiotic prescription rates. MEASURE(S): Frequency, rates, and type of antibiotics prescribed in intervention and controls groups. RESULTS: 33 primary care practices participated with 541 providers and 100,573 patient visits. Intervention providers completed the tool in 6.9% of eligible visits. Antibiotics were prescribed in 35% and 36% of intervention and control visits, respectively, showing no statistically significant difference. There were also no differences in rates of orders for rapid streptococcal tests (RR, 0.94; P = 0.11) or chest X-rays (RR, 1.01; P = 0.999) between groups. CONCLUSIONS: The iCPR tool was not effective in reducing antibiotic prescription rates for upper respiratory infections in diverse primary care settings. This has implications for the generalizability of CDS tools as they are adapted to heterogeneous clinical contexts. TRIAL REGISTRATION: Clinicaltrials.gov (NCT02534987). Registered August 26, 2015 at https://clinicaltrials.gov.