Brief report: Improving communication of diagnostic uncertainty to families of hospitalized children.

OBJECTIVES: Diagnostic uncertainty is not reliably communicated to patients and caregivers. This study aims to identify barriers and facilitators to effective communication of diagnostic uncertainty, including development of potential tools and strategies for improvement, as perceived by healthcare professionals and caregivers. METHODS: We completed structured interviews with providers and caregivers of hospitalized children with uncertain diagnoses (UD). The interview guides addressed barriers to communication, key components for communication of uncertainty, and qualities of effective communication. The interviews concluded with respondents prioritizing potential interventions to improve communication of uncertainty. Interviews were audio recorded, transcribed, and independently analyzed by two team members to identify common themes. RESULTS: Ten provider and five caregiver interviews were conducted. Common barriers to communication of uncertainty included time constraints, language barriers, and lack of clear definition of UD. Caregiver suggestions for improvement included sharing expectations of the diagnostic process and use of both written and visual communication tools. Interview respondents favored interventions of a sign summarizing the key components of diagnostic uncertainty for display in patient rooms and a structured diagnostic pause during daily rounds. CONCLUSIONS: We identified several potential interventions that may enhance communication of diagnostic uncertainty and better engage patients and caregivers in the diagnostic process.

Developing a machine learning model to detect diagnostic uncertainty in clinical documentation.

BACKGROUND AND OBJECTIVE: Diagnostic uncertainty, when unrecognized or poorly communicated, can result in diagnostic error. However, diagnostic uncertainty is challenging to study due to a lack of validated identification methods. This study aims to identify distinct linguistic patterns associated with diagnostic uncertainty in clinical documentation. DESIGN, SETTING AND PARTICIPANTS: This case-control study compares the clinical documentation of hospitalized children who received a novel uncertain diagnosis (UD) diagnosis label during their admission to a set of matched controls. Linguistic analyses identified potential linguistic indicators (i.e., words or phrases) of diagnostic uncertainty that were then manually reviewed by a linguist and clinical experts to identify those most relevant to diagnostic uncertainty. A natural language processing program categorized medical terminology into semantic types (i.e., sign or symptom), from which we identified a subset of these semantic types that both categorized reliably and were relevant to diagnostic uncertainty. Finally, a competitive machine learning modeling strategy utilizing the linguistic indicators and semantic types compared different predictive models for identifying diagnostic uncertainty. RESULTS: Our cohort included 242 UD-labeled patients and 932 matched controls with a combination of 3070 clinical notes. The best-performing model was a random forest, utilizing a combination of linguistic indicators and semantic types, yielding a sensitivity of 89.4% and a positive predictive value of 96.7%. CONCLUSION: Expert labeling, natural language processing, and machine learning methods combined with human validation resulted in highly predictive models to detect diagnostic uncertainty in clinical documentation and represent a promising approach to detecting, studying, and ultimately mitigating diagnostic uncertainty in clinical practice.

Diagnosis Code and Health Care Utilization Patterns Associated With Diagnostic Uncertainty.

BACKGROUND AND OBJECTIVES: Diagnostic uncertainty is challenging to identify and study in clinical practice. This study compares differences in diagnosis code and health care utilization between a unique cohort of hospitalized children with uncertain diagnoses (UD) and matched controls. PATIENTS AND METHODS: This case-control study was conducted at Cincinnati Children's Hospital Medical Center. Cases were defined as patients admitted to the pediatric hospital medicine service and having UDs during their hospitalization. Control patients were matched on age strata, biological sex, and time of year. Outcomes included type of diagnosis codes used (ie, disease- or nondisease-based) and change in code from admission to discharge. Differences in diagnosis codes were evaluated using conditional logistic regression. Health care utilization outcomes included hospital length of stay (LOS), hospital transfer, consulting service utilization, rapid response team activations, escalation to intensive care, and 30-day health care reutilization. Differences in health care utilization were assessed using bivariate statistics. RESULTS: Our final cohort included 240 UD cases and 911 matched controls. Compared with matched controls, UD cases were 8 times more likely to receive a nondisease-based diagnosis code (odds ratio [OR], 8.0; 95% confidence interval [CI], 5.7-11.2) and 2.5 times more likely to have a change in their primary International Classification of Disease, 10th revision, diagnosis code between admission and discharge (OR, 2.5; 95% CI, 1.9-3.4). UD cases had a longer average LOS and higher transfer rates to our main hospital campus, consulting service use, and 30-day readmission rates. CONCLUSIONS: Hospitalized children with UDs have meaningfully different patterns of diagnosis code use and increased health care utilization compared with matched controls.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts.

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

Aligning Provider Prescribing With Guidelines for Soft Tissue Infections.

OBJECTIVE: Despite evidence-based guidelines, antibiotics prescribed for uncomplicated skin and soft tissue infections can involve inappropriate microbial coverage. Our aim was to evaluate the appropriateness of antibiotic prescribing practices for mild nonpurulent cellulitis in a pediatric tertiary academic medical center over a 1-year period. METHODS: Eligible patients treated in the emergency department or urgent care settings for mild nonpurulent cellulitis from January 2017 to December 2017 were identified by an International Classification of Diseases, Tenth Revision, code for cellulitis. The primary outcome was appropriateness of prescribed antibiotics as delineated by adherence with the Infectious Diseases Society of America guidelines. Secondary outcomes include reutilization rate as defined by revisit to the emergency department/urgent cares within 14 days of the initial encounter. RESULTS: A total of 967 encounters were evaluated with 60.0% overall having guideline-adherent care. Common reasons for nonadherence included inappropriate coverage of MRSA with clindamycin (n = 217, 56.1%) and single-agent coverage with sulfamethoxazole-trimethoprim (n = 129, 33.3%). There were 29 revisits within 14 days of initial patient encounters or a reutilization rate of 3.0%, which was not significantly associated with the Infectious Diseases Society of America adherence. CONCLUSIONS: Our data show antibiotic prescription for nonpurulent cellulitis as a potential area of standardization and optimization of care at our center.

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics.

BACKGROUND: Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden. OBJECTIVE: (1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics. MATERIALS AND METHODS: We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric. RESULTS: Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden. CONCLUSION: Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.

Declaring Uncertainty: Using Quality Improvement Methods to Change the Conversation of Diagnosis.

BACKGROUND: Diagnostic uncertainty may be a sign that a patient's working diagnosis is incorrect, but literature on proactively identifying diagnostic uncertainty is lacking. Using quality improvement methodologies, we aimed to create a process for identifying patients with uncertain diagnoses (UDs) on a pediatric inpatient unit and communicating about them with the interdisciplinary health care team. METHODS: Plan-do-study-act cycles were focused on interdisciplinary communication, structured handoffs, and integration of diagnostic uncertainty into the electronic medical record. Our definition of UD was as follows: "you wouldn't be surprised if the patient had a different diagnosis that required a change in management." The primary measure, which was tracked on an annotated run chart, was percentage agreement between the charge nurse and primary clinician regarding which patients had a UD. Secondary measures included the percentage of patient days during which patients had UDs. Data were collected 3 times daily by text message polls. RESULTS: Over 13 months, the percentage agreement between the charge nurse and primary clinician about which patients had UDs increased from a baseline of 19% to a median of 84%. On average, patients had UDs during 11% of patient days. CONCLUSIONS: We created a novel and effective process to improve shared recognition of patients with diagnostic uncertainty among the interdisciplinary health care team, which is an important first step in improving care for these patients.

Uncertain diagnoses in a children's hospital: patient characteristics and outcomes.

BACKGROUND: A quality improvement initiative at our institution resulted in a new process for prospectively identifying pediatric hospital medicine (PHM) patients with uncertain diagnoses (UD). This study describes the clinical characteristics and healthcare utilization patterns of patients with UD. METHODS: This single center cross-sectional study included all PHM patients identified with UD during their admission. A structured chart review was used to abstract patient demographics, primary symptoms, discharge diagnoses, and healthcare utilization patterns, including consult service use, length of stay (LOS), escalation in care, and 30-day healthcare reutilization. Appropriate descriptive statistics were used for categorical and continuous variables. RESULTS: This study includes 200 PHM patients identified with UD. Gastrointestinal symptoms were the primary finding in 45% of patients with UD. Consult service use was highly variable, with a range of 0-8 consult services for individual patients. The median LOS was 1.6 days and only 5% required a rapid response team evaluation. As for reutilization, 7% of patients were readmitted within 30 days. CONCLUSIONS: This descriptive study highlights the heterogeneity of patients with uncertain diagnoses. Ongoing work is needed to further understand the impact of UD and to optimize the care of these patients.

Increasing Physician Reporting of Diagnostic Learning Opportunities.

BACKGROUND: An estimated 10% of Americans experience a diagnostic error annually, yet little is known about pediatric diagnostic errors. Physician reporting is a promising method for identifying diagnostic errors. However, our pediatric hospital medicine (PHM) division had only 1 diagnostic-related safety report in the preceding 4 years. We aimed to improve attending physician reporting of suspected diagnostic errors from 0 to 2 per 100 PHM patient admissions within 6 months. METHODS: Our improvement team used the Model for Improvement, targeting the PHM service. To promote a safe reporting culture, we used the term diagnostic learning opportunity (DLO) rather than diagnostic error, defined as a "potential opportunity to make a better or more timely diagnosis." We developed an electronic reporting form and encouraged its use through reminders, scheduled reflection time, and monthly progress reports. The outcome measure, the number of DLO reports per 100 patient admissions, was tracked on an annotated control chart to assess the effect of our interventions over time. We evaluated DLOs using a formal 2-reviewer process. RESULTS: Over the course of 13 weeks, there was an increase in the number of reports filed from 0 to 1.6 per 100 patient admissions, which met special cause variation, and was subsequently sustained. Most events (66%) were true diagnostic errors and were found to be multifactorial after formal review. CONCLUSIONS: We used quality improvement methodology, focusing on psychological safety, to increase physician reporting of DLOs. This growing data set has generated nuanced learnings that will guide future improvement work.

Feedback at the Point of Care to Decrease Medication Alert Rates in an Electronic Health Record.

Frequently overridden alerts in the electronic health record can highlight alerts that may need revision. This method is a way of fine-tuning clinical decision support. We evaluated the feasibility of a complementary, yet different method that directly involved pediatric emergency department (PED) providers in identifying additional medication alerts that were potentially incorrect or intrusive. We then evaluated the effect subsequent resulting modifications had on alert salience. METHODS: We performed a prospective, interventional study over 34 months (March 6, 2014, to December 31, 2016) in the PED. We implemented a passive alert feedback mechanism by enhancing the native electronic health record functionality on alert reviews. End-users flagged potentially incorrect/bothersome alerts for review by the study's team. The alerts were updated when clinically appropriate and trends of the impact were evaluated. RESULTS: More than 200 alerts were reported from both inside and outside the PED, suggesting an intuitive approach. On average, we processed 4 reviews per week from the PED, with attending physicians as major contributors. The general trend of the impact of these changes seems favorable. DISCUSSION: The implementation of the review mechanism for user-selected alerts was intuitive and sustainable and seems to be able to detect alerts that are bothersome to the end-users. The method should be run in parallel with the traditional data-driven approach to support capturing of inaccurate alerts. CONCLUSIONS: User-centered, context-specific alert feedback can be used for selecting suboptimal, interruptive medication alerts.

Inpatient Communication Networks: Leveraging Secure Text-Messaging Platforms to Gain Insight into Inpatient Communication Systems.

OBJECTIVE: This study attempts to characterize the inpatient communication network within a quaternary pediatric academic medical center by applying network analysis methods to secure text-messaging data. METHODS: We used network graphing and statistical software to create network models of an inpatient communication system with secure text-messaging data from physicians, nurses, and other ancillary staff in an academic medical center. Descriptive statistics about the network, users within the network, and visualizations informed the team's understanding of the network and its components. RESULTS: Analysis of messages exchanged over approximately 23 days revealed a large, scale-free network with 4,442 nodes and 59,913 edges. Quantitative description of user behavior (messages sent and received) and network metrics (i.e., importance of nodes within a network) revealed several operational and clinical roles both sending and receiving > 1,000 messages over this time period. While some of these nodes represented expected "dispatcher" roles in our inpatient system, others occupied important frontline clinical roles responsible for bedside clinical care. CONCLUSION: Quantitative and network analysis of secure text-messaging logs revealed several key operational and clinical roles at risk for alert fatigue and information overload. This analysis also revealed a communication network highly reliant on these key roles, meaning disruption to these individuals or their workflows could lead to dysfunction of the communication network. While secure text-messaging applications play increasingly important roles in facilitating inpatient communication, little is understood about the impact these systems have on health care providers. Developing methods to understand and optimize communication between inpatient providers might help operational and clinical leaders to proactively prevent poorly understood pitfalls associated with these systems and build resilient and effective communication structures.

Integrating Bayesian reasoning into medical education using smartphone apps.

Background Diagnostic reasoning is an important topic in medical education, and diagnostic errors are increasingly recognized as large contributors to patient morbidity and mortality. One way to improve learner understanding of the diagnostic process is to teach the concepts of Bayesian reasoning and to make these concepts practical for clinical use. Many clinician educators do not fully understand Bayesian concepts and they lack the tools to incorporate Bayesian reasoning into clinical practice and teaching. Methods The authors developed an interactive workshop using visual models of probabilities and thresholds, clinical cases, and available smartphone apps to teach learners about Bayesian concepts. Results Evaluations from 3 years of workshops at a national internal medicine chief resident conference showed high satisfaction, with narrative comments suggesting learners found the visual and smartphone tools useful for applying the concepts with future learners. Conclusions Visual models, clinical cases, and smartphone apps were well received by chief residents as a way to learn and teach Bayesian reasoning. Further study will be needed to understand if these tools can improve diagnostic accuracy or patient outcomes.

Reducing Interdisciplinary Communication Failures Through Secure Text Messaging: A Quality Improvement Project.

INTRODUCTION: Interdisciplinary communication failures contribute to medical mistakes and adverse events. At our institution, provider communication previously occurred through unidirectional pager systems. We utilized quality improvement methodology to (1) implement a secure text messaging system for providers on a pediatric ward and (2) evaluate its impact on communication failures. We aimed to reduce potential communication failures between providers by > 25% within 1 month. METHODS: Implementation of secure text messaging occurred via Plan-Do-Study-Act cycles focused on education, feedback, and electronic health record interventions. We collected pager data before implementation and both pager and secure text messaging data after intervention. Potential communication failures were identified a priori through manual review of the messaging data to capture lack of closed-loop communication. A run chart was used to track daily potential communication failures and total communication volumes. RESULTS: Before implementation of secure text messaging, the median daily potential communication failure rate was 5.5%. Usage of secure text messaging increased after implementation, representing 3.5 of 7.2 communications per patient-day. Paging communications decreased from 4.2 to 3.7 per patient-day. Potential communication failures decreased to a median daily rate of 2.2%, representing a 59% reduction in communication failures. CONCLUSION: Implementation of secure text messaging using quality improvement methods resulted in a significant reduction in potential communication failures between residents and nurses. Future interventions will be aimed at maintaining and augmenting providers' use of secure text messaging to ensure the potential for communication failure remains low.

Reducing Electrolyte Testing in Hospitalized Children by Using Quality Improvement Methods.

BACKGROUND AND OBJECTIVES: Despite studies indicating a high rate of overuse, electrolyte testing remains common in pediatric inpatient care. Frequently repeated electrolyte tests often return normal results and can lead to patient harm and increased cost. We aimed to reduce electrolyte testing within a hospital medicine service by >25% within 6 months. METHODS: We conducted an improvement project in which we targeted 6 hospital medicine teams at a large academic children's hospital system by using the Model for Improvement. Interventions included standardizing communication about the electrolyte testing plan and education about the costs and risks associated with overuse of electrolyte testing. Our primary outcome measure was the number of electrolyte tests per patient day. Secondary measures included testing charges and usage rates of specific high-charge panels. We tracked medical emergency team calls and readmission rates as balancing measures. RESULTS: The mean baseline rate of electrolyte testing was 2.0 laboratory draws per 10 patient days, and this rate decreased by 35% after 1 month of initial educational interventions to 1.3 electrolyte laboratory draws per 10 patient days. This change has been sustained for 9 months and could save an estimated $292 000 in patient-level charges over the course of a year. Use of our highest-charge electrolyte panel decreased from 67% to 22% of testing. No change in rates of medical emergency team calls or readmission were found. CONCLUSIONS: Our improvement intervention was associated with significant and rapid reduction in electrolyte testing and has not been associated with unintended adverse events.

The Effects of Medication Alerts on Prescriber Response in a Pediatric Hospital.

OBJECTIVE: More than 70% of hospitals in the United States have electronic health records (EHRs). Clinical decision support (CDS) presents clinicians with electronic alerts during the course of patient care; however, alert fatigue can influence a provider's response to any EHR alert. The primary goal was to evaluate the effects of alert burden on user response to the alerts. METHODS: We performed a retrospective study of medication alerts over a 24-month period (1/2013-12/2014) in a large pediatric academic medical center. The institutional review board approved this study. The primary outcome measure was alert salience, a measure of whether or not the prescriber took any corrective action on the order that generated an alert. We estimated the ideal number of alerts to maximize salience. Salience rates were examined for providers at each training level, by day of week, and time of day through logistic regressions. RESULTS: While salience never exceeded 38%, 49 alerts/day were associated with maximal salience in our dataset. The time of day an order was placed was associated with alert salience (maximal salience 2am). The day of the week was also associated with alert salience (maximal salience on Wednesday). Provider role did not have an impact on salience. CONCLUSION: Alert burden plays a role in influencing provider response to medication alerts. An increased number of alerts a provider saw during a one-day period did not directly lead to decreased response to alerts. Given the multiple factors influencing the response to alerts, efforts focused solely on burden are not likely to be effective.

Following the (Clinical Decision) Rules: Opportunities for Improving Safety and Resource Utilization With the Bacterial Meningitis Score.

BACKGROUND: The Bacterial Meningitis Score accurately classifies children with cerebrospinal fluid (CSF) pleocytosis at very low risk (VLR) versus not very low risk (non-VLR) for bacterial meningitis. Most children with CSF pleocytosis detected during emergency department evaluation are hospitalized despite the high accuracy of this prediction rule and the decreasing incidence of bacterial meningitis. The lack of widespread use of this rule may contribute to unnecessary risk exposure and costs. METHODS: This cross-sectional study included 1049 patients who, between January 2010 and May 2013, had suspicion for meningitis and underwent both a complete blood cell count and CSF studies during their emergency department evaluation. We then examined their hospitalizations to characterize exposure to drugs, radiologic studies, and the costs associated with their care to determine the safety and value repercussions of these VLR admissions. Primary outcomes include duration of antibiotics, exposure to drugs and radiology studies, safety events, and costs incurred during these VLR admissions. RESULTS: Twenty patients classified as VLR were admitted to the hospital. On average they received 35 hours of antibiotic therapy. There was 1 adverse drug event and 1 safety event. CONCLUSIONS: The VLR patients admitted to the hospital were exposed to risk and costs despite their low risk stratification. Systematic application of the Bacterial Meningitis Score could prevent these exposures and costs.