Feedback at the Point of Care to Decrease Medication Alert Rates in an Electronic Health Record.

Frequently overridden alerts in the electronic health record can highlight alerts that may need revision. This method is a way of fine-tuning clinical decision support. We evaluated the feasibility of a complementary, yet different method that directly involved pediatric emergency department (PED) providers in identifying additional medication alerts that were potentially incorrect or intrusive. We then evaluated the effect subsequent resulting modifications had on alert salience. METHODS: We performed a prospective, interventional study over 34 months (March 6, 2014, to December 31, 2016) in the PED. We implemented a passive alert feedback mechanism by enhancing the native electronic health record functionality on alert reviews. End-users flagged potentially incorrect/bothersome alerts for review by the study's team. The alerts were updated when clinically appropriate and trends of the impact were evaluated. RESULTS: More than 200 alerts were reported from both inside and outside the PED, suggesting an intuitive approach. On average, we processed 4 reviews per week from the PED, with attending physicians as major contributors. The general trend of the impact of these changes seems favorable. DISCUSSION: The implementation of the review mechanism for user-selected alerts was intuitive and sustainable and seems to be able to detect alerts that are bothersome to the end-users. The method should be run in parallel with the traditional data-driven approach to support capturing of inaccurate alerts. CONCLUSIONS: User-centered, context-specific alert feedback can be used for selecting suboptimal, interruptive medication alerts.

Special Requirements of Electronic Health Record Systems in Pediatrics: Clinical Report.

Pediatricians' use of electronic health record (EHR) systems has become nearly ubiquitous in the United States, yet many systems lack full functionality to deliver effective and efficient pediatric care. This clinical report seeks to provide a compendium of core pediatric functionality of importance to child health care providers that may serve as the focus for EHR developers and clinicians as they evaluate their EHR needs. Also reviewed are important but less critical functions, any of which might be of importance in a specific pediatric context. The major areas described here are immunization management, growth and development, social drivers of health tracking, decision support for orders, patient identification, data normalization, privacy, and system functionality standards in pediatric contexts.

Clinical genomics in the world of the electronic health record.

The widespread adoption of electronic health records presents a number of benefits to the field of clinical genomics. They include the ability to return results to the practitioner, to use genetic findings in clinical decision support, and to have data collected in the electronic health record that serve as a source of phenotypic information for analysis purposes. Not all electronic health records are created equal, however. They differ in their features, capabilities, and ease of use. Therefore, to understand the potential of the electronic health record, it is first necessary to understand its capabilities and the impact that implementation strategy has on usability. Specifically, we focus on the following areas: (i) how the electronic health record is used to capture data in clinical practice settings; (ii) how the implementation and configuration of the electronic health record affect the quality and availability of data; (iii) the management of clinical genetic test results and the feasibility of electronic health record integration; and (iv) the challenges of implementing an electronic health record in a research-intensive environment. This is followed by a discussion of the minimum functional requirements that an electronic health record must meet to enable the satisfactory integration of genomic results as well as the open issues that remain.

Body Mass Index Increased at a Large Midwestern Children's Hospital During the COVID-19 Pandemic.

Background: The COVID-19 pandemic has presented a great challenge to children and their families with stay-at-home orders, school closures, decreased exercise opportunities, stress, and potential overeating with home confinement. Our study describes the body mass index (BMI) changes over an entire decade, including a year of the COVID-19 pandemic at a large children's hospital. Methods: With our retrospective observational study, data were extracted from Cincinnati Children's Hospital's Epic electronic medical record, a free-standing children's hospital with 670 inpatient beds and >1.2 million patient encounters per year. Children aged 19 years and under with at least one height and weight were included in the analysis. Results: In all, 2,344,391 encounters were analyzed with 712,945 visits in years 2018-2021. The prevalence of overweight/obesity was relatively stable with a gradual rise from 35% to 36.4% from 2011 to 2020. However, the year of the COVID-19 stay at home and restrictions (2020-2021) showed an increase in overweight/obesity to 39.7% (8.3% increase), with the greatest increase in those with Class 3 obesity from 3.0% to 3.8%. When viewing the change in BMI percentile during the pandemic year compared with the 2 years prior, there was a significantly increasing trend (p < 0.0001). Conclusions: Children attending a large children's hospital showed an increase in overweight/obesity during the COVID-19 pandemic. These data suggest greater efforts are needed to reverse the increase in weight status from the COVID-19 pandemic as obesity is a risk factor for poor outcomes with COVID-19.

User-Centered Evaluation of a Visual Annotation Tool for Rapid Assessment of Pediatric Weight Entry Errors.

Weight entry errors can cause significant patient harm in pediatrics due to pervasive weight-based dosing practices. While computerized algorithms can assist in error detection, they have not achieved high sensitivity and specificity to be further developed as a clinical decision support tool. To train an advanced algorithm, expert-annotated weight errors are essential but difficult to collect. In this study, we developed a visual annotation tool to gather large amounts of expertly annotated pediatric weight charts and conducted a formal user-centered evaluation. Key features of the tool included configurable grid sizes and annotation styles. The user feedback was collected through a structured survey and user clicks on the interface. The results show that the visual annotation tool has high usability (average SUS=86.4). Different combinations of the key features, however, did not significantly improve the annotation efficiency and duration. We have used this tool to collect expert annotations for algorithm development and benchmarking.

Development and Evaluation of an Automated Approach to Detect Weight Abnormalities in Pediatric Weight Charts.

Inaccurate body weight measures can cause critical safety events in clinical settings as well as hindering utilization of clinical data for retrospective research. This study focused on developing a machine learning-based automated weight abnormality detector (AWAD) to analyze growth dynamics in pediatric weight charts and detect abnormal weight values. In two reference-standard based evaluation of real-world clinical data, the machine learning models showed good capacity for detecting weight abnormalities and they significantly outperformed the methods proposed in literature (p-value<0.05). A deep learning model with bi-directional long short-term memory networks achieved the best predictive performance, with AUCs ≥0.989 across the two datasets. The positive predictive value and sensitivity achieved by the system suggested more than 98% screening effort reduction potential in weight abnormality detection. Consequently, we hypothesize that the AWAD, when fully deployed, holds great potential to facilitate clinical research and healthcare delivery that rely on accurate and reliable weight measures.

Inpatient Communication Networks: Leveraging Secure Text-Messaging Platforms to Gain Insight into Inpatient Communication Systems.

OBJECTIVE: This study attempts to characterize the inpatient communication network within a quaternary pediatric academic medical center by applying network analysis methods to secure text-messaging data. METHODS: We used network graphing and statistical software to create network models of an inpatient communication system with secure text-messaging data from physicians, nurses, and other ancillary staff in an academic medical center. Descriptive statistics about the network, users within the network, and visualizations informed the team's understanding of the network and its components. RESULTS: Analysis of messages exchanged over approximately 23 days revealed a large, scale-free network with 4,442 nodes and 59,913 edges. Quantitative description of user behavior (messages sent and received) and network metrics (i.e., importance of nodes within a network) revealed several operational and clinical roles both sending and receiving > 1,000 messages over this time period. While some of these nodes represented expected "dispatcher" roles in our inpatient system, others occupied important frontline clinical roles responsible for bedside clinical care. CONCLUSION: Quantitative and network analysis of secure text-messaging logs revealed several key operational and clinical roles at risk for alert fatigue and information overload. This analysis also revealed a communication network highly reliant on these key roles, meaning disruption to these individuals or their workflows could lead to dysfunction of the communication network. While secure text-messaging applications play increasingly important roles in facilitating inpatient communication, little is understood about the impact these systems have on health care providers. Developing methods to understand and optimize communication between inpatient providers might help operational and clinical leaders to proactively prevent poorly understood pitfalls associated with these systems and build resilient and effective communication structures.

Development and Preliminary Evaluation of a Visual Annotation Tool to Rapidly Collect Expert-Annotated Weight Errors in Pediatric Growth Charts.

Patient weights can be entered incorrectly into electronic health record (EHR) systems. These weight errors can cause significant patient harm especially in pediatrics where weight-based dosing is pervasively used. Determining weight errors through manual chart reviews is impractical in busy clinics, and current EHR alerts are rudimentary. To address these issues, we seek to develop an advanced algorithm to detect weight errors using supervised machine learning techniques. The critical first step is to collect labelled weight errors for algorithm training. In this paper, we designed and preliminarily evaluated a visual annotation tool using Agile software development to achieve the goal of supporting the rapid collection of expert-annotated weight errors. The design was based on the fact that weight errors are infrequent and medical experts can easily spot potential errors. The results show positive user feedback and prepared us for the formal user-centered evaluation as the next step.

A Comparison of Existing Methods to Detect Weight Data Errors in a Pediatric Academic Medical Center.

Dosing errors due to erroneous body weight entry can be mitigated through algorithms designed to detect anomalies in weight patterns. To prepare for the development of a new algorithm for weight-entry error detection, we compared methods for detecting weight anomalies to human annotation, including a regression-based method employed in a real-time web service. Using a random sample of 4,000 growth charts, annotators identified clinically important anomalies with good inter-rater reliability. Performance of the three detection algorithms was variable, with the best performance from the algorithm that takes into account weights collected after the anomaly was recorded. All methods were highly specific, but positive predictive value ranged from < 5% to over 82%. There were 203 records of missed errors, but all of these were either due to no prior data points or errors too small to be clinically significant. This analysis illustrates the need for better weight-entry error detection algorithms.

The Effects of Medication Alerts on Prescriber Response in a Pediatric Hospital.

OBJECTIVE: More than 70% of hospitals in the United States have electronic health records (EHRs). Clinical decision support (CDS) presents clinicians with electronic alerts during the course of patient care; however, alert fatigue can influence a provider's response to any EHR alert. The primary goal was to evaluate the effects of alert burden on user response to the alerts. METHODS: We performed a retrospective study of medication alerts over a 24-month period (1/2013-12/2014) in a large pediatric academic medical center. The institutional review board approved this study. The primary outcome measure was alert salience, a measure of whether or not the prescriber took any corrective action on the order that generated an alert. We estimated the ideal number of alerts to maximize salience. Salience rates were examined for providers at each training level, by day of week, and time of day through logistic regressions. RESULTS: While salience never exceeded 38%, 49 alerts/day were associated with maximal salience in our dataset. The time of day an order was placed was associated with alert salience (maximal salience 2am). The day of the week was also associated with alert salience (maximal salience on Wednesday). Provider role did not have an impact on salience. CONCLUSION: Alert burden plays a role in influencing provider response to medication alerts. An increased number of alerts a provider saw during a one-day period did not directly lead to decreased response to alerts. Given the multiple factors influencing the response to alerts, efforts focused solely on burden are not likely to be effective.

Automated identification of antibiotic overdoses and adverse drug events via analysis of prescribing alerts and medication administration records.

OBJECTIVES: Electronic trigger detection tools hold promise to reduce Adverse drug event (ADEs) through efficiencies of scale and real-time reporting. We hypothesized that such a tool could automatically detect medication dosing errors as well as manage and evaluate dosing rule modifications. MATERIALS AND METHODS: We created an order and alert analysis system that identified antibiotic medication orders and evaluated user response to dosing alerts. Orders associated with overridden alerts were examined for evidence of administration and the delivered dose was compared to pharmacy-derived dosing rules to confirm true overdoses. True overdose cases were reviewed for association with known ADEs. RESULTS: Of 55 546 orders reviewed, 539 were true overdose orders, which lead to 1965 known overdose administrations. Documentation of loose stools and diarrhea was significantly increased following drug administration in the overdose group. Dosing rule thresholds were altered to reflect clinically accurate dosing. These rule changes decreased overall alert burden and improved the salience of alerts. DISCUSSION: Electronic algorithm-based detection systems can identify antibiotic overdoses that are clinically relevant and are associated with known ADEs. The system also serves as a platform for evaluating the effects of modifying electronic dosing rules. These modifications lead to decreased alert burden and improvements in response to decision support alerts. CONCLUSION: The success of this test case suggests that gains are possible in reducing medication errors and improving patient safety with automated algorithm-based detection systems. Follow-up studies will determine if the positive effects of the system persist and if these changes lead to improved safety outcomes.

Implementing pediatric growth charts into an electronic health record system.

Electronic health record (EHR) systems are increasingly being adopted in pediatric practices; however, requirements for integrated growth charts are poorly described and are not standardized in current systems. The authors integrated growth chart functionality into an EHR system being developed and installed in a multispecialty pediatric clinic in an academic medical center. During a three-year observation period, rates of electronically documented values for weight, stature, and head circumference increased from fewer than ten total per weekday, up to 488 weight values, 293 stature values, and 74 head circumference values (p<0.001 for each measure). By the end of the observation period, users accessed the growth charts an average 175 times per weekday, compared to 127 patient visits per weekday to the sites that most closely monitored pediatric growth. Because EHR systems and integrated growth charts can manipulate data, perform calculations, and adapt to user preferences and patient characteristics, users may expect greater functionality from electronic growth charts than from paper-based growth charts.

Meaningful Use of Electronic Health Records: Experiences From the Field and Future Opportunities.

BACKGROUND: With the aim of improving health care processes through health information technology (HIT), the US government has promulgated requirements for "meaningful use" (MU) of electronic health records (EHRs) as a condition for providers receiving financial incentives for the adoption and use of these systems. Considerable uncertainty remains about the impact of these requirements on the effective application of EHR systems. OBJECTIVE: The Agency for Healthcare Research and Quality (AHRQ)-sponsored Centers for Education and Research in Therapeutics (CERTs) critically examined the impact of the MU policy relating to the use of medications and jointly developed recommendations to help inform future HIT policy. METHODS: We gathered perspectives from a wide range of stakeholders (N=35) who had experience with MU requirements, including academicians, practitioners, and policy makers from different health care organizations including and beyond the CERTs. Specific issues and recommendations were discussed and agreed on as a group. RESULTS: Stakeholders' knowledge and experiences from implementing MU requirements fell into 6 domains: (1) accuracy of medication lists and medication reconciliation, (2) problem list accuracy and the shift in HIT priorities, (3) accuracy of allergy lists and allergy-related standards development, (4) support of safer and effective prescribing for children, (5) considerations for rural communities, and (6) general issues with achieving MU. Standards are needed to better facilitate the exchange of data elements between health care settings. Several organizations felt that their preoccupation with fulfilling MU requirements stifled innovation. Greater emphasis should be placed on local HIT configurations that better address population health care needs. CONCLUSIONS: Although MU has stimulated adoption of EHRs, its effects on quality and safety remain uncertain. Stakeholders felt that MU requirements should be more flexible and recognize that integrated models may achieve information-sharing goals in alternate ways. Future certification rules and requirements should enhance EHR functionalities critical for safer prescribing of medications in children.

A model for assessing information retrieval and application skills of medical students.

PURPOSE: To develop and evaluate a model for assessing information retrieval and application skills, and to compare the performances on the assessment exercises of students who were and were not instructed in these skills. METHOD: The authors developed a set of four examination stations, each with multiple subtasks, and administered the exams to students at two medical schools. Students at one school had intensive instruction in literature searching and filtering skills for information quality (instructed group), and those at the other school had minimal instruction in these areas (uninstructed group). The stations addressed pediatrics content and the skills of searching Medline and the World Wide Web, evaluating research articles, evaluating the accuracy of information from the Web, and using the information to make recommendations to patients. The authors determined the psychometric characteristics of the stations and compared the performances of the two groups of students. RESULTS: Students in the instructed group performed significantly better and with less variability than the uninstructed group on four tasks and no differently on seven tasks. There was no task on which the uninstructed group performed significantly better than the instructed group. CONCLUSION: The prototype stations showed predictable differences across curricula, indicating that they have promise as assessment tools for the essential skills of information retrieval and application.

Barriers to EMR adoption in internal medicine and pediatric outpatient practices.

BACKGROUND: Although electronic medical records (EMRs) are widely regarded as valuable tools in patient care, physicians in outpatient practices have been slow to adopt them. We sought to determine the current use of EMRs in area practices and identify physician attitudes related to their adoption. METHODS: Fax and mail survey of randomly selected physician representatives of all outpatient practices of Internal Medicine (n=51) and Pediatrics (n=24) in Shelby County, Tenn. Scores on eight physician attitudes regarding barriers to EMR adoption were obtained using a Likert scale. RESULTS: Survey response rate was 55%, with 18% reporting current EMR use. This corresponds to an EMR penetration of 20% for Shelby County. Current users were significantly less likely (P=0.005) than non-users to feel that an EMR interferes with doctor-patient interaction and less likely (P=0.019) to have EMR privacy concerns. While differences noted in other attitudes did not reach statistical significance, a trend was seen toward EMR users being less concerned (P=.0502) about reliability of an EMR. Large practices were no more likely than smaller ones to be using an EMR. Internal Medicine and Pediatric participants responded similarly to all items. The number of years in practice had no demonstrable impact on physician responses to these survey items. CONCLUSIONS: In this West Tennessee physician population, EMR user and non-user attitudes markedly differed about impact on doctor-patient interaction and patient privacy. If such concerns could be addressed to the satisfaction of physicians considering EMRs in their practice, adoption rates might be increased.

Evaluating the accuracy of electronic pediatric drug dosing rules.

OBJECTIVE: To determine the accuracy of vendor-supplied dosing eRules for pediatric medication orders. Inaccurate or absent dosing rules can lead to high numbers of false alerts or undetected prescribing errors and may potentially compromise safety in this already vulnerable population. MATERIALS AND METHODS: 7 months of medication orders and alerts from a large pediatric hospital were analyzed. 30 medications were selected for study across 5 age ranges and 5 dosing parameters. The resulting 750 dosing rules from a commercial system formed the study corpus and were examined for accuracy against a gold standard created from traditional clinical resources. RESULTS: Overall accuracy of the rules in the study corpus was 55.1% when the rules were transformed to fit a priori age ranges. Over a pediatric lifetime, the dosing rules were accurate an average of 57.6% of the days. Dosing rules pertaining to the newborn age range were as accurate as other age ranges on average, but exhibited more variability. Daily frequency dosing parameters showed more accuracy than total daily dose, single dose minimum, or single dose maximum. DISCUSSION: The accuracy of a vendor-supplied set of dosing eRules is suboptimal when compared with traditional dosing sources, exposing a gap between dosing rules in commercial products and actual prescribing practices by pediatric care providers. More research on vendor-supplied eRules is warranted in order to understand the effects of these products on safe prescribing in children.

Impact of electronic health record implementation on patient flow metrics in a pediatric emergency department.

Implementing electronic health records (EHR) in healthcare settings incurs challenges, none more important than maintaining efficiency and safety during rollout. This report quantifies the impact of offloading low-acuity visits to an alternative care site from the emergency department (ED) during EHR implementation. In addition, the report evaluated the effect of EHR implementation on overall patient length of stay (LOS), time to medical provider, and provider productivity during implementation of the EHR. Overall LOS and time to doctor increased during EHR implementation. On average, admitted patients' LOS was 6-20% longer. For discharged patients, LOS was 12-22% longer. Attempts to reduce patient volumes by diverting patients to another clinic were not effective in minimizing delays in care during this EHR implementation. Delays in ED throughput during EHR implementation are real and significant despite additional providers in the ED, and in this setting resolved by 3 months post-implementation.

Automated dose-rounding recommendations for pediatric medications.

BACKGROUND: Although pediatric electronic prescribing systems are increasingly being used in pediatric care, many of these systems lack the clinical decision-support infrastructure needed to calculate a safe and effective rounded medication dose. This infrastructure is required to facilitate tailoring of established dosing guidance while maintaining the medication's therapeutic intent. OBJECTIVE: The goal of this project was to establish best practices for generating an appropriate medication dose and to create an interoperable rounding knowledge base combining best practices and dose-rounding information. METHODS: We interviewed 19 pediatric health care and pediatric pharmacy experts and conducted a literature review. After using these data to construct initial rounding tolerances, we used a Delphi process to achieve consensus about the rounding tolerance for each commonly prescribed medication. RESULTS: Three categories for medication-rounding philosophy emerged from our literature review: (1) medications for which rounding is used judiciously to retain the intended effect; (2) medications that are rounded with attention to potential unintended effects; and (3) medications that are rarely rounded because of the potential for toxicity. We assigned a small subset of medications to a fourth category-inadequate data-for which there was insufficient information to provide rounding recommendations. For all 102 medications, we were able to arrive at a consensus recommendation for rounding a given calculated dose. CONCLUSIONS: Results of this study provide the pediatric information technology community with a primary set of recommended rounding tolerances for commonly prescribed drugs. The interoperable knowledge base developed here can be integrated with existing and developing electronic prescribing systems, potentially improving prescribing safety and reducing cognitive workload.