Developing a sampling method and preliminary taxonomy for classifying COVID-19 public health guidance for healthcare organizations and the general public.

BACKGROUND: Development and dissemination of public health (PH) guidance to healthcare organizations and the general public (e.g., businesses, schools, individuals) during emergencies like the COVID-19 pandemic is vital for policy, clinical, and public decision-making. Yet, the rapidly evolving nature of these events poses significant challenges for guidance development and dissemination strategies predicated on well-understood concepts and clearly defined access and distribution pathways. Taxonomies are an important but underutilized tool for guidance authoring, dissemination and updating in such dynamic scenarios. OBJECTIVE: To design a rapid, semi-automated method for sampling and developing a PH guidance taxonomy using widely available Web crawling tools and streamlined manual content analysis. METHODS: Iterative samples of guidance documents were taken from four state PH agency websites, the US Center for Disease Control and Prevention, and the World Health Organization. Documents were used to derive and refine a preliminary taxonomy of COVID-19 PH guidance via content analysis. RESULTS: Eight iterations of guidance document sampling and taxonomy revisions were performed, with a final corpus of 226 documents. The preliminary taxonomy contains 110 branches distributed between three major domains: stakeholders (24 branches), settings (25 branches) and topics (61 branches). Thematic saturation measures indicated rapid saturation (≤5% change) for the domains of "stakeholders" and "settings", and "topic"-related branches for clinical decision-making. Branches related to business reopening and economic consequences remained dynamic throughout sampling iterations. CONCLUSION: The PH guidance taxonomy can support public health agencies by aligning guidance development with curation and indexing strategies; supporting targeted dissemination; increasing the speed of updates; and enhancing public-facing guidance repositories and information retrieval tools. Taxonomies are essential to support knowledge management activities during rapidly evolving scenarios such as disease outbreaks and natural disasters.

Implementations of the HL7 Context-Aware Knowledge Retrieval ("Infobutton") Standard: challenges, strengths, limitations, and uptake.

UNLABELLED: To support clinical decision-making, computerized information retrieval tools known as "infobuttons" deliver contextually-relevant knowledge resources into clinical information systems. The Health Level Seven International (HL7) Context-Aware Knowledge Retrieval (Infobutton) Standard specifies a standard mechanism to enable infobuttons on a large scale. OBJECTIVE: To examine the experience of organizations in the course of implementing the HL7 Infobutton Standard. METHOD: Cross-sectional online survey and in-depth phone interviews. RESULTS: A total of 17 organizations participated in the study. Analysis of the in-depth interviews revealed 20 recurrent themes. Implementers underscored the benefits, simplicity, and flexibility of the HL7 Infobutton Standard. Yet, participants voiced the need for easier access to standard specifications and improved guidance to beginners. Implementers predicted that the Infobutton Standard will be widely or at least fairly well adopted in the next 5 years, but uptake will depend largely on adoption among electronic health record (EHR) vendors. To accelerate EHR adoption of the Infobutton Standard, implementers recommended HL7-compliant infobutton capabilities to be included in the United States Meaningful Use Certification Criteria for EHR systems. LIMITATIONS: Opinions and predictions should be interpreted with caution, since all the participant organizations have successfully implemented the standard and over half of the organizations were actively engaged in the development of the standard. CONCLUSION: Overall, implementers reported a very positive experience with the HL7 Infobutton Standard. Despite indications of increasing uptake, measures should be taken to stimulate adoption of the Infobutton Standard among EHR vendors. Widespread adoption of the Infobutton Standard has the potential to bring contextually relevant clinical decision support content into the healthcare provider workflow.

A study of diverse clinical decision support rule authoring environments and requirements for integration.

BACKGROUND: Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules. METHODS: The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools. RESULTS: While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users. CONCLUSIONS: A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Tiering drug-drug interaction alerts by severity increases compliance rates.

OBJECTIVE: Few data exist measuring the effect of differentiating drug-drug interaction (DDI) alerts in computerized provider order entry systems (CPOE) by level of severity ("tiering"). We sought to determine if rates of provider compliance with DDI alerts in the inpatient setting differed when a tiered presentation was implemented. DESIGN: We performed a retrospective analysis of alert log data on hospitalized patients at two academic medical centers during the period from 2/1/2004 through 2/1/2005. Both inpatient CPOE systems used the same DDI checking service, but one displayed alerts differentially by severity level (tiered presentation, including hard stops for the most severe alerts) while the other did not. Participants were adult inpatients who generated a DDI alert, and providers who wrote the orders. Alerts were presented during the order entry process, providing the clinician with the opportunity to change the patient's medication orders to avoid the interaction. MEASUREMENTS: Rate of compliance to alerts at a tiered site compared to a non-tiered site. RESULTS: We reviewed 71,350 alerts, of which 39,474 occurred at the non-tiered site and 31,876 at the tiered site. Compliance with DDI alerts was significantly higher at the site with tiered DDI alerts compared to the non-tiered site (29% vs. 10%, p < 0.001). At the tiered site, 100% of the most severe alerts were accepted, vs. only 34% at the non-tiered site; moderately severe alerts were also more likely to be accepted at the tiered site (29% vs. 10%). CONCLUSION: Tiered alerting by severity was associated with higher compliance rates of DDI alerts in the inpatient setting, and lack of tiering was associated with a high override rate of more severe alerts.

Cost-benefit analysis of a hospital pharmacy bar code solution.

BACKGROUND: Bar coding can reduce hospital pharmacy dispensing errors, but it is unclear if the benefits of this technology justify its costs. The purpose of this study was to assess the costs and benefits and determine the return on investment at the institutional level for implementing a pharmacy bar code system. METHODS: We performed a cost-benefit analysis of a bar code-assisted medication-dispensing system within a large, academic, nonprofit tertiary care hospital pharmacy. We took the implementing hospital's perspective for a 5-year horizon. The primary outcome was the net financial cost and benefit after 5 years. The secondary outcome was the time until total benefits equaled total costs. Single-variable, 2-variable, and multiple-variable Monte Carlo sensitivity analyses were performed to test the stability of the outcomes. RESULTS: In inflation- and time value-adjusted 2005 dollars, total costs during 5 years were $2.24 million ($1.31 million in 1-time costs during the initial 3.5 years and $342 000 per year in recurring costs starting in year 3). The primary benefit was a decrease in adverse drug events from dispensing errors (517 events annually), resulting in an annual savings of $2.20 million. The net benefit after 5 years was $3.49 million. The break-even point for the hospital's investment occurred within 1 year after becoming fully operational. A net benefit was achieved within 10 years under almost all sensitivity scenarios. In the Monte Carlo simulation, the net benefit during 5 years was $3.2 million (95% confidence interval, -$1.2 million to $12.1 million), and the break-even point for return on investment occurred after 51 months (95% confidence interval, 30 to 180 months). CONCLUSION: Implementation of a bar code-assisted medication-dispensing system in hospital pharmacies can result in a positive financial return on investment for the health care organization.

The Value of Monitoring Clinical Decision Support Interventions.

BACKGROUND: Well-functioning clinical decision support (CDS) can facilitate provider workflow, improve patient care, promote better outcomes, and reduce costs. However, poorly functioning CDS may lead to alert fatigue, cause providers to ignore important CDS interventions, and increase provider dissatisfaction. OBJECTIVE: The purpose of this article is to describe one institution's experience in implementing a program to create and maintain properly functioning CDS by systematically monitoring CDS firing rates and patterns. METHODS: Four types of CDS monitoring activities were implemented as part of the CDS lifecycle. One type of monitoring occurs prior to releasing active CDS, while the other types occur at different points after CDS activation. RESULTS: Two hundred and forty-eight CDS interventions were monitored over a 2-year period. The rate of detecting a malfunction or significant opportunity for improvement was 37% during preactivation and 18% during immediate postactivation monitoring. Monitoring also informed the process of responding to user feedback about alerts. Finally, an automated alert detection tool identified 128 instances of alert pattern change over the same period. A subset of cases was evaluated by knowledge engineers to identify true and false positives, the results of which were used to optimize the tool's pattern detection algorithms. CONCLUSION: CDS monitoring can identify malfunctions and/or significant improvement opportunities even after careful design and robust testing. CDS monitoring provides information when responding to user feedback. Ongoing, continuous, and automated monitoring can detect malfunctions in real time, before users report problems. Therefore, CDS monitoring should be part of any systematic program of implementing and maintaining CDS.

Patient-care questions that physicians are unable to answer.

OBJECTIVE: To describe the characteristics of unanswered clinical questions and propose interventions that could improve the chance of finding answers. DESIGN: In a previous study, investigators observed primary care physicians in their offices and recorded questions that arose during patient care. Questions that were pursued by the physician, but remained unanswered, were grouped into generic types. In the present study, investigators attempted to answer these questions and developed recommendations aimed at improving the success rate of finding answers. MEASUREMENTS: Frequency of unanswered question types and recommendations to increase the chance of finding answers. RESULTS: In an earlier study, 48 physicians asked 1062 questions during 192 half-day office observations. Physicians could not find answers to 237 (41%) of the 585 questions they pursued. The present study grouped the unanswered questions into 19 generic types. Three types accounted for 128 (54%) of the unanswered questions: (1) "Undiagnosed finding" questions asked about the management of abnormal clinical findings, such as symptoms, signs, and test results (What is the approach to finding X?); (2) "Conditional" questions contained qualifying conditions that were appended to otherwise simple questions (What is the management of X, given Y? where "given Y" is the qualifying condition that makes the question difficult.); and (3) "Compound" questions asked about the association between two highly specific elements (Can X cause Y?). The study identified strategies to improve clinical information retrieval, listed below. CONCLUSION: To improve the chance of finding answers, physicians should change their search strategies by rephrasing their questions and searching more clinically oriented resources. Authors of clinical information resources should anticipate questions that may arise in practice, and clinical information systems should provide clearer and more explicit answers.

Modeling Contextual Knowledge for Clinical Decision Support.

In theory, the logic of decision rules should be atomic. In practice, this is not always possible; initially simple logic statements tend to be overloaded with additional conditions restricting the scope of such rules. By doing so, the original logic soon becomes encumbered with contextual knowledge. Contextual knowledge is re-usable on its own and could be modeled separately from the logic of a rule without losing the intended functionality. We model constraints to explicitly define the context where knowledge of decision rules is actionable. We borrowed concepts from Semantic Web, Complex Adaptive Systems, and Contextual Reasoning. The proposed approach provides the means for identifying and modeling contextual knowledge in a simple, sound manner. The methodology presented herein facilitates rule authoring, fosters consistency in rules implementation and maintenance; facilitates developing authoritative knowledge repositories to promote quality, safety and efficacy of healthcare; and paves the road for future work in knowledge discovery.

KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs.

OBJECTIVE: Infobuttons are message-based content search and retrieval functions embedded within other applications that dynamically return information relevant to the clinical task at hand. The objective of this study was to determine whether infobuttons effectively answer providers' questions about medications or affect patient care decisions. DESIGN: The authors implemented and evaluated a medication infobutton application called KnowledgeLink. Health care providers at 18 outpatient clinics were randomized to one of two versions of KnowledgeLink, one that linked to information from Micromedex (Thomson Micromedex, Greenwood Village, Co) and the other to material from SkolarMD (Wolters Kluwer Health, Palo Alto, CA). MEASUREMENTS: Data were collected about the frequency of use and demographics of users, patients, and drugs that were queried. Users were periodically surveyed with short questionnaires and then with a more extensive survey at the end of one year. RESULTS: During the first year, KnowledgeLink was used 7,972 times by 359 users to look up information about 1,723 medications for 4,961 patients. Clinicians used KnowledgeLink twice a month on average, and during an average of 1.2% of patient encounters. KnowledgeLink was used by a wide variety of medical staff, not just physicians and nurse practitioners. The frequency of usage and the questions asked varied with user role (primary care physician, specialist physician, nurse practitioner). Although the median KnowledgeLink session was brief (21 seconds), KnowledgeLink answered users' queries 84% of the time, and altered patient care decisions 15% of the time. Users rated KnowledgeLink favorably on multiple scales, recommended extending KnowledgeLink to other content domains, and suggested enhancing the interface to allow refinement of the query and selection of the target resource. CONCLUSION: An infobutton can satisfy information needs about medications. Although used infrequently and for brief sessions, KnowledgeLink was positively received, answered most users' questions, and had a significant impact on medical decision making. The next steps would be to broaden the domains that KnowledgeLink covers to more specifically tailor results to the user type, to provide options when queries are not immediately answered, and to implement KnowledgeLink within other electronic clinical applications.

Patient safety concerns arising from test results that return after hospital discharge.

BACKGROUND: Failure to relay information about test results pending when patients are discharged from the hospital may pose an important patient-safety problem. Few data are available on the epidemiology of test results pending at discharge or on physician awareness of these results. OBJECTIVE: To determine the prevalence, characteristics, and physician awareness of potentially actionable laboratory and radiologic test results returning after hospital discharge. DESIGN: Cross-sectional study. SETTING: Two tertiary care academic hospitals. PATIENTS: 2644 consecutive patients discharged from hospitalist services from February to June 2004. MEASUREMENTS: The main outcomes were the prevalence and characteristics of potentially actionable test results returning after hospital discharge, awareness of these results by inpatient and primary care physicians, and satisfaction of inpatient physicians with current systems for follow-up on test results. The authors prospectively collected data on test results pending at the time of discharge and, as results returned after discharge, surveyed hospitalists, junior residents, and primary care physicians about those results that were potentially actionable according to a physician-reviewer. RESULTS: A total of 1095 patients (41%) had 2033 test results return after discharge. Of these results, 191 (9.4% [95% CI, 8.0% to 11.0%]) were potentially actionable. Surveys were sent regarding 155 results, and 105 responses were returned. Of the 105 results in the surveys with responses, physicians had been unaware of 65 (61.6% [CI, 51.3% to 70.9%]); of these 65, they agreed with physician-reviewers that 24 (37.1% [CI, 25.7% to 50.2%]) were actionable and 8 (12.6% [CI, 6.4% to 23.3%]) required urgent action. Inpatient physicians were dissatisfied with their systems for following up on test results returning after discharge. LIMITATIONS: The authors were unable to determine whether physicians' lack of awareness of test results returning after discharge was associated with adverse outcomes. CONCLUSIONS: Many patients are discharged from hospitals with test results still pending, and physicians are often unaware of potentially actionable test results returning after discharge. Further work is needed to design better follow-up systems for test results returning after hospital discharge.

Ambulatory hypercholesterolemia management in patients with atherosclerosis. Gender and race differences in processes and outcomes.

OBJECTIVE: To determine whether outpatient cholesterol management varies by gender or race among patients with atherosclerosis, and assess factors related to subsequent cholesterol control. DESIGN: Retrospective cohort study. SETTING: Primary care clinics affiliated with an academic medical center. PARTICIPANTS: Two hundred forty-three patients with coronary heart disease, cerebrovascular disease, or peripheral vascular disease and low-density lipoprotein cholesterol (LDL-C)>130 mg/dl. MEASUREMENTS AND MAIN RESULTS: The primary process of care assessed for 1,082 office visits was cholesterol management (medication intensification or LDL-C monitoring). Cholesterol management occurred at 31.2% of women's and 38.5% of men's visits (P=.01), and 37.3% of black and 31.7% of white patients' visits (P=.09). Independent predictors of cholesterol management included female gender (adjusted risk ratio [ARR], 0.77; 95% confidence interval [CI], 0.60 to 0.97), seeing a primary care clinician other than the patient's primary care physician (ARR, 0.23; 95% CI, 0.11 to 0.45), and having a new clinical problem addressed (ARR, 0.60; 95% CI, 0.48 to 0.74). After 1 year, LDL-C <130 mg/dl occurred less often for women than men (41% vs 61%; P=.003), black than white patients (39% vs 58%; P=.01), and patients with only Medicare insurance than with commercial insurance (37% vs 58%; P=.008). Adjustment for clinical characteristics and management attenuated the relationship between achieving an LDL-C <130 mg/dl and gender. CONCLUSIONS: In this high-risk population with uncontrolled cholesterol, cholesterol management was less intensive for women than men but similar for black and white patients. Less intense cholesterol management accounted for some of the disparity in cholesterol control between women and men but not between black and white patients.

A Practical Approach to Governance and Optimization of Structured Data Elements.

Definition and configuration of clinical content in an enterprise-wide electronic health record (EHR) implementation is highly complex. Sharing of data definitions across applications within an EHR implementation project may be constrained by practical limitations, including time, tools, and expertise. However, maintaining rigor in an approach to data governance is important for sustainability and consistency. With this understanding, we have defined a practical approach for governance of structured data elements to optimize data definitions given limited resources. This approach includes a 10 step process: 1) identification of clinical topics, 2) creation of draft reference models for clinical topics, 3) scoring of downstream data needs for clinical topics, 4) prioritization of clinical topics, 5) validation of reference models for clinical topics, and 6) calculation of gap analyses of EHR compared against reference model, 7) communication of validated reference models across project members, 8) requested revisions to EHR based on gap analysis, 9) evaluation of usage of reference models across project, and 10) Monitoring for new evidence requiring revisions to reference model.

Automating complex guidelines for chronic disease: lessons learned.

There is scant published experience with implementing complex, multistep computerized practice guidelines for the long-term management of chronic diseases. We have implemented a system for creating, maintaining, and navigating computer-based clinical algorithms integrated with our electronic medical record. This article describes our progress and reports on lessons learned that might guide future work in this field. We discuss issues and obstacles related to choosing and adapting a guideline for electronic implementation, representing and executing the guideline as a computerized algorithm, and integrating it into the clinical workflow of outpatient care. Although obstacles were encountered at each of these steps, the most difficult were related to workflow integration.

Automated coded ambulatory problem lists: evaluation of a vocabulary and a data entry tool.

BACKGROUND: Problem lists are fundamental to electronic medical records (EMRs). However, obtaining an appropriate problem list dictionary is difficult, and getting users to code their problems at the time of data entry can be challenging. OBJECTIVE: To develop a problem list dictionary and search algorithm for an EMR system and evaluate its use. METHODS: We developed a problem list dictionary and lookup tool and implemented it in several EMR systems. A sample of 10,000 problem entries was reviewed from each system to assess overall coding rates. We also performed a manual review of a subset of entries to determine the appropriateness of coded entries, and to assess the reasons other entries were left uncoded. RESULTS: The overall coding rate varied significantly between different EMR implementations (63-79%). Coded entries were virtually always appropriate (99%). The most frequent reasons for uncoded entries were due to user interface failures (44-45%), insufficient dictionary coverage (20-32%), and non-problem entries (10-12%). CONCLUSION: The problem list dictionary and search algorithm has achieved a good coding rate, but the rate is dependent on the specific user interface implementation. Problem coding is essential for providing clinical decision support, and improving usability should result in better coding rates.

Practical choices for infobutton customization: experience from four sites.

Context-aware links between electronic health records (EHRs) and online knowledge resources, commonly called "infobuttons" are being used increasingly as part of EHR "meaningful use" requirements. While an HL7 standard exists for specifying how the links should be constructed, there is no guidance on what links to construct. Collectively, the authors manage four infobutton systems that serve 16 institutions. The purpose of this paper is to publish our experience with linking various resources and specifying particular criteria that can be used by infobutton managers to select resources that are most relevant for a given situation. This experience can be used directly by those wishing to customize their own EHRs, for example by using the OpenInfobutton infobutton manager and its configuration tool, the Librarian Infobutton Tailoring Environment.

Structured representation for core elements of common clinical decision support interventions to facilitate knowledge sharing.

At present, there are no widely accepted, standard approaches for representing computer-based clinical decision support (CDS) intervention types and their structural components. This study aimed to identify key requirements for the representation of five widely utilized CDS intervention types: alerts and reminders, order sets, infobuttons, documentation templates/forms, and relevant data presentation. An XML schema was proposed for representing these interventions and their core structural elements (e.g., general metadata, applicable clinical scenarios, CDS inputs, CDS outputs, and CDS logic) in a shareable manner. The schema was validated by building CDS artifacts for 22 different interventions, targeted toward guidelines and clinical conditions called for in the 2011 Meaningful Use criteria. Custom style sheets were developed to render the XML files in human-readable form. The CDS knowledge artifacts were shared via a public web portal. Our experience also identifies gaps in existing standards and informs future development of standards for CDS knowledge representation and sharing.

Modeling decision support rule interactions in a clinical setting.

Traditionally, rule interactions are handled at implementation time through rule task properties that control the order in which rules are executed. By doing so, knowledge about the behavior and interactions of decision rules is not captured at modeling time. We argue that this is important knowledge that should be integrated in the modeling phase. In this project, we build upon current work on a conceptual schema to represent clinical knowledge for decision support in the form of if then rules. This schema currently captures provenance of the clinical content, context where such content is actionable (i.e. constraints) and the logic of the rule itself. For this project, we borrowed concepts from both the Semantic Web (i.e., Ontologies) and Complex Adaptive Systems (CAS), to explore a conceptual approach for modeling rule interactions in an enterprise-wide clinical setting. We expect that a more comprehensive modeling will facilitate knowledge authoring, editing and update; foster consistency in rules implementation and maintenance; and develop authoritative knowledge repositories to promote quality, safety and efficacy of healthcare.

Fatty emaciation: a case report of suspected fat overload syndrome in oral refeeding.

Refeeding syndrome has been observed in patients receiving nutrition after a prolonged period of malnourishment and is characterized by multiple metabolic derangements. Besides hypophosphatemia and hypoglycemia, lipemia has been described in association with parenteral nutrition administration to the malnourished. The authors describe one anorexic patient who developed lipemia during oral refeeding, followed by a precipitous drop in hematocrit suggestive of fat overload syndrome.

Adverse drug events caused by serious medication administration errors.

OBJECTIVE: To determine how often serious or life-threatening medication administration errors with the potential to cause harm (potential adverse drug events) result in actual harm (adverse drug events (ADEs)) in the hospital setting. DESIGN: Retrospective chart review of clinical events following observed medication administration errors. BACKGROUND: Medication errors are common at the medication administration stage for inpatients. While many errors can cause harm, it is unclear exactly how often. METHODS: In a previous study where 14 041 medication administrations were directly observed, 1271 medication administration errors were discovered, of which 133 had the potential to cause serious or life-threatening harm and were considered serious or life-threatening potential adverse drug events. As a follow-up, clinical reviewers conducted detailed chart review of serious or life-threatening potential ADEs to determine if they caused an ADE. Reviewers assessed severity of the ADE and attribution to the error. RESULTS: Ten (7.5% (95% CI 6.98 to 8.01)) actual ADEs resulted from the 133 serious and life-threatening potential ADEs, of which 6 resulted in significant, three in serious, and one life threatening injury. Therefore 4 (3% (95% CI 2.12 to 3.6)) of serious or life threatening potential ADEs led to serious or life threatening ADEs. Half of the ADEs were caused by dosage or monitoring errors for anti-hypertensives. CONCLUSIONS: Unintercepted potential ADEs at the medication administration stage can cause serious patient harm. At hospitals where 6 million doses are administered per year, about 4000 preventable ADEs would be attributable to medication administration errors annually.