Developing a sampling method and preliminary taxonomy for classifying COVID-19 public health guidance for healthcare organizations and the general public.

BACKGROUND: Development and dissemination of public health (PH) guidance to healthcare organizations and the general public (e.g., businesses, schools, individuals) during emergencies like the COVID-19 pandemic is vital for policy, clinical, and public decision-making. Yet, the rapidly evolving nature of these events poses significant challenges for guidance development and dissemination strategies predicated on well-understood concepts and clearly defined access and distribution pathways. Taxonomies are an important but underutilized tool for guidance authoring, dissemination and updating in such dynamic scenarios. OBJECTIVE: To design a rapid, semi-automated method for sampling and developing a PH guidance taxonomy using widely available Web crawling tools and streamlined manual content analysis. METHODS: Iterative samples of guidance documents were taken from four state PH agency websites, the US Center for Disease Control and Prevention, and the World Health Organization. Documents were used to derive and refine a preliminary taxonomy of COVID-19 PH guidance via content analysis. RESULTS: Eight iterations of guidance document sampling and taxonomy revisions were performed, with a final corpus of 226 documents. The preliminary taxonomy contains 110 branches distributed between three major domains: stakeholders (24 branches), settings (25 branches) and topics (61 branches). Thematic saturation measures indicated rapid saturation (≤5% change) for the domains of "stakeholders" and "settings", and "topic"-related branches for clinical decision-making. Branches related to business reopening and economic consequences remained dynamic throughout sampling iterations. CONCLUSION: The PH guidance taxonomy can support public health agencies by aligning guidance development with curation and indexing strategies; supporting targeted dissemination; increasing the speed of updates; and enhancing public-facing guidance repositories and information retrieval tools. Taxonomies are essential to support knowledge management activities during rapidly evolving scenarios such as disease outbreaks and natural disasters.

Modeling Contextual Knowledge for Clinical Decision Support.

In theory, the logic of decision rules should be atomic. In practice, this is not always possible; initially simple logic statements tend to be overloaded with additional conditions restricting the scope of such rules. By doing so, the original logic soon becomes encumbered with contextual knowledge. Contextual knowledge is re-usable on its own and could be modeled separately from the logic of a rule without losing the intended functionality. We model constraints to explicitly define the context where knowledge of decision rules is actionable. We borrowed concepts from Semantic Web, Complex Adaptive Systems, and Contextual Reasoning. The proposed approach provides the means for identifying and modeling contextual knowledge in a simple, sound manner. The methodology presented herein facilitates rule authoring, fosters consistency in rules implementation and maintenance; facilitates developing authoritative knowledge repositories to promote quality, safety and efficacy of healthcare; and paves the road for future work in knowledge discovery.

Modeling decision support rule interactions in a clinical setting.

Traditionally, rule interactions are handled at implementation time through rule task properties that control the order in which rules are executed. By doing so, knowledge about the behavior and interactions of decision rules is not captured at modeling time. We argue that this is important knowledge that should be integrated in the modeling phase. In this project, we build upon current work on a conceptual schema to represent clinical knowledge for decision support in the form of if then rules. This schema currently captures provenance of the clinical content, context where such content is actionable (i.e. constraints) and the logic of the rule itself. For this project, we borrowed concepts from both the Semantic Web (i.e., Ontologies) and Complex Adaptive Systems (CAS), to explore a conceptual approach for modeling rule interactions in an enterprise-wide clinical setting. We expect that a more comprehensive modeling will facilitate knowledge authoring, editing and update; foster consistency in rules implementation and maintenance; and develop authoritative knowledge repositories to promote quality, safety and efficacy of healthcare.

Practical choices for infobutton customization: experience from four sites.

Context-aware links between electronic health records (EHRs) and online knowledge resources, commonly called "infobuttons" are being used increasingly as part of EHR "meaningful use" requirements. While an HL7 standard exists for specifying how the links should be constructed, there is no guidance on what links to construct. Collectively, the authors manage four infobutton systems that serve 16 institutions. The purpose of this paper is to publish our experience with linking various resources and specifying particular criteria that can be used by infobutton managers to select resources that are most relevant for a given situation. This experience can be used directly by those wishing to customize their own EHRs, for example by using the OpenInfobutton infobutton manager and its configuration tool, the Librarian Infobutton Tailoring Environment.

A study of diverse clinical decision support rule authoring environments and requirements for integration.

BACKGROUND: Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules. METHODS: The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools. RESULTS: While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users. CONCLUSIONS: A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Fatty emaciation: a case report of suspected fat overload syndrome in oral refeeding.

Refeeding syndrome has been observed in patients receiving nutrition after a prolonged period of malnourishment and is characterized by multiple metabolic derangements. Besides hypophosphatemia and hypoglycemia, lipemia has been described in association with parenteral nutrition administration to the malnourished. The authors describe one anorexic patient who developed lipemia during oral refeeding, followed by a precipitous drop in hematocrit suggestive of fat overload syndrome.

Implementations of the HL7 Context-Aware Knowledge Retrieval ("Infobutton") Standard: challenges, strengths, limitations, and uptake.

UNLABELLED: To support clinical decision-making, computerized information retrieval tools known as "infobuttons" deliver contextually-relevant knowledge resources into clinical information systems. The Health Level Seven International (HL7) Context-Aware Knowledge Retrieval (Infobutton) Standard specifies a standard mechanism to enable infobuttons on a large scale. OBJECTIVE: To examine the experience of organizations in the course of implementing the HL7 Infobutton Standard. METHOD: Cross-sectional online survey and in-depth phone interviews. RESULTS: A total of 17 organizations participated in the study. Analysis of the in-depth interviews revealed 20 recurrent themes. Implementers underscored the benefits, simplicity, and flexibility of the HL7 Infobutton Standard. Yet, participants voiced the need for easier access to standard specifications and improved guidance to beginners. Implementers predicted that the Infobutton Standard will be widely or at least fairly well adopted in the next 5 years, but uptake will depend largely on adoption among electronic health record (EHR) vendors. To accelerate EHR adoption of the Infobutton Standard, implementers recommended HL7-compliant infobutton capabilities to be included in the United States Meaningful Use Certification Criteria for EHR systems. LIMITATIONS: Opinions and predictions should be interpreted with caution, since all the participant organizations have successfully implemented the standard and over half of the organizations were actively engaged in the development of the standard. CONCLUSION: Overall, implementers reported a very positive experience with the HL7 Infobutton Standard. Despite indications of increasing uptake, measures should be taken to stimulate adoption of the Infobutton Standard among EHR vendors. Widespread adoption of the Infobutton Standard has the potential to bring contextually relevant clinical decision support content into the healthcare provider workflow.

Supratherapeutic dosing of acetaminophen among hospitalized patients.

BACKGROUND: We investigated acetaminophen use and identify factors contributing to supratherapeutic dosing of acetaminophen in hospitalized patients. METHODS: We retrospectively reviewed the electronic health records of adult patients who were admitted to 2 academic tertiary care hospitals (hospital A amd hospital B) from June 1, 2010, to August 31, 2010, and who received acetaminophen during their hospitalization. Patients' acetaminophen administration records (including drug name, dose, administration time, hospital units, etc), demographic data, diagnoses, and results from liver function tests were obtained. The main outcome measures included acetaminophen exposure rate and supratherapeutic dosing rate among hospitalized patients, hazard ratios (HRs) and 95% confidence intervals (CIs) for risk factors for supratherapeutic dosing, and changes in liver function test before and after supratherapeutic dosing. RESULTS: A total of 14 411 patients (60.7%) were exposed to acetaminophen, of whom 955 (6.6%) exceeded the 4 g per day maximum recommended dose. In addition, 22.3% of patients who were 65 years or older and 17.6% of patients with chronic liver diseases exceeded the recommended limit of 3 g per day. Patients receiving excessive doses of acetaminophen tended to have significant alkaline phosphatase elevations, although causal relationship cannot be concluded. A significantly higher risk of supratherapeutic dosing was observed in hospital A (HR, 1.6 [95% CI, 1.4-1.8]), white patients (HR, 1.5 [95% CI, 1.3-1.7]), patients diagnosed as having osteoarthritis (HR, 1.4 [95% CI, 1.3-1.6]), and those who received scheduled administrations (HR, 16.6 [95% CI, 13.5-20.6]), multiple product formulations (HR, 2.4 [95% CI 2.0-2.9]), or the 500-mg strength formulation (HR, 1.9 [95% CI, 1.5-2.3]). A lower risk was found for pro re nata (as needed) administrations (HR, 0.7 [95% CI, 0.6-0.9]) and in nonsurgical and non­intensive care units (HR, 0.6 [95% CI, 0.5-0.7]). CONCLUSIONS: Supratherapeutic dosing of acetaminophen was significantly associated with multiple factors. Interventions to reduce the incidence of some risk factors may prevent supratherapeutic acetaminophen dosing in hospitalized patients.

Governance for clinical decision support: case studies and recommended practices from leading institutions.

OBJECTIVE: Clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety; however, effective implementation of CDS requires effective clinical and technical governance structures. The authors sought to determine the range and variety of these governance structures and identify a set of recommended practices through observational study. DESIGN: Three site visits were conducted at institutions across the USA to learn about CDS capabilities and processes from clinical, technical, and organizational perspectives. Based on the results of these visits, written questionnaires were sent to the three institutions visited and two additional sites. Together, these five organizations encompass a variety of academic and community hospitals as well as small and large ambulatory practices. These organizations use both commercially available and internally developed clinical information systems. MEASUREMENTS: Characteristics of clinical information systems and CDS systems used at each site as well as governance structures and content management approaches were identified through extensive field interviews and follow-up surveys. RESULTS: Six recommended practices were identified in the area of governance, and four were identified in the area of content management. Key similarities and differences between the organizations studied were also highlighted. CONCLUSION: Each of the five sites studied contributed to the recommended practices presented in this paper for CDS governance. Since these strategies appear to be useful at a diverse range of institutions, they should be considered by any future implementers of decision support.

A large-scale knowledge management method based on the analysis of the use of online knowledge resources.

UNLABELLED: Online health knowledge resources can be integrated into electronic health record systems using decision support tools known as "infobuttons." In this study we describe a knowledge management method based on the analysis of knowledge resource use via infobuttons in multiple institutions. METHODS: We conducted a two-phase analysis of laboratory test infobutton sessions at three healthcare institutions accessing two knowledge resources. The primary study measure was session coverage, i.e. the rate of infobutton sessions in which resources retrieved relevant content. RESULTS: In Phase One, resources covered 78.5% of the study sessions. In addition, a subset of 38 noncovered tests that most frequently raised questions was identified. In Phase Two, content development guided by the outcomes of Phase One resulted in a 4% average coverage increase. CONCLUSION: The described method is a valuable approach to large-scale knowledge management in rapidly changing domains.

Creating shareable decision support services: an interdisciplinary challenge.

Creating shareable decision support services is a complex task requiring effort from multiple interdisciplinary role players with a wide variety of experience and expertise. The CDS Consortium research project has developed such a service, defining a multi-layer representation of knowledge and building upon an architectural service design created at Partners Health Care, and is demonstrating its use in both a local and an external institutional setting. The process was iterative, and we encountered unexpected requirements based on decisions made at various points. We report in this paper on challenges we faced while pursuing this research: knowledge representation and modeling, data interchange and standards adoption, the process of getting agreement on content, logistics of integrating into a system that already has multiple CDS interventions, legal issues around privacy and access, inter-team communication and organization.

Tiering drug-drug interaction alerts by severity increases compliance rates.

OBJECTIVE: Few data exist measuring the effect of differentiating drug-drug interaction (DDI) alerts in computerized provider order entry systems (CPOE) by level of severity ("tiering"). We sought to determine if rates of provider compliance with DDI alerts in the inpatient setting differed when a tiered presentation was implemented. DESIGN: We performed a retrospective analysis of alert log data on hospitalized patients at two academic medical centers during the period from 2/1/2004 through 2/1/2005. Both inpatient CPOE systems used the same DDI checking service, but one displayed alerts differentially by severity level (tiered presentation, including hard stops for the most severe alerts) while the other did not. Participants were adult inpatients who generated a DDI alert, and providers who wrote the orders. Alerts were presented during the order entry process, providing the clinician with the opportunity to change the patient's medication orders to avoid the interaction. MEASUREMENTS: Rate of compliance to alerts at a tiered site compared to a non-tiered site. RESULTS: We reviewed 71,350 alerts, of which 39,474 occurred at the non-tiered site and 31,876 at the tiered site. Compliance with DDI alerts was significantly higher at the site with tiered DDI alerts compared to the non-tiered site (29% vs. 10%, p < 0.001). At the tiered site, 100% of the most severe alerts were accepted, vs. only 34% at the non-tiered site; moderately severe alerts were also more likely to be accepted at the tiered site (29% vs. 10%). CONCLUSION: Tiered alerting by severity was associated with higher compliance rates of DDI alerts in the inpatient setting, and lack of tiering was associated with a high override rate of more severe alerts.

Knowledge management and content integration: a collaborative approach.

Knowledge Management (KM) is the development and maintenance of decision support knowledge within clinical healthcare systems. At Partners Healthcare System (PHS), Knowledge Management is facilitated by a dedicated team consisting of Subject Matter Experts (SME), Knowledge Engineers (KE), and software developers. This group relies upon sound knowledge management principles to support the knowledge lifecycle.

Cost-benefit analysis of a hospital pharmacy bar code solution.

BACKGROUND: Bar coding can reduce hospital pharmacy dispensing errors, but it is unclear if the benefits of this technology justify its costs. The purpose of this study was to assess the costs and benefits and determine the return on investment at the institutional level for implementing a pharmacy bar code system. METHODS: We performed a cost-benefit analysis of a bar code-assisted medication-dispensing system within a large, academic, nonprofit tertiary care hospital pharmacy. We took the implementing hospital's perspective for a 5-year horizon. The primary outcome was the net financial cost and benefit after 5 years. The secondary outcome was the time until total benefits equaled total costs. Single-variable, 2-variable, and multiple-variable Monte Carlo sensitivity analyses were performed to test the stability of the outcomes. RESULTS: In inflation- and time value-adjusted 2005 dollars, total costs during 5 years were $2.24 million ($1.31 million in 1-time costs during the initial 3.5 years and $342 000 per year in recurring costs starting in year 3). The primary benefit was a decrease in adverse drug events from dispensing errors (517 events annually), resulting in an annual savings of $2.20 million. The net benefit after 5 years was $3.49 million. The break-even point for the hospital's investment occurred within 1 year after becoming fully operational. A net benefit was achieved within 10 years under almost all sensitivity scenarios. In the Monte Carlo simulation, the net benefit during 5 years was $3.2 million (95% confidence interval, -$1.2 million to $12.1 million), and the break-even point for return on investment occurred after 51 months (95% confidence interval, 30 to 180 months). CONCLUSION: Implementation of a bar code-assisted medication-dispensing system in hospital pharmacies can result in a positive financial return on investment for the health care organization.

Patient-care questions that physicians are unable to answer.

OBJECTIVE: To describe the characteristics of unanswered clinical questions and propose interventions that could improve the chance of finding answers. DESIGN: In a previous study, investigators observed primary care physicians in their offices and recorded questions that arose during patient care. Questions that were pursued by the physician, but remained unanswered, were grouped into generic types. In the present study, investigators attempted to answer these questions and developed recommendations aimed at improving the success rate of finding answers. MEASUREMENTS: Frequency of unanswered question types and recommendations to increase the chance of finding answers. RESULTS: In an earlier study, 48 physicians asked 1062 questions during 192 half-day office observations. Physicians could not find answers to 237 (41%) of the 585 questions they pursued. The present study grouped the unanswered questions into 19 generic types. Three types accounted for 128 (54%) of the unanswered questions: (1) "Undiagnosed finding" questions asked about the management of abnormal clinical findings, such as symptoms, signs, and test results (What is the approach to finding X?); (2) "Conditional" questions contained qualifying conditions that were appended to otherwise simple questions (What is the management of X, given Y? where "given Y" is the qualifying condition that makes the question difficult.); and (3) "Compound" questions asked about the association between two highly specific elements (Can X cause Y?). The study identified strategies to improve clinical information retrieval, listed below. CONCLUSION: To improve the chance of finding answers, physicians should change their search strategies by rephrasing their questions and searching more clinically oriented resources. Authors of clinical information resources should anticipate questions that may arise in practice, and clinical information systems should provide clearer and more explicit answers.

KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs.

OBJECTIVE: Infobuttons are message-based content search and retrieval functions embedded within other applications that dynamically return information relevant to the clinical task at hand. The objective of this study was to determine whether infobuttons effectively answer providers' questions about medications or affect patient care decisions. DESIGN: The authors implemented and evaluated a medication infobutton application called KnowledgeLink. Health care providers at 18 outpatient clinics were randomized to one of two versions of KnowledgeLink, one that linked to information from Micromedex (Thomson Micromedex, Greenwood Village, Co) and the other to material from SkolarMD (Wolters Kluwer Health, Palo Alto, CA). MEASUREMENTS: Data were collected about the frequency of use and demographics of users, patients, and drugs that were queried. Users were periodically surveyed with short questionnaires and then with a more extensive survey at the end of one year. RESULTS: During the first year, KnowledgeLink was used 7,972 times by 359 users to look up information about 1,723 medications for 4,961 patients. Clinicians used KnowledgeLink twice a month on average, and during an average of 1.2% of patient encounters. KnowledgeLink was used by a wide variety of medical staff, not just physicians and nurse practitioners. The frequency of usage and the questions asked varied with user role (primary care physician, specialist physician, nurse practitioner). Although the median KnowledgeLink session was brief (21 seconds), KnowledgeLink answered users' queries 84% of the time, and altered patient care decisions 15% of the time. Users rated KnowledgeLink favorably on multiple scales, recommended extending KnowledgeLink to other content domains, and suggested enhancing the interface to allow refinement of the query and selection of the target resource. CONCLUSION: An infobutton can satisfy information needs about medications. Although used infrequently and for brief sessions, KnowledgeLink was positively received, answered most users' questions, and had a significant impact on medical decision making. The next steps would be to broaden the domains that KnowledgeLink covers to more specifically tailor results to the user type, to provide options when queries are not immediately answered, and to implement KnowledgeLink within other electronic clinical applications.

Patient safety concerns arising from test results that return after hospital discharge.

BACKGROUND: Failure to relay information about test results pending when patients are discharged from the hospital may pose an important patient-safety problem. Few data are available on the epidemiology of test results pending at discharge or on physician awareness of these results. OBJECTIVE: To determine the prevalence, characteristics, and physician awareness of potentially actionable laboratory and radiologic test results returning after hospital discharge. DESIGN: Cross-sectional study. SETTING: Two tertiary care academic hospitals. PATIENTS: 2644 consecutive patients discharged from hospitalist services from February to June 2004. MEASUREMENTS: The main outcomes were the prevalence and characteristics of potentially actionable test results returning after hospital discharge, awareness of these results by inpatient and primary care physicians, and satisfaction of inpatient physicians with current systems for follow-up on test results. The authors prospectively collected data on test results pending at the time of discharge and, as results returned after discharge, surveyed hospitalists, junior residents, and primary care physicians about those results that were potentially actionable according to a physician-reviewer. RESULTS: A total of 1095 patients (41%) had 2033 test results return after discharge. Of these results, 191 (9.4% [95% CI, 8.0% to 11.0%]) were potentially actionable. Surveys were sent regarding 155 results, and 105 responses were returned. Of the 105 results in the surveys with responses, physicians had been unaware of 65 (61.6% [CI, 51.3% to 70.9%]); of these 65, they agreed with physician-reviewers that 24 (37.1% [CI, 25.7% to 50.2%]) were actionable and 8 (12.6% [CI, 6.4% to 23.3%]) required urgent action. Inpatient physicians were dissatisfied with their systems for following up on test results returning after discharge. LIMITATIONS: The authors were unable to determine whether physicians' lack of awareness of test results returning after discharge was associated with adverse outcomes. CONCLUSIONS: Many patients are discharged from hospitals with test results still pending, and physicians are often unaware of potentially actionable test results returning after discharge. Further work is needed to design better follow-up systems for test results returning after hospital discharge.