Developing a sampling method and preliminary taxonomy for classifying COVID-19 public health guidance for healthcare organizations and the general public.

BACKGROUND: Development and dissemination of public health (PH) guidance to healthcare organizations and the general public (e.g., businesses, schools, individuals) during emergencies like the COVID-19 pandemic is vital for policy, clinical, and public decision-making. Yet, the rapidly evolving nature of these events poses significant challenges for guidance development and dissemination strategies predicated on well-understood concepts and clearly defined access and distribution pathways. Taxonomies are an important but underutilized tool for guidance authoring, dissemination and updating in such dynamic scenarios. OBJECTIVE: To design a rapid, semi-automated method for sampling and developing a PH guidance taxonomy using widely available Web crawling tools and streamlined manual content analysis. METHODS: Iterative samples of guidance documents were taken from four state PH agency websites, the US Center for Disease Control and Prevention, and the World Health Organization. Documents were used to derive and refine a preliminary taxonomy of COVID-19 PH guidance via content analysis. RESULTS: Eight iterations of guidance document sampling and taxonomy revisions were performed, with a final corpus of 226 documents. The preliminary taxonomy contains 110 branches distributed between three major domains: stakeholders (24 branches), settings (25 branches) and topics (61 branches). Thematic saturation measures indicated rapid saturation (≤5% change) for the domains of "stakeholders" and "settings", and "topic"-related branches for clinical decision-making. Branches related to business reopening and economic consequences remained dynamic throughout sampling iterations. CONCLUSION: The PH guidance taxonomy can support public health agencies by aligning guidance development with curation and indexing strategies; supporting targeted dissemination; increasing the speed of updates; and enhancing public-facing guidance repositories and information retrieval tools. Taxonomies are essential to support knowledge management activities during rapidly evolving scenarios such as disease outbreaks and natural disasters.

Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations.

Real time data provided by frontline clinicians could be used to direct immediate resources during a public health emergency and inform increased preparedness for future events. The United States Critical Illness and Injury Trials Group Program for Emergency Preparedness (USCIIT-PREP), a group of expert critical care and emergency medicine physicians at various academic medical centers across the US, aims to enhance the national capability of rapid electronic data collection, along with analysis and dissemination of findings. To achieve these aims, USCIIT-PREP created a process for real-time data capture that relies on a curated and engaged network of clinical providers from various geographical regions to respond to short online "Pulse" queries about healthcare system stress. During a period of three years, five queries were created and distributed. The first two queries were used to develop and validate the data collection infrastructure. Results are reported for the last three queries between June 2015 and March 2016. Response rates consistently ranged from 39% to 42%. Our team demonstrated that our system and processes were ready for creation and rapid dissemination of episodic queries for rapid data collection, transmittal, and analysis through a curated national network of clinician responders during a public health emergency. USCIIT-PREP aims to further increase the response rate through additional engagement efforts within the network, to continue to grow the clinician responder database, and to optimize additional query content.

Hospital Readmission and Social Risk Factors Identified from Physician Notes.

OBJECTIVE: To evaluate the prevalence of seven social factors using physician notes as compared to claims and structured electronic health records (EHRs) data and the resulting association with 30-day readmissions. STUDY SETTING: A multihospital academic health system in southeastern Massachusetts. STUDY DESIGN: An observational study of 49,319 patients with cardiovascular disease admitted from January 1, 2011, to December 31, 2013, using multivariable logistic regression to adjust for patient characteristics. DATA COLLECTION/EXTRACTION METHODS: All-payer claims, EHR data, and physician notes extracted from a centralized clinical registry. PRINCIPAL FINDINGS: All seven social characteristics were identified at the highest rates in physician notes. For example, we identified 14,872 patient admissions with poor social support in physician notes, increasing the prevalence from 0.4 percent using ICD-9 codes and structured EHR data to 16.0 percent. Compared to an 18.6 percent baseline readmission rate, risk-adjusted analysis showed higher readmission risk for patients with housing instability (readmission rate 24.5 percent; p < .001), depression (20.6 percent; p < .001), drug abuse (20.2 percent; p = .01), and poor social support (20.0 percent; p = .01). CONCLUSIONS: The seven social risk factors studied are substantially more prevalent than represented in administrative data. Automated methods for analyzing physician notes may enable better identification of patients with social needs.

Next generation terminology infrastructure to support interprofessional care planning.

OBJECTIVE: Develop a prototype of an interprofessional terminology and information model infrastructure that can enable care planning applications to facilitate patient-centered care, learn care plan linkages and associations, provide decision support, and enable automated, prospective analytics. DESIGN: The study steps included a 3 step approach: (1) Process model and clinical scenario development, and (2) Requirements analysis, and (3) Development and validation of information and terminology models. RESULTS: Components of the terminology model include: Health Concerns, Goals, Decisions, Interventions, Assessments, and Evaluations. A terminology infrastructure should: (A) Include discrete care plan concepts; (B) Include sets of profession-specific concerns, decisions, and interventions; (C) Communicate rationales, anticipatory guidance, and guidelines that inform decisions among the care team; (D) Define semantic linkages across clinical events and professions; (E) Define sets of shared patient goals and sub-goals, including patient stated goals; (F) Capture evaluation toward achievement of goals. These requirements were mapped to AHRQ Care Coordination Measures Framework. LIMITATIONS: This study used a constrained set of clinician-validated clinical scenarios. Terminology models for goals and decisions are unavailable in SNOMED CT, limiting the ability to evaluate these aspects of the proposed infrastructure. CONCLUSIONS: Defining and linking subsets of care planning concepts appears to be feasible, but also essential to model interprofessional care planning for common co-occurring conditions and chronic diseases. We recommend the creation of goal dynamics and decision concepts in SNOMED CT to further enable the necessary models. Systems with flexible terminology management infrastructure may enable intelligent decision support to identify conflicting and aligned concerns, goals, decisions, and interventions in shared care plans, ultimately decreasing documentation effort and cognitive burden for clinicians and patients.

Gap Analysis and Refinement Recommendations of Skin Alteration and Pressure Ulcer Enterprise Reference Models against Nursing Flowsheet Data Elements.

Reference models are an essential instrument to provide structure and guidance in the creation and use of data elements within an organizations' electronic health record (EHR). Standardization of data elements is imperative to ensure clinical data is consistently and reliably captured for use in clinical documentation, care communication, and a variety of downstream data uses. Ongoing assessment and refinement of reference models and data elements are necessary to ascertain clinical data capture is applicable and inclusive across a variety of caregivers and domains. We performed a gap analysis on current state nursing data elements against two validated interprofessional reference models: skin alteration and pressure ulcer assessments. We present our findings along with recommendations for reference model refinements. We also highlight additional findings of inconsistencies and redundancies within data elements used for nursing documentation and highlight recommendations for improvement.

Evolution of an Implementation-Ready Interprofessional Pain Assessment Reference Model.

Standards to increase consistency of comprehensive pain assessments are important for safety, quality, and analytics activities, including meeting Joint Commission requirements and learning the best management strategies and interventions for the current prescription Opioid epidemic. In this study we describe the development and validation of a Pain Assessment Reference Model ready for implementation on EHR forms and flowsheets. Our process resulted in 5 successive revisions of the reference model, which more than doubled the number of data elements to 47. The organization of the model evolved during validation sessions with panels totaling 48 subject matter experts (SMEs) to include 9 sets of data elements, with one set recommended as a minimal data set. The reference model also evolved when implemented into EHR forms and flowsheets, indicating specifications such as cascading logic that are important to inform secondary use of data.

Modeling Contextual Knowledge for Clinical Decision Support.

In theory, the logic of decision rules should be atomic. In practice, this is not always possible; initially simple logic statements tend to be overloaded with additional conditions restricting the scope of such rules. By doing so, the original logic soon becomes encumbered with contextual knowledge. Contextual knowledge is re-usable on its own and could be modeled separately from the logic of a rule without losing the intended functionality. We model constraints to explicitly define the context where knowledge of decision rules is actionable. We borrowed concepts from Semantic Web, Complex Adaptive Systems, and Contextual Reasoning. The proposed approach provides the means for identifying and modeling contextual knowledge in a simple, sound manner. The methodology presented herein facilitates rule authoring, fosters consistency in rules implementation and maintenance; facilitates developing authoritative knowledge repositories to promote quality, safety and efficacy of healthcare; and paves the road for future work in knowledge discovery.

Prioritization and Refinement of Clinical Data Elements within EHR Systems.

Standardization of clinical data element (CDE) definitions is foundational to track, interpret, and analyze patient states, populations, and costs across providers, settings and time - critical activities to achieve the Triple Aim: improving the experience of care, improving the health of populations, and reducing per capita healthcare costs. We defined and implemented two analytical methods to prioritize and refine CDE definitions within electronic health records (EHRs), taking into account resource restrictions to carry out the analysis and configuration changes: 1) analysis of downstream data needs to identify high priority clinical topics, and 2) gap analysis of EHR CDEs when compared to reference models for the same clinical topics. We present use cases for six clinical topics. Pain Assessment and Skin Alteration Assessment were topics with the highest regulatory and non-regulatory downstream data needs and with significant gaps across documention artifacts in our system, confirming that these topics should be refined first.

A Decade of Experience in Creating and Maintaining Data Elements for Structured Clinical Documentation in EHRs.

Structured clinical documentation is an important component of electronic health records (EHRs) and plays an important role in clinical care, administrative functions, and research activities. Clinical data elements serve as basic building blocks for composing the templates used for generating clinical documents (such as notes and forms). We present our experience in creating and maintaining data elements for three different EHRs (one home-grown and two commercial systems) across different clinical settings, using flowsheet data elements as examples in our case studies. We identified basic but important challenges (including naming convention, links to standard terminologies, and versioning and change management) and possible solutions to address them. We also discussed more complicated challenges regarding governance, documentation vs. structured data capture, pre-coordination vs. post-coordination, reference information models, as well as monitoring, communication and training.

Identifying Patients with Depression Using Free-text Clinical Documents.

About 1 in 10 adults are reported to exhibit clinical depression and the associated personal, societal, and economic costs are significant. In this study, we applied the MTERMS NLP system and machine learning classification algorithms to identify patients with depression using discharge summaries. Domain experts reviewed both the training and test cases, and classified these cases as depression with a high, intermediate, and low confidence. For depression cases with high confidence, all of the algorithms we tested performed similarly, with MTERMS' knowledge-based decision tree slightly better than the machine learning classifiers, achieving an F-measure of 89.6%. MTERMS also achieved the highest F-measure (70.6%) on intermediate confidence cases. The RIPPER rule learner was the best performing machine learning method, with an F-measure of 70.0%, and a higher precision but lower recall than MTERMS. The proposed NLP-based approach was able to identify a significant portion of the depression cases (about 20%) that were not on the coded diagnosis list.

A continuum of sociotechnical requirements for patient-centered problem lists.

Specific requirements for patient-centered health information technology remain ill-defined. To create operational definitions of patient-centered problem lists, we propose a continuum of sociotechnical requirements with five stages: 1) Intradisciplinary Care Planning: Viewing and searching for problems by discipline; 2) Multi-disciplinary Care Planning: Categorizing problem states to meet discipline-specific needs; 3) Interdisciplinary Care Planning: Sharing and linking problems between disciplines; 4) Integrated and Coordinated Care Planning: Associating problems with assessments, tasks, interventions and outcomes across disciplines for coordination, knowledge development, and reporting; and 5) Patient-Centered Care Planning: Engaging patients in identification of problems and maintenance of their problem list.

Modeling decision support rule interactions in a clinical setting.

Traditionally, rule interactions are handled at implementation time through rule task properties that control the order in which rules are executed. By doing so, knowledge about the behavior and interactions of decision rules is not captured at modeling time. We argue that this is important knowledge that should be integrated in the modeling phase. In this project, we build upon current work on a conceptual schema to represent clinical knowledge for decision support in the form of if then rules. This schema currently captures provenance of the clinical content, context where such content is actionable (i.e. constraints) and the logic of the rule itself. For this project, we borrowed concepts from both the Semantic Web (i.e., Ontologies) and Complex Adaptive Systems (CAS), to explore a conceptual approach for modeling rule interactions in an enterprise-wide clinical setting. We expect that a more comprehensive modeling will facilitate knowledge authoring, editing and update; foster consistency in rules implementation and maintenance; and develop authoritative knowledge repositories to promote quality, safety and efficacy of healthcare.

Evaluating standard terminologies for encoding allergy information.

OBJECTIVE: Allergy documentation and exchange are vital to ensuring patient safety. This study aims to analyze and compare various existing standard terminologies for representing allergy information. METHODS: Five terminologies were identified, including the Systemized Nomenclature of Medical Clinical Terms (SNOMED CT), National Drug File-Reference Terminology (NDF-RT), Medication Dictionary for Regulatory Activities (MedDRA), Unique Ingredient Identifier (UNII), and RxNorm. A qualitative analysis was conducted to compare desirable characteristics of each terminology, including content coverage, concept orientation, formal definitions, multiple granularities, vocabulary structure, subset capability, and maintainability. A quantitative analysis was also performed to compare the content coverage of each terminology for (1) common food, drug, and environmental allergens and (2) descriptive concepts for common drug allergies, adverse reactions (AR), and no known allergies. RESULTS: Our qualitative results show that SNOMED CT fulfilled the greatest number of desirable characteristics, followed by NDF-RT, RxNorm, UNII, and MedDRA. Our quantitative results demonstrate that RxNorm had the highest concept coverage for representing drug allergens, followed by UNII, SNOMED CT, NDF-RT, and MedDRA. For food and environmental allergens, UNII demonstrated the highest concept coverage, followed by SNOMED CT. For representing descriptive allergy concepts and adverse reactions, SNOMED CT and NDF-RT showed the highest coverage. Only SNOMED CT was capable of representing unique concepts for encoding no known allergies. CONCLUSIONS: The proper terminology for encoding a patient's allergy is complex, as multiple elements need to be captured to form a fully structured clinical finding. Our results suggest that while gaps still exist, a combination of SNOMED CT and RxNorm can satisfy most criteria for encoding common allergies and provide sufficient content coverage.

A study of diverse clinical decision support rule authoring environments and requirements for integration.

BACKGROUND: Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules. METHODS: The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools. RESULTS: While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users. CONCLUSIONS: A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Mapping Partners Master Drug Dictionary to RxNorm using an NLP-based approach.

OBJECTIVE: To develop an automated method based on natural language processing (NLP) to facilitate the creation and maintenance of a mapping between RxNorm and a local medication terminology for interoperability and meaningful use purposes. METHODS: We mapped 5961 terms from Partners Master Drug Dictionary (MDD) and 99 of the top prescribed medications to RxNorm. The mapping was conducted at both term and concept levels using an NLP tool, called MTERMS, followed by a manual review conducted by domain experts who created a gold standard mapping. The gold standard was used to assess the overall mapping between MDD and RxNorm and evaluate the performance of MTERMS. RESULTS: Overall, 74.7% of MDD terms and 82.8% of the top 99 terms had an exact semantic match to RxNorm. Compared to the gold standard, MTERMS achieved a precision of 99.8% and a recall of 73.9% when mapping all MDD terms, and a precision of 100% and a recall of 72.6% when mapping the top prescribed medications. CONCLUSION: The challenges and gaps in mapping MDD to RxNorm are mainly due to unique user or application requirements for representing drug concepts and the different modeling approaches inherent in the two terminologies. An automated approach based on NLP followed by human expert review is an efficient and feasible way for conducting dynamic mapping.

Supratherapeutic dosing of acetaminophen among hospitalized patients.

BACKGROUND: We investigated acetaminophen use and identify factors contributing to supratherapeutic dosing of acetaminophen in hospitalized patients. METHODS: We retrospectively reviewed the electronic health records of adult patients who were admitted to 2 academic tertiary care hospitals (hospital A amd hospital B) from June 1, 2010, to August 31, 2010, and who received acetaminophen during their hospitalization. Patients' acetaminophen administration records (including drug name, dose, administration time, hospital units, etc), demographic data, diagnoses, and results from liver function tests were obtained. The main outcome measures included acetaminophen exposure rate and supratherapeutic dosing rate among hospitalized patients, hazard ratios (HRs) and 95% confidence intervals (CIs) for risk factors for supratherapeutic dosing, and changes in liver function test before and after supratherapeutic dosing. RESULTS: A total of 14 411 patients (60.7%) were exposed to acetaminophen, of whom 955 (6.6%) exceeded the 4 g per day maximum recommended dose. In addition, 22.3% of patients who were 65 years or older and 17.6% of patients with chronic liver diseases exceeded the recommended limit of 3 g per day. Patients receiving excessive doses of acetaminophen tended to have significant alkaline phosphatase elevations, although causal relationship cannot be concluded. A significantly higher risk of supratherapeutic dosing was observed in hospital A (HR, 1.6 [95% CI, 1.4-1.8]), white patients (HR, 1.5 [95% CI, 1.3-1.7]), patients diagnosed as having osteoarthritis (HR, 1.4 [95% CI, 1.3-1.6]), and those who received scheduled administrations (HR, 16.6 [95% CI, 13.5-20.6]), multiple product formulations (HR, 2.4 [95% CI 2.0-2.9]), or the 500-mg strength formulation (HR, 1.9 [95% CI, 1.5-2.3]). A lower risk was found for pro re nata (as needed) administrations (HR, 0.7 [95% CI, 0.6-0.9]) and in nonsurgical and non­intensive care units (HR, 0.6 [95% CI, 0.5-0.7]). CONCLUSIONS: Supratherapeutic dosing of acetaminophen was significantly associated with multiple factors. Interventions to reduce the incidence of some risk factors may prevent supratherapeutic acetaminophen dosing in hospitalized patients.

How many medication orders are entered through free-text in EHRs?--a study on hypoglycemic agents.

Computerized Provider Order Entry (CPOE) can reduce medication errors; however, its benefits are only achieved when data are entered in a structured format and entries are properly coded. This paper aims to explore the incidence of free-text medication order entries involving hypoglycemic agents in an ambulatory electronic health record (EHR) system with CPOE. Our results showed that free-text order entry continues to be frequent. During 2010, 9.3% of hypoglycemic agents were entered as free-text for 2,091 patients. 17.4% of the entries contained misspellings. The highest proportion of free-text entries were found in urgent care clinics (49.4%) and among registered nurses (31.5%). Additionally, 92 drug-drug interaction alerts were not triggered due to free-text entries. Only 25.9% of the patients had diabetes recorded in their problem list. Solutions will require policy to enforce structured entry, ongoing improvement in user-interface design, improved training for users, and strategies for maintaining a complete medication list.

A legal framework to enable sharing of Clinical Decision Support knowledge and services across institutional boundaries.

The goal of the CDS Consortium (CDSC) is to assess, define, demonstrate, and evaluate best practices for knowledge management and clinical decision support in healthcare information technology at scale - across multiple ambulatory care settings and Electronic Health Record technology platforms. In the course of the CDSC research effort, it became evident that a sound legal foundation was required for knowledge sharing and clinical decision support services in order to address data sharing, intellectual property, accountability, and liability concerns. This paper outlines the framework utilized for developing agreements in support of sharing, accessing, and publishing content via the CDSC Knowledge Management Portal as well as an agreement in support of deployment and consumption of CDSC developed web services in the context of a research project under IRB oversight.

Using Medical Text Extraction, Reasoning and Mapping System (MTERMS) to process medication information in outpatient clinical notes.

Clinical information is often coded using different terminologies, and therefore is not interoperable. Our goal is to develop a general natural language processing (NLP) system, called Medical Text Extraction, Reasoning and Mapping System (MTERMS), which encodes clinical text using different terminologies and simultaneously establishes dynamic mappings between them. MTERMS applies a modular, pipeline approach flowing from a preprocessor, semantic tagger, terminology mapper, context analyzer, and parser to structure inputted clinical notes. Evaluators manually reviewed 30 free-text and 10 structured outpatient clinical notes compared to MTERMS output. MTERMS achieved an overall F-measure of 90.6 and 94.0 for free-text and structured notes respectively for medication and temporal information. The local medication terminology had 83.0% coverage compared to RxNorm's 98.0% coverage for free-text notes. 61.6% of mappings between the terminologies are exact match. Capture of duration was significantly improved (91.7% vs. 52.5%) from systems in the third i2b2 challenge.