Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

BACKGROUND: Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR. OBJECTIVES: This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic. METHODS: The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software. RESULTS: The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system. CONCLUSION: This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Machine learning links unresolving secondary pneumonia to mortality in patients with severe pneumonia, including COVID-19.

BACKGROUNDDespite guidelines promoting the prevention and aggressive treatment of ventilator-associated pneumonia (VAP), the importance of VAP as a driver of outcomes in mechanically ventilated patients, including patients with severe COVID-19, remains unclear. We aimed to determine the contribution of unsuccessful treatment of VAP to mortality for patients with severe pneumonia.METHODSWe performed a single-center, prospective cohort study of 585 mechanically ventilated patients with severe pneumonia and respiratory failure, 190 of whom had COVID-19, who underwent at least 1 bronchoalveolar lavage. A panel of intensive care unit (ICU) physicians adjudicated the pneumonia episodes and endpoints on the basis of clinical and microbiological data. Given the relatively long ICU length of stay (LOS) among patients with COVID-19, we developed a machine-learning approach called CarpeDiem, which grouped similar ICU patient-days into clinical states based on electronic health record data.RESULTSCarpeDiem revealed that the long ICU LOS among patients with COVID-19 was attributable to long stays in clinical states characterized primarily by respiratory failure. While VAP was not associated with mortality overall, the mortality rate was higher for patients with 1 episode of unsuccessfully treated VAP compared with those with successfully treated VAP (76.4% versus 17.6%, P < 0.001). For all patients, including those with COVID-19, CarpeDiem demonstrated that unresolving VAP was associated with a transitions to clinical states associated with higher mortality.CONCLUSIONSUnsuccessful treatment of VAP is associated with higher mortality. The relatively long LOS for patients with COVID-19 was primarily due to prolonged respiratory failure, placing them at higher risk of VAP.FUNDINGNational Institute of Allergy and Infectious Diseases (NIAID), NIH grant U19AI135964; National Heart, Lung, and Blood Institute (NHLBI), NIH grants R01HL147575, R01HL149883, R01HL153122, R01HL153312, R01HL154686, R01HL158139, P01HL071643, and P01HL154998; National Heart, Lung, and Blood Institute (NHLBI), NIH training grants T32HL076139 and F32HL162377; National Institute on Aging (NIA), NIH grants K99AG068544, R21AG075423, and P01AG049665; National Library of Medicine (NLM), NIH grant R01LM013337; National Center for Advancing Translational Sciences (NCATS), NIH grant U01TR003528; Veterans Affairs grant I01CX001777; Chicago Biomedical Consortium grant; Northwestern University Dixon Translational Science Award; Simpson Querrey Lung Institute for Translational Science (SQLIFTS); Canning Thoracic Institute of Northwestern Medicine.

Accelerating Food Allergy Research: Need for a Data Commons.

Food allergy is a significant health problem affecting approximately 8% of children and 11% of adults in the United States. It exhibits all the characteristics of a "complex" genetic trait; therefore, it is necessary to look at very large numbers of patients, far more than exist at any single organization, to eliminate gaps in the current understanding of this complex chronic disorder. Advances may be achieved by bringing together food allergy data from large numbers of patients into a Data Commons, a secure and efficient platform for researchers, comprising standardized data, available in a common interface for download and/or analysis, in accordance with the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. Prior data commons initiatives indicate that research community consensus and support, formal food allergy ontology, data standards, an accepted platform and data management tools, an agreed upon infrastructure, and trusted governance are the foundation of any successful data commons. In this article, we will present the justification for the creation of a food allergy data commons and describe the core principles that can make it successful and sustainable.

Developing a standardized but extendable framework to increase the findability of infectious disease datasets.

Biomedical datasets are increasing in size, stored in many repositories, and face challenges in FAIRness (findability, accessibility, interoperability, reusability). As a Consortium of infectious disease researchers from 15 Centers, we aim to adopt open science practices to promote transparency, encourage reproducibility, and accelerate research advances through data reuse. To improve FAIRness of our datasets and computational tools, we evaluated metadata standards across established biomedical data repositories. The vast majority do not adhere to a single standard, such as Schema.org, which is widely-adopted by generalist repositories. Consequently, datasets in these repositories are not findable in aggregation projects like Google Dataset Search. We alleviated this gap by creating a reusable metadata schema based on Schema.org and catalogued nearly 400 datasets and computational tools we collected. The approach is easily reusable to create schemas interoperable with community standards, but customized to a particular context. Our approach enabled data discovery, increased the reusability of datasets from a large research consortium, and accelerated research. Lastly, we discuss ongoing challenges with FAIRness beyond discoverability.

Characterizing variability of electronic health record-driven phenotype definitions.

OBJECTIVE: The aim of this study was to analyze a publicly available sample of rule-based phenotype definitions to characterize and evaluate the variability of logical constructs used. MATERIALS AND METHODS: A sample of 33 preexisting phenotype definitions used in research that are represented using Fast Healthcare Interoperability Resources and Clinical Quality Language (CQL) was analyzed using automated analysis of the computable representation of the CQL libraries. RESULTS: Most of the phenotype definitions include narrative descriptions and flowcharts, while few provide pseudocode or executable artifacts. Most use 4 or fewer medical terminologies. The number of codes used ranges from 5 to 6865, and value sets from 1 to 19. We found that the most common expressions used were literal, data, and logical expressions. Aggregate and arithmetic expressions are the least common. Expression depth ranges from 4 to 27. DISCUSSION: Despite the range of conditions, we found that all of the phenotype definitions consisted of logical criteria, representing both clinical and operational logic, and tabular data, consisting of codes from standard terminologies and keywords for natural language processing. The total number and variety of expressions are low, which may be to simplify implementation, or authors may limit complexity due to data availability constraints. CONCLUSIONS: The phenotype definitions analyzed show significant variation in specific logical, arithmetic, and other operators but are all composed of the same high-level components, namely tabular data and logical expressions. A standard representation for phenotype definitions should support these formats and be modular to support localization and shared logic.

Development of a social and environmental determinants of health informatics maturity model.

Introduction: Integrating social and environmental determinants of health (SEDoH) into enterprise-wide clinical workflows and decision-making is one of the most important and challenging aspects of improving health equity. We engaged domain experts to develop a SEDoH informatics maturity model (SIMM) to help guide organizations to address technical, operational, and policy gaps. Methods: We established a core expert group consisting of developers, informaticists, and subject matter experts to identify different SIMM domains and define maturity levels. The candidate model (v0.9) was evaluated by 15 informaticists at a Center for Data to Health community meeting. After incorporating feedback, a second evaluation round for v1.0 collected feedback and self-assessments from 35 respondents from the National COVID Cohort Collaborative, the Center for Leading Innovation and Collaboration's Informatics Enterprise Committee, and a publicly available online self-assessment tool. Results: We developed a SIMM comprising seven maturity levels across five domains: data collection policies, data collection methods and technologies, technology platforms for analysis and visualization, analytics capacity, and operational and strategic impact. The evaluation demonstrated relatively high maturity in analytics and technological capacity, but more moderate maturity in operational and strategic impact among academic medical centers. Changes made to the tool in between rounds improved its ability to discriminate between intermediate maturity levels. Conclusion: The SIMM can help organizations identify current gaps and next steps in improving SEDoH informatics. Improving the collection and use of SEDoH data is one important component of addressing health inequities.

Development of Food Allergy Data Dictionary: Toward a Food Allergy Data Commons.

BACKGROUND: Food allergy (FA) data lacks a common base of terminology and hinders data exchange among institutions. OBJECTIVE: To examine the current FA concept coverage by clinical terminologies and to develop and evaluate a Food Allergy Data Dictionary (FADD). METHODS: Allergy/immunology templates and patient intake forms from 4 academic medical centers with expertise in FA were systematically reviewed, and in-depth discussions with a panel of FA experts were conducted to identify important FA clinical concepts and data elements. The candidate ontology was iteratively refined through a series of virtual meetings. The concepts were mapped to existing clinical terminologies manually with the ATHENA vocabulary browser. Finally, the revised dictionary document was vetted with experts across 22 academic FA centers and 3 industry partners. RESULTS: A consensus version 1.0 FADD was finalized in November 2020. The FADD v1.0 contained 936 discrete FA concepts that were grouped into 14 categories. The categories included both FA-specific concepts, such as foods triggering reactions, and general health care categories, such as medications. Although many FA concepts are included in existing clinical terminologies, some critical concepts are missing. CONCLUSIONS: The FADD provides a pragmatic tool that can enable improved structured coding of FA data for both research and clinical uses, as well as lay the foundation for the development of standardized FA structured data entry forms.

Examining perspectives on the adoption and use of computer-based patient-reported outcomes among clinicians and health professionals: a Q methodology study.

OBJECTIVE: To determine factors that influence the adoption and use of patient-reported outcomes (PROs) in the electronic health record (EHR) among users. MATERIALS AND METHODS: Q methodology, supported by focus groups, semistructured interviews, and a review of the literature was used for data collection about opinions on PROs in the EHR. An iterative thematic analysis resulted in 49 statements that study participants sorted, from most unimportant to most important, under the following condition of instruction: "What issues are most important or most unimportant to you when you think about the adoption and use of patient-reported outcomes within the electronic health record in routine clinical care?" Using purposive sampling, 50 participants were recruited to rank and sort the 49 statements online, using HTMLQ software. Principal component analysis and Varimax rotation were used for data analysis using the PQMethod software. RESULTS: Participants were mostly physicians (24%) or physician/researchers (20%). Eight factors were identified. Factors included the ability of PROs in the EHR to enable: efficient and reliable use; care process improvement and accountability; effective and better symptom assessment; patient involvement for care quality; actionable and practical clinical decisions; graphical review and interpretation of results; use for holistic care planning to reflect patients' needs; and seamless use for all users. DISCUSSION: The success of PROs in the EHR in clinical settings is not dependent on a "one size fits all" strategy, demonstrated by the diversity of viewpoints identified in this study. A sociotechnical approach for implementing PROs in the EHR may help improve its success and sustainability. CONCLUSIONS: PROs in the EHR are most important to users when the technology is used to improve patient outcomes. Future research must focus on the impact of embedding this EHR functionality on care processes.

Infobuttons for Genomic Medicine: Requirements and Barriers.

OBJECTIVES: The study aimed to understand potential barriers to the adoption of health information technology projects that are released as free and open source software (FOSS). METHODS: We conducted a survey of research consortia participants engaged in genomic medicine implementation to assess perceived institutional barriers to the adoption of three systems: ClinGen electronic health record (EHR) Toolkit, DocUBuild, and MyResults.org. The survey included eight barriers from the Consolidated Framework for Implementation Research (CFIR), with additional barriers identified from a qualitative analysis of open-ended responses. RESULTS: We analyzed responses from 24 research consortia participants from 18 institutions. In total, 14 categories of perceived barriers were evaluated, which were consistent with other observed barriers to FOSS adoption. The most frequent perceived barriers included lack of adaptability of the system, lack of institutional priority to implement, lack of trialability, lack of advantage of alternative systems, and complexity. CONCLUSION: In addition to understanding potential barriers, we recommend some strategies to address them (where possible), including considerations for genomic medicine. Overall, FOSS developers need to ensure systems are easy to trial and implement and need to clearly articulate benefits of their systems, especially when alternatives exist. Institutional champions will remain a critical component to prioritizing genomic medicine projects.

A retrospective look at the predictions and recommendations from the 2009 AMIA policy meeting: did we see EHR-related clinician burnout coming?

Clinicians often attribute much of their burnout experience to use of the electronic health record, the adoption of which was greatly accelerated by the Health Information Technology for Economic and Clinical Health Act of 2009. That same year, AMIA's Policy Meeting focused on possible unintended consequences associated with rapid implementation of electronic health records, generating 17 potential consequences and 15 recommendations to address them. At the 2020 annual meeting of the American College of Medical Informatics (ACMI), ACMI fellows participated in a modified Delphi process to assess the accuracy of the 2009 predictions and the response to the recommendations. Among the findings, the fellows concluded that the degree of clinician burnout and its contributing factors, such as increased documentation requirements, were significantly underestimated. Conversely, problems related to identify theft and fraud were overestimated. Only 3 of the 15 recommendations were adjudged more than half-addressed.

The Implementation Chasm Hindering Genome-informed Health Care.

The promises of precision medicine are often heralded in the medical and lay literature, but routine integration of genomics in clinical practice is still limited. While the "last mile' infrastructure to bring genomics to the bedside has been demonstrated in some healthcare settings, a number of challenges remain - both in the receptivity of today's health system and in its technical and educational readiness to respond to this evolution in care. To improve the impact of genomics on health and disease management, we will need to integrate both new knowledge and new care processes into existing workflows. This change will be onerous and time-consuming, but hopefully valuable to the provision of high quality, economically feasible care worldwide.

Planning for patient-reported outcome implementation: Development of decision tools and practical experience across four clinics.

INTRODUCTION: Many institutions are attempting to implement patient-reported outcome (PRO) measures. Because PROs often change clinical workflows significantly for patients and providers, implementation choices can have major impact. While various implementation guides exist, a stepwise list of decision points covering the full implementation process and drawing explicitly on a sociotechnical conceptual framework does not exist. METHODS: To facilitate real-world implementation of PROs in electronic health records (EHRs) for use in clinical practice, members of the EHR Access to Seamless Integration of Patient-Reported Outcomes Measurement Information System (PROMIS) Consortium developed structured PRO implementation planning tools. Each institution pilot tested the tools. Joint meetings led to the identification of critical sociotechnical success factors. RESULTS: Three tools were developed and tested: (1) a PRO Planning Guide summarizes the empirical knowledge and guidance about PRO implementation in routine clinical care; (2) a Decision Log allows decision tracking; and (3) an Implementation Plan Template simplifies creation of a sharable implementation plan. Seven lessons learned during implementation underscore the iterative nature of planning and the importance of the clinician champion, as well as the need to understand aims, manage implementation barriers, minimize disruption, provide ample discussion time, and continuously engage key stakeholders. CONCLUSIONS: Highly structured planning tools, informed by a sociotechnical perspective, enabled the construction of clear, clinic-specific plans. By developing and testing three reusable tools (freely available for immediate use), our project addressed the need for consolidated guidance and created new materials for PRO implementation planning. We identified seven important lessons that, while common to technology implementation, are especially critical in PRO implementation.

Rich Text Formatted EHR Narratives: A Hidden and Ignored Trove.

This study presents an approach for mining structured information from clinical narratives in Electronic Health Records (EHRs) by using Rich Text Formatted (RTF) records. RTF is adopted by many medical information management systems. There is rich structural information in these files which can be extracted and interpreted, yet such information is largely ignored. We investigate multiple types of EHR narratives in the Enterprise Data Warehouse from a multisite large healthcare chain consisting of both, an academic medical center and community hospitals. We focus on the RTF constructs related to tables and sections that are not available in plain text EHR narratives. We show how to parse these RTF constructs, analyze their prevalence and characteristics in the context of multiple types of EHR narratives. Our case study demonstrates the additional utility of the features derived from RTF constructs over plain text oriented NLP.

Provider perspectives on the integration of patient-reported outcomes in an electronic health record.

OBJECTIVE: Integrating patient-reported outcomes (PROs) into electronic health records (EHRs) can improve patient-provider communication and delivery of care. However, new system implementation in health-care institutions is often accompanied by a change in clinical workflow and organizational culture. This study examines how well an EHR-integrated PRO system fits clinical workflows and individual needs of different provider groups within 2 clinics. MATERIALS AND METHODS: Northwestern Medicine developed and implemented an EHR-integrated PRO system within the orthopedics and oncology departments. We conducted interviews with 11 providers who had interacted with the system. Through thematic analysis, we synthesized themes regarding provider perspectives on clinical workflow, individual needs, and system features. RESULTS: Our findings show that EHR-integrated PROs facilitate targeted conversation with patients and automated triage for psychosocial care. However, physicians, psychosocial providers, and medical assistants faced different challenges in their use of the PRO system. Barriers mainly stemmed from a lack of actionable data, workflow disruption, technical issues, and a lack of incentives. DISCUSSION: This study sheds light on the ecosystem around EHR-integrated PRO systems (such as user needs and organizational factors). We present recommendations to address challenges facing PRO implementation, such as optimizing data collection and auto-referral processes, improving data visualizations, designing effective educational materials, and prioritizing the primary user group. CONCLUSION: PRO integration into routine care can be beneficial but also require effective technology design and workflow configuration to reach full potential use. This study provides insights into how patient-generated health data can be better integrated into clinical practice and care delivery processes.

An ancillary genomics system to support the return of pharmacogenomic results.

Existing approaches to managing genetic and genomic test results from external laboratories typically include filing of text reports within the electronic health record, making them unavailable in many cases for clinical decision support. Even when structured computable results are available, the lack of adopted standards requires considerations for processing the results into actionable knowledge, in addition to storage and management of the data. Here, we describe the design and implementation of an ancillary genomics system used to receive and process heterogeneous results from external laboratories, which returns a descriptive phenotype to the electronic health record in support of pharmacogenetic clinical decision support.

Pharmacogenomic clinical decision support design and multi-site process outcomes analysis in the eMERGE Network.

To better understand the real-world effects of pharmacogenomic (PGx) alerts, this study aimed to characterize alert design within the eMERGE Network, and to establish a method for sharing PGx alert response data for aggregate analysis. Seven eMERGE sites submitted design details and established an alert logging data dictionary. Six sites participated in a pilot study, sharing alert response data from their electronic health record systems. PGx alert design varied, with some consensus around the use of active, post-test alerts to convey Clinical Pharmacogenetics Implementation Consortium recommendations. Sites successfully shared response data, with wide variation in acceptance and follow rates. Results reflect the lack of standardization in PGx alert design. Standards and/or larger studies will be necessary to fully understand PGx impact. This study demonstrated a method for sharing PGx alert response data and established that variation in system design is a significant barrier for multi-site analyses.

Biomedical informatics meets data science: current state and future directions for interaction.

There are an ever-increasing number of reports and commentaries that describe the challenges and opportunities associated with the use of big data and data science (DS) in the context of biomedical education, research, and practice. These publications argue that there are substantial benefits resulting from the use of data-centric approaches to solve complex biomedical problems, including an acceleration in the rate of scientific discovery, improved clinical decision making, and the ability to promote healthy behaviors at a population level. In addition, there is an aligned and emerging body of literature that describes the ethical, legal, and social issues that must be addressed to responsibly use big data in such contexts. At the same time, there has been growing recognition that the challenges and opportunities being attributed to the expansion in DS often parallel those experienced by the biomedical informatics community. Indeed, many informaticians would consider some of these issues relevant to the core theories and methods incumbent to the field of biomedical informatics science and practice. In response to this topic area, during the 2016 American College of Medical Informatics Winter Symposium, a series of presentations and focus group discussions intended to define the current state and identify future directions for interaction and collaboration between people who identify themselves as working on big data, DS, and biomedical informatics were conducted. We provide a perspective concerning these discussions and the outcomes of that meeting, and also present a set of recommendations that we have generated in response to a thematic analysis of those same outcomes. Ultimately, this report is intended to: (1) summarize the key issues currently being discussed by the biomedical informatics community as it seeks to better understand how to constructively interact with the emerging biomedical big data and DS fields; and (2) propose a framework and agenda that can serve to advance this type of constructive interaction, with mutual benefit accruing to both fields.

Segment convolutional neural networks (Seg-CNNs) for classifying relations in clinical notes.

We propose Segment Convolutional Neural Networks (Seg-CNNs) for classifying relations from clinical notes. Seg-CNNs use only word-embedding features without manual feature engineering. Unlike typical CNN models, relations between 2 concepts are identified by simultaneously learning separate representations for text segments in a sentence: preceding, concept1, middle, concept2, and succeeding. We evaluate Seg-CNN on the i2b2/VA relation classification challenge dataset. We show that Seg-CNN achieves a state-of-the-art micro-average F-measure of 0.742 for overall evaluation, 0.686 for classifying medical problem-treatment relations, 0.820 for medical problem-test relations, and 0.702 for medical problem-medical problem relations. We demonstrate the benefits of learning segment-level representations. We show that medical domain word embeddings help improve relation classification. Seg-CNNs can be trained quickly for the i2b2/VA dataset on a graphics processing unit (GPU) platform. These results support the use of CNNs computed over segments of text for classifying medical relations, as they show state-of-the-art performance while requiring no manual feature engineering.

Natural Language Processing for EHR-Based Pharmacovigilance: A Structured Review.

The goal of pharmacovigilance is to detect, monitor, characterize and prevent adverse drug events (ADEs) with pharmaceutical products. This article is a comprehensive structured review of recent advances in applying natural language processing (NLP) to electronic health record (EHR) narratives for pharmacovigilance. We review methods of varying complexity and problem focus, summarize the current state-of-the-art in methodology advancement, discuss limitations and point out several promising future directions. The ability to accurately capture both semantic and syntactic structures in clinical narratives becomes increasingly critical to enable efficient and accurate ADE detection. Significant progress has been made in algorithm development and resource construction since 2000. Since 2012, statistical analysis and machine learning methods have gained traction in automation of ADE mining from EHR narratives. Current state-of-the-art methods for NLP-based ADE detection from EHRs show promise regarding their integration into production pharmacovigilance systems. In addition, integrating multifaceted, heterogeneous data sources has shown promise in improving ADE detection and has become increasingly adopted. On the other hand, challenges and opportunities remain across the frontier of NLP application to EHR-based pharmacovigilance, including proper characterization of ADE context, differentiation between off- and on-label drug-use ADEs, recognition of the importance of polypharmacy-induced ADEs, better integration of heterogeneous data sources, creation of shared corpora, and organization of shared-task challenges to advance the state-of-the-art.