Data standards for interoperability of care team information to support care coordination of complex pediatric patients.

OBJECTIVE: Seamless access to information about the individuals and organizations involved in the care of a specific patient ("care teams") is crucial to effective and efficient care coordination. This is especially true for vulnerable and complex patient populations such as pediatric patients with special needs. Despite wide adoption of electronic health records (EHR), current EHR systems do not adequately support the visualization and management of care teams within and across health care organizations. Electronic health information exchange has the potential to address this issue. In the present study, we assessed the adequacy of available health information exchange data standards to support the information needs related to care coordination of complex pediatric patients. METHODS: We derived data elements from the information needs of clinicians and parents to support patient care teams; and mapped them to data elements in the Health Level Seven (HL7) Consolidated Clinical Document Architecture (C-CDA) standard and in the HL7 Fast Healthcare Interoperability Resources (FHIR) standard. We also identified additional C-CDA data elements and FHIR resources that include patients' care team members. RESULTS: Information about care team members involved in patient care is generally well-represented in the C-CDA and FHIR specifications. However, there are gaps related to patients' non-clinical events and care team actions. In addition, there is no single place to find information about care team members; rather, information about practitioners and organizations may be available in several different types of C-CDA data elements and FHIR resources. CONCLUSION: Through standards-based electronic health information exchange, it appears to be feasible to build patient care team representations irrespective of the location of patient care. In order to gather care team information across disparate systems, exchange of multiple C-CDA documents and/or execution of multiple FHIR queries will be necessary. This approach has the potential to enable comprehensive patient care team views that may help improve care coordination.

Comprehensive methodology to monitor longitudinal change patterns during EHR implementations: a case study at a large health care delivery network.

OBJECTIVE: To test a systematic methodology to monitor longitudinal change patterns on quality, productivity, and safety outcomes during a large-scale commercial Electronic Health Record (EHR) implementation. MATERIALS AND METHODS: Our method combines an interrupted time-series design with control sites and 41 consensus outcomes including quality (11 measures), productivity (20 measures), and safety (10 measures). The intervention consisted of a phased commercial EHR implementation at a large health care delivery network. Four medium-size hospitals and 39 clinics from 5 geographic regions implementing the new EHR were compared against a parallel control consisting of one medium-size and one large hospital and 10 clinics that had not implemented the new EHR at the time of this study. We collected monthly data from February 2013 to July 2017. RESULTS: The proposed methodology was successfully implemented and significant changes were observed in most measured variables. A significant change attributable to the intervention was observed in 12 (29%) measures in three or more regions; in 32 (78%) measures in two or more regions; and in 40 (98%) measures in at least one region. A similar pattern (i.e., same impact in three or more regions) was detected for nine (22%) measures, a mixed pattern (i.e., same impact in two regions, and different impact in other regions) was detected for nine (22%) measures, and an inconsistent pattern (i.e., did not detect the same impact across regions) was detected for 23 (56%) measures. DISCUSSION: Using a formal methodology to assess changes in a set of consensus measures, we detected various patterns of impact and mixed time-sensitive effects. With an increasing adoption of EHR systems, it is critical for health care organizations to systematically monitor their EHR implementations. The proposed method provides a robust and consistent approach to monitor EHR implementations longitudinally allowing for continuous monitoring after the system becomes stable in order to avoid unexpected effects. CONCLUSION: Our results and methodology can guide the broader medical and informatics communities by informing what and how to continuously monitor EHR impact on quality, productivity, and safety.

Development and classification of a robust inventory of near real-time outcome measurements for assessing information technology interventions in health care.

OBJECTIVE: To develop and classify an inventory of near real-time outcome measures for assessing information technology (IT) interventions in health care and assess their relevance as perceived by experts in the field. MATERIALS AND METHODS: To verify the robustness and coverage of a previously published inventory of measures and taxonomy, we conducted semi-structured interviews with clinical and administrative leaders from a large care delivery system to collect suggestions of outcome measures that can be calculated with data available in electronic format for near real-time monitoring of EHR implementations. We combined these measures with the most commonly reported in the literature. We then conducted two online surveys with subject-matter experts to collect their perceptions of the relevance of the measures, and identify other potentially relevant measures. RESULTS: With input from experienced health care leaders and informaticists, we developed an inventory of 102 outcome measures. These measures were classified into a taxonomy of commonly used measures around the categories of quality, productivity, and safety. Safety measures were rated as most relevant by subject-matter experts, especially those measuring medication processes. Clinician satisfaction and measures assessing mean time to complete tasks and time spent on electronic documentation were also rated as highly relevant. DISCUSSION: By expanding the coverage of our previously published inventory and taxonomy, we expect to help providers, health IT vendors and researchers to more effectively and consistently monitor the impact of EHR implementations in near real-time, and report more standardized outcomes in future studies. We identified several measures not commonly assessed by previous studies of IT implementations, especially those of safety and productivity, which deserve more attention from the broader informatics community. CONCLUSION: Our inventory of measures and taxonomy will help researchers identify gaps in their measurement approaches and report more standardized measurements of IT interventions that could be shared among researchers, hopefully facilitating comparison across future studies and increasing our understanding of the impact of IT interventions in health care.

Health information technology adoption: Understanding research protocols and outcome measurements for IT interventions in health care.

OBJECTIVE: To classify and characterize the variables commonly used to measure the impact of Information Technology (IT) adoption in health care, as well as settings and IT interventions tested, and to guide future research. MATERIALS AND METHODS: We conducted a descriptive study screening a sample of 236 studies from a previous systematic review to identify outcome measures used and the availability of data to calculate these measures. We also developed a taxonomy of commonly used measures and explored setting characteristics and IT interventions. RESULTS: Clinical decision support is the most common intervention tested, primarily in non-hospital-based clinics and large academic hospitals. We identified 15 taxa representing the 79 most commonly used measures. Quality of care was the most common category of these measurements with 62 instances, followed by productivity (11 instances) and patient safety (6 instances). Measures used varied according to type of setting, IT intervention and targeted population. DISCUSSION: This study provides an inventory and a taxonomy of commonly used measures that will help researchers select measures in future studies as well as identify gaps in their measurement approaches. The classification of the other protocol components such as settings and interventions will also help researchers identify underexplored areas of research on the impact of IT interventions in health care. CONCLUSION: A more robust and standardized measurement system and more detailed descriptions of interventions and settings are necessary to enable comparison between studies and a better understanding of the impact of IT adoption in health care settings.

A domain analysis model for eIRB systems: addressing the weak link in clinical research informatics.

Institutional Review Boards (IRBs) are a critical component of clinical research and can become a significant bottleneck due to the dramatic increase, in both volume and complexity of clinical research. Despite the interest in developing clinical research informatics (CRI) systems and supporting data standards to increase clinical research efficiency and interoperability, informatics research in the IRB domain has not attracted much attention in the scientific community. The lack of standardized and structured application forms across different IRBs causes inefficient and inconsistent proposal reviews and cumbersome workflows. These issues are even more prominent in multi-institutional clinical research that is rapidly becoming the norm. This paper proposes and evaluates a domain analysis model for electronic IRB (eIRB) systems, paving the way for streamlined clinical research workflow via integration with other CRI systems and improved IRB application throughput via computer-assisted decision support.

Assessing clinical researchers' information needs to create responsive portals and tools: my Research Assistant (MyRA) at the University of Utah: a case study.

QUESTION: How can health sciences librarians and biomedical informaticians offer relevant support to Clinical and Translational Science Award (CTSA) personnel? SETTING: The Spencer S. Eccles Health Sciences Library and the associate vice president for information technology for the health sciences office at the University of Utah conducted a needs assessment. METHODS: Faculty and staff from these two units, with the services of a consultant and other CTSA partners, employed a survey, focus groups, interviews, and committee discussions. An information portal was created to meet identified needs. RESULTS: A directive white paper was created. The process employed to plan a virtual and physical collaborative, collegial space for clinical researchers at the university and its three inter-institutional CTSA partners is described. CONCLUSION: The university's model can assist other librarians and informaticians with how to become part of a CTSA-focused infrastructure for clinical and translational research and serve researchers in general.

GARDE: a standards-based clinical decision support platform for identifying population health management cohorts.

Population health management (PHM) is an important approach to promote wellness and deliver health care to targeted individuals who meet criteria for preventive measures or treatment. A critical component for any PHM program is a data analytics platform that can target those eligible individuals. OBJECTIVE: The aim of this study was to design and implement a scalable standards-based clinical decision support (CDS) approach to identify patient cohorts for PHM and maximize opportunities for multi-site dissemination. MATERIALS AND METHODS: An architecture was established to support bidirectional data exchanges between heterogeneous electronic health record (EHR) data sources, PHM systems, and CDS components. HL7 Fast Healthcare Interoperability Resources and CDS Hooks were used to facilitate interoperability and dissemination. The approach was validated by deploying the platform at multiple sites to identify patients who meet the criteria for genetic evaluation of familial cancer. RESULTS: The Genetic Cancer Risk Detector (GARDE) platform was created and is comprised of four components: (1) an open-source CDS Hooks server for computing patient eligibility for PHM cohorts, (2) an open-source Population Coordinator that processes GARDE requests and communicates results to a PHM system, (3) an EHR Patient Data Repository, and (4) EHR PHM Tools to manage patients and perform outreach functions. Site-specific deployments were performed on onsite virtual machines and cloud-based Amazon Web Services. DISCUSSION: GARDE's component architecture establishes generalizable standards-based methods for computing PHM cohorts. Replicating deployments using one of the established deployment methods requires minimal local customization. Most of the deployment effort was related to obtaining site-specific information technology governance approvals.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center.

OBJECTIVE: To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations. MATERIALS AND METHODS: We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations. RESULTS: University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape. DISCUSSION: Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation. CONCLUSION: EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Evaluation of a flowchart-based EHR query system: a case study of RetroGuide.

Provision of query systems which are intuitive for non-experts has been recognized as an important informatics challenge. We developed a prototype of a flowchart-based analytical framework called RetroGuide that enables non-experts to formulate query tasks using a step-based, patient-centered paradigm inspired by workflow technology. We present results of the evaluation of RetroGuide in comparison to Structured Query Language (SQL) in laboratory settings using a mixed method design. We asked 18 human subjects with limited database experience to solve query tasks in RetroGuide and SQL, and quantitatively compared their test scores. A follow-up questionnaire was designed to compare both technologies qualitatively and investigate RetroGuide technology acceptance. The quantitative comparison of test scores showed that the study subjects achieved significantly higher scores using the RetroGuide technology. Qualitative study results indicated that 94% of subjects preferred RetroGuide to SQL because RetroGuide was easier to learn, it better supported temporal tasks, and it seemed to be a more logical modeling paradigm. Additional qualitative evaluation results, based on a technology acceptance model, suggested that a fully developed RetroGuide-like technology would be well accepted by users. Our study is an example of a structure validation study of a prototype query system, results of which provided significant guidance in further development of a novel query paradigm for EHR data. We discuss the strengths and weakness of our study design and results, and their implication for future evaluations of query systems in general.

Developing and Validating a Model for Detecting Longitudinal Inconsistencies in the Electronic Problem List.

Clinicians from different care settings can distort the problem list from conveying a patient's actual health status, affecting quality and patient safety. To measure this effect, a reference standard was built to derive a problem-list based model. Real-world problem lists were used to derive an ideal categorization cutoff score. The model was tested against patient records to categorize problem lists as either having longitudinal inconsistencies or not. The model was able to successfully categorize these events with ~87% accuracy, ~83% sensitivity, and ~89% specificity. This new model can be used to quantify intervention effects, can be reported in problem list studies, and can be used to measure problem list changes based on policy, workflow, or system changes.

Patient-level analysis of outcomes using structured labor and delivery data.

This paper presents methods for identifying and analyzing associations among nursing care processes, patient attributes, and patient outcomes using unit-level and patient-level representations of care derived from computerized nurse documentation. The retrospective, descriptive analysis included documented nursing events for 900 Labor and Delivery patients at three hospitals over the 2-month period of January and February 2006. Two models were used to produce quantified measurements of nursing care received by each patient. The first model considered only the hourly census of nurses and patients. The second model considered the size of nurses' patient loads as represented by computerized nurse-entered documentation. Significant relationships were identified between durations of labor and nursing care scores generated by the second model. In addition to the clinical associations identified, the study demonstrated an approach with global application for representing the amount of nursing care received at the individual patient level in analyses of patient outcomes.

A case for using grid architecture for state public health informatics: the Utah perspective.

This paper presents the rationale for designing and implementing the next-generation of public health information systems using grid computing concepts and tools. Our attempt is to evaluate all grid types including data grids for sharing information and computational grids for accessing computational resources on demand. Public health is a broad domain that requires coordinated uses of disparate and heterogeneous information systems. System interoperability in public health is limited. The next-generation public health information systems must overcome barriers to integration and interoperability, leverage advances in information technology, address emerging requirements, and meet the needs of all stakeholders. Grid-based architecture provides one potential technical solution that deserves serious consideration. Within this context, we describe three discrete public health information system problems and the process by which the Utah Department of Health (UDOH) and the Department of Biomedical Informatics at the University of Utah in the United States has approached the exploration for eventual deployment of a Utah Public Health Informatics Grid. These three problems are: i) integration of internal and external data sources with analytic tools and computational resources; ii) provide external stakeholders with access to public health data and services; and, iii) access, integrate, and analyze internal data for the timely monitoring of population health status and health services. After one year of experience, we have successfully implemented federated queries across disparate administrative domains, and have identified challenges and potential solutions concerning the selection of candidate analytic grid services, data sharing concerns, security models, and strategies for reducing expertise required at a public health agency to implement a public health grid.

Looking Behind the Curtain: Identifying Factors Contributing to Changes on Care Outcomes During a Large Commercial EHR Implementation.

OBJECTIVE: To identify factors contributing to changes on quality, productivity, and safety outcomes during a large commercial electronic health record (EHR) implementation and to guide future research. METHODS: We conducted a mixed-methods study assessing the impact of a commercial EHR implementation. The method consisted of a quantitative longitudinal evaluation followed by qualitative semi-structured, in-depth interviews with clinical employees from the same implementation. Fourteen interviews were recorded and transcribed. Three authors independently coded interview narratives and via consensus identified factors contributing to changes on 15 outcomes of quality, productivity, and safety. RESULTS: We identified 14 factors that potentially affected the outcomes previously monitored. Our findings demonstrate that several factors related to the implementation (e.g., incomplete data migration), partially related (e.g., intentional decrease in volume of work), and not related (e.g., health insurance changes) may affect outcomes in different ways. DISCUSSION: This is the first study to investigate factors contributing to changes on a broad set of quality, productivity, and safety outcomes during an EHR implementation guided by the results of a large longitudinal evaluation. The diversity of factors identified indicates that the need for organizational adaptation to take full advantage of new technologies is as important for health care as it is for other services sectors. CONCLUSIONS: We recommend continuous identification and monitoring of these factors in future evaluations to hopefully increase our understanding of the full impact of health information technology interventions.

Evaluating the accuracy of existing EMR data as predictors of follow-up providers.

In order to evaluate the accuracy of existing EMR data in predicting follow-up providers, a retrospective analysis was performed on six months of data for inpatient and ED encounters occurring at two hospitals, and on related outpatient data. Sensitivity and Positive Predictive Value (PPV) were calculated for each of eight predictors, to determine their effectiveness in predicting follow-up providers. Our findings indicate that access to longitudinal patient care records can improve prediction of which providers a patient is likely to see post-discharge compared to simply using Primary Care Provider data from admissions records. Of the predictors evaluated, a patient's past appointment history was the best predictor of which providers they would see in the future (PPV = 48% following inpatient visits, 35% following emergency department visits). However, even the best performing predictors failed to predict more than half of the follow-up providers and might generate many "false" alerts.

Effectiveness of topic-specific infobuttons: a randomized controlled trial.

OBJECTIVE: Infobuttons are decision support tools that provide links within electronic medical record systems to relevant content in online information resources. The aim of infobuttons is to help clinicians promptly meet their information needs. The objective of this study was to determine whether infobutton links that direct to specific content topics ("topic links") are more effective than links that point to general overview content ("nonspecific links"). DESIGN: Randomized controlled trial with a control and an intervention group. Clinicians in the control group had access to nonspecific links, while those in the intervention group had access to topic links. MEASUREMENTS: Infobutton session duration, number of infobutton sessions, session success rate, and the self-reported impact that the infobutton session produced on decision making. RESULTS: The analysis was performed on 90 subjects and 3,729 infobutton sessions. Subjects in the intervention group spent 17.4% less time seeking for information (35.5 seconds vs. 43 seconds, p = 0.008) than those in the control group. Subjects in the intervention group used infobuttons 20.5% (22 sessions vs. 17.5 sessions, p = 0.21) more often than in the control group, but the difference was not significant. The information seeking success rate was equally high in both groups (89.4% control vs. 87.2% intervention, p = 0.99). Subjects reported a high positive clinical impact (i.e., decision enhancement or knowledge update) in 62% of the sessions. Limitations The exclusion of users with a low frequency of infobutton use and the focus on medication-related information needs may limit the generalization of the results. The session outcomes measurement was based on clinicians' self-assessment and therefore prone to bias. CONCLUSION: The results support the hypothesis that topic links are more efficient than nonspecific links regarding the time seeking for information. It is unclear whether the statistical difference demonstrated will result in a clinically significant impact. However, the overall results confirm previous evidence that infobuttons are effective at helping clinicians to answer questions at the point of care and demonstrate a modest incremental change in the efficiency of information delivery for routine users of this tool.

Computerized alerts improve outpatient laboratory monitoring of transplant patients.

Authors evaluated the impact of computerized alerts on the quality of outpatient laboratory monitoring for transplant patients. For 356 outpatient liver transplant patients managed at LDS Hospital, Salt Lake City, this observational study compared traditional laboratory result reporting, using faxes and printouts, to computerized alerts implemented in 2004. Study alerts within the electronic health record notified clinicians of new results and overdue new orders for creatinine tests and immunosuppression drug levels. After implementing alerts, completeness of reporting increased from 66 to >99 %, as did positive predictive value that a report included new information (from 46 to >99 %). Timeliness of reporting and clinicians' responses improved after implementing alerts (p <0.001): median times for clinicians to receive and complete actions decreased to 9 hours from 33 hours using the prior traditional reporting system. Computerized alerts led to more efficient, complete, and timely management of laboratory information.

Modeling the distribution of Nursing Effort using structured Labor and Delivery documentation.

Our study objectives included the development and evaluation of models for representing the distribution of shared unit-wide nursing care resources among individual Labor and Delivery patients using quantified measurements of nursing care, referred to as Nursing Effort. The models were intended to enable discrimination between the amounts of care delivered to patient subsets defined by attributes such as patient acuity. For each of five proposed models, scores were generated using an analysis set of 686,402 computerized nurse-documented events associated with 1093 patients at three hospitals during January and February 2006. Significant differences were detected in Nursing Effort scores according to patient acuity, care facility, and in scores generated during shift change versus non-shift change hours. The development of nursing care quantification strategies proposed in this study supports outcomes analysis by establishing a foundation for measuring the effect of patient-level nursing care on individual patient outcomes.

Electronic problem lists: a thematic analysis of a systematic literature review to identify aspects critical to success.

Objective: Problem list data is a driving force for many beneficial clinical tools, yet these data remain underutilized. We performed a systematic literature review, pulling insights from previous research, aggregating insights into themes, and distilling themes into actionable advice. We sought to learn what changes we could make to existing applications, to the clinical workflow, and to clinicians' perceptions that would improve problem list utilization and increase the prevalence of problems data in the electronic medical record. Materials and Methods: We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to systematically curate a corpus of pertinent articles. We performed a thematic analysis, looking for interesting excerpts and ideas. By aggregating excerpts from many authors, we gained broader, more inclusive insights into what makes a good problem list and what factors are conducive to its success. Results: Analysis led to a list of 7 benefits of using the problem list, 15 aspects critical to problem list success, and knowledge to help inform policy development, such as consensus on what belongs on the problem list, who should maintain the problem list, and when. Conclusions: A list of suggestions is made on ways in which the problem list can be improved to increase utilization by clinicians. There is also a need for standard measurements of the problem list, so that lists can be measured, compared, and discussed with rigor and a common vocabulary.

Promoting national and international standards to build interoperable clinical applications.

During the last decade, software supporting healthcare delivery has proliferated. This software can be divided into electronic medical record (EHR) systems and applications that treat EHRs as platforms. These collect, manage, and interpret medical data, thereby adding value to associated EHRs. To reduce the burden of developing for multiple EHR platforms, a group of standards has evolved that allow software written for one vendor's EHR to be introduced into settings supported by other vendors. The Health Services Platform Consortium (HSPC) is a collaborative effort to advocate for standards that will make healthcare applications truly interoperable. In this document, we discuss the approach adopted by the consortium and the standards central to this approach. We discriminate between interoperability standards that support the plug-and-play transfer of applications from one vendor's EHR to another and knowledge portability standards that allow knowledge artifacts used in one software environment to be introduced effectively in others.

Enhancing a Commercial EMR with an Open, Standards-Based Publish-Subscribe Infrastructure.

Intermountain Healthcare has designed and implemented a publish-subscribe (PubSub) infrastructure to support essential event processing workflows across our organization. A recent implementation of a commercial EMR highlighted the need to provide this capability on top of the EMR to support external applications and services that require access to triggering events within the EMR. A description of the PubSub architecture is presented. Use cases for health information exchange, public health reporting, and pulmonary embolism diagnosis that utilize PubSub are described, along with benefits of using the paradigm. Besides providing support for these external applications, the PubSub infrastructure allows additional event handling functionality not available in the commercial EMR. The open, standards-based nature of the design should allow other organizations to implement the system in their information systems environment.