Comprehensive methodology to monitor longitudinal change patterns during EHR implementations: a case study at a large health care delivery network.

OBJECTIVE: To test a systematic methodology to monitor longitudinal change patterns on quality, productivity, and safety outcomes during a large-scale commercial Electronic Health Record (EHR) implementation. MATERIALS AND METHODS: Our method combines an interrupted time-series design with control sites and 41 consensus outcomes including quality (11 measures), productivity (20 measures), and safety (10 measures). The intervention consisted of a phased commercial EHR implementation at a large health care delivery network. Four medium-size hospitals and 39 clinics from 5 geographic regions implementing the new EHR were compared against a parallel control consisting of one medium-size and one large hospital and 10 clinics that had not implemented the new EHR at the time of this study. We collected monthly data from February 2013 to July 2017. RESULTS: The proposed methodology was successfully implemented and significant changes were observed in most measured variables. A significant change attributable to the intervention was observed in 12 (29%) measures in three or more regions; in 32 (78%) measures in two or more regions; and in 40 (98%) measures in at least one region. A similar pattern (i.e., same impact in three or more regions) was detected for nine (22%) measures, a mixed pattern (i.e., same impact in two regions, and different impact in other regions) was detected for nine (22%) measures, and an inconsistent pattern (i.e., did not detect the same impact across regions) was detected for 23 (56%) measures. DISCUSSION: Using a formal methodology to assess changes in a set of consensus measures, we detected various patterns of impact and mixed time-sensitive effects. With an increasing adoption of EHR systems, it is critical for health care organizations to systematically monitor their EHR implementations. The proposed method provides a robust and consistent approach to monitor EHR implementations longitudinally allowing for continuous monitoring after the system becomes stable in order to avoid unexpected effects. CONCLUSION: Our results and methodology can guide the broader medical and informatics communities by informing what and how to continuously monitor EHR impact on quality, productivity, and safety.

Development and classification of a robust inventory of near real-time outcome measurements for assessing information technology interventions in health care.

OBJECTIVE: To develop and classify an inventory of near real-time outcome measures for assessing information technology (IT) interventions in health care and assess their relevance as perceived by experts in the field. MATERIALS AND METHODS: To verify the robustness and coverage of a previously published inventory of measures and taxonomy, we conducted semi-structured interviews with clinical and administrative leaders from a large care delivery system to collect suggestions of outcome measures that can be calculated with data available in electronic format for near real-time monitoring of EHR implementations. We combined these measures with the most commonly reported in the literature. We then conducted two online surveys with subject-matter experts to collect their perceptions of the relevance of the measures, and identify other potentially relevant measures. RESULTS: With input from experienced health care leaders and informaticists, we developed an inventory of 102 outcome measures. These measures were classified into a taxonomy of commonly used measures around the categories of quality, productivity, and safety. Safety measures were rated as most relevant by subject-matter experts, especially those measuring medication processes. Clinician satisfaction and measures assessing mean time to complete tasks and time spent on electronic documentation were also rated as highly relevant. DISCUSSION: By expanding the coverage of our previously published inventory and taxonomy, we expect to help providers, health IT vendors and researchers to more effectively and consistently monitor the impact of EHR implementations in near real-time, and report more standardized outcomes in future studies. We identified several measures not commonly assessed by previous studies of IT implementations, especially those of safety and productivity, which deserve more attention from the broader informatics community. CONCLUSION: Our inventory of measures and taxonomy will help researchers identify gaps in their measurement approaches and report more standardized measurements of IT interventions that could be shared among researchers, hopefully facilitating comparison across future studies and increasing our understanding of the impact of IT interventions in health care.

Health information technology adoption: Understanding research protocols and outcome measurements for IT interventions in health care.

OBJECTIVE: To classify and characterize the variables commonly used to measure the impact of Information Technology (IT) adoption in health care, as well as settings and IT interventions tested, and to guide future research. MATERIALS AND METHODS: We conducted a descriptive study screening a sample of 236 studies from a previous systematic review to identify outcome measures used and the availability of data to calculate these measures. We also developed a taxonomy of commonly used measures and explored setting characteristics and IT interventions. RESULTS: Clinical decision support is the most common intervention tested, primarily in non-hospital-based clinics and large academic hospitals. We identified 15 taxa representing the 79 most commonly used measures. Quality of care was the most common category of these measurements with 62 instances, followed by productivity (11 instances) and patient safety (6 instances). Measures used varied according to type of setting, IT intervention and targeted population. DISCUSSION: This study provides an inventory and a taxonomy of commonly used measures that will help researchers select measures in future studies as well as identify gaps in their measurement approaches. The classification of the other protocol components such as settings and interventions will also help researchers identify underexplored areas of research on the impact of IT interventions in health care. CONCLUSION: A more robust and standardized measurement system and more detailed descriptions of interventions and settings are necessary to enable comparison between studies and a better understanding of the impact of IT adoption in health care settings.

Measuring use of electronic health record functionality using system audit information.

Meaningful and efficient methods for measuring Electronic Health Record (EHR) adoption and functional usage patterns have recently become important for hospitals, clinics, and health care networks in the United State due to recent government initiatives to increase EHR use. To date, surveys have been the method of choice to measure EHR adoption. This paper describes another method for measuring EHR adoption which capitalizes on audit logs, which are often common components of modern EHRs. An Audit Data Mart is described which identified EHR functionality within 836 Departments, within 22 Hospitals and 170 clinics at Intermountain Healthcare, a large integrated delivery system. The Audit Data Mart successfully identified important and differing EHR functional usage patterns. These patterns were useful in strategic planning, tracking EHR implementations, and will likely be utilized to assist in documentation of "Meaningful Use" of EHR functionality.

Cervicoisthmic cerclage: transabdominal vs transvaginal approach.

OBJECTIVE: We sought to compare the outcomes of cervicoisthmic (CI) cerclage using traditional transabdominal (TA) approach vs the lesser used transvaginal (TV) approach. STUDY DESIGN: We conducted a retrospective cohort study of women who underwent placement of a CI cerclage. RESULTS: Before CI placement, the abdominal group had a total of 100 pregnancies that continued beyond the first trimester and had 27 (27%) surviving infants. After cerclage placement, there were 34 pregnancies and 24 (71%) surviving infants. Before cerclage placement, the vaginal group had a total of 90 pregnancies that continued beyond the first trimester and had 11 (12%) surviving infants. After cerclage placement, there were 29 pregnancies and 20 (69%) surviving infants. The vaginal cerclage group had a significantly shorter mean operative time of 33 vs 69 minutes, and shorter hospital stay of 0.5 vs 3.2 days. CONCLUSION: Both TV and TA CI cerclage offers select patients with cervical insufficiency improved neonatal survival. The TV placement of a CI has less morbidity than the TA approach with a comparable neonatal survival.

Looking Behind the Curtain: Identifying Factors Contributing to Changes on Care Outcomes During a Large Commercial EHR Implementation.

OBJECTIVE: To identify factors contributing to changes on quality, productivity, and safety outcomes during a large commercial electronic health record (EHR) implementation and to guide future research. METHODS: We conducted a mixed-methods study assessing the impact of a commercial EHR implementation. The method consisted of a quantitative longitudinal evaluation followed by qualitative semi-structured, in-depth interviews with clinical employees from the same implementation. Fourteen interviews were recorded and transcribed. Three authors independently coded interview narratives and via consensus identified factors contributing to changes on 15 outcomes of quality, productivity, and safety. RESULTS: We identified 14 factors that potentially affected the outcomes previously monitored. Our findings demonstrate that several factors related to the implementation (e.g., incomplete data migration), partially related (e.g., intentional decrease in volume of work), and not related (e.g., health insurance changes) may affect outcomes in different ways. DISCUSSION: This is the first study to investigate factors contributing to changes on a broad set of quality, productivity, and safety outcomes during an EHR implementation guided by the results of a large longitudinal evaluation. The diversity of factors identified indicates that the need for organizational adaptation to take full advantage of new technologies is as important for health care as it is for other services sectors. CONCLUSIONS: We recommend continuous identification and monitoring of these factors in future evaluations to hopefully increase our understanding of the full impact of health information technology interventions.

Problem list completeness in electronic health records: A multi-site study and assessment of success factors.

OBJECTIVE: To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness. METHODS: We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United Kingdom, and Argentina who use a variety of electronic health record systems: four self-developed and six commercial. At each site, we assessed the proportion of patients who have diabetes recorded on their problem list out of all patients with a hemoglobin A1c elevation>=7.0%, which is diagnostic of diabetes. We then conducted interviews with informatics leaders at the four highest performing sites to determine factors associated with success. Finally, we surveyed all the sites about common practices implemented at the top performing sites to determine whether there was an association between problem list management practices and problem list completeness. RESULTS: Problem list completeness across the ten sites ranged from 60.2% to 99.4%, with a mean of 78.2%. Financial incentives, problem-oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture were identified as success factors at the four hospitals with problem list completeness at or near 90.0%. DISCUSSION: Incomplete problem lists represent a global data integrity problem that could compromise quality of care and put patients at risk. There was a wide range of problem list completeness across the healthcare facilities. Nevertheless, some facilities have achieved high levels of problem list completeness, and it is important to better understand the factors that contribute to success to improve patient safety. CONCLUSION: Problem list completeness varies substantially across healthcare facilities. In our review of EHR systems at ten healthcare facilities, we identified six success factors which may be useful for healthcare organizations seeking to improve the quality of their problem list documentation: financial incentives, problem oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture.

Impacts of EHR Certification and Meaningful Use Implementation on an Integrated Delivery Network.

Three years ago Intermountain Healthcare made the decision to participate in the Medicare and Medicaid Electronic Heath Record (EHR) Incentive Program which required that hospitals and providers use a certified EHR in a meaningful way. At that time, the barriers to enhance our home grown system, and change clinician workflows were numerous and large. This paper describes the time and effort required to enhance our legacy systems in order to pass certification, including filling 47 gaps in (EHR) functionality. We also describe the processes and resources that resulted in successful changes to many clinical workflows required by clinicians to meet meaningful use requirements. In 2011 we set meaningful use targets of 75% of employed physicians and 75% of our hospitals to meet Stage 1 of meaningful use by 2013. By the end of 2013, 87% of 696 employed eligible professionals and 100% of 22 Intermountain hospitals had successfully attested for Stage 1. This paper describes documented and perceived costs to Intermountain including time, effort, resources, postponement of other projects, as well as documented and perceived benefits of attainment of meaningful use.

Problem management module: an innovative system to improve problem list workflow.

Electronic problem lists are essential to modern health record systems, with a primary goal to serve as the repository of a patient's current health issues. Additionally, coded problems can be used to drive downstream activities such as decision support, evidence-based medicine, billing, and cohort generation for research. Meaningful Use also requires use of a coded problem list. Over the course of three years, Intermountain Healthcare developed a problem management module (PMM) that provided innovative functionality to improve clinical workflow and boost problem list adoption, e.g. smart search, user customizable views, problem evolution, and problem timelines. In 23 months of clinical use, clinicians entered over 70,000 health issues, the percentage of free-text items dropped to 1.2%, completeness of problem list items increased by 14%, and more collaborative habits were initiated.

Progress and challenge in meeting meaningful use at an integrated delivery network.

Intermountain Healthcare hospitals and providers are eligible for approximately $95 million in incentives from the Health Information Technology for Economic and Clinical Health Act (HITECH), which requires that hospitals and providers use a certified electronic health record (EHR) in a meaningful way. This paper describes the our progress in readying legacy systems for certification, including measuring, and filling gaps in (EHR) functionality. Also addressed are some of the challenges and successes in meeting meaningful use. Methods for measuring and tracking levels of clinician meaningful use behaviors, and our most recent results impacting meaningful use behaviors in a large integrated delivery network are described. We identified 20 EHR requirements we can certify now, 16 requirements with minor issues to resolve, and 38 requirements which are still in some state of development. We also identified 6 meaningful use workflows that will require significant work to bring all of our hospitals and providers above the measure requirement.

Assessing readiness for meeting meaningful use: identifying electronic health record functionality and measuring levels of adoption.

With the passage, in 2009, of the Health Information Technology for Economic and Clinical Health Act (HITECH),part of the American Recovery and Reinvestment Act (ARRA), over 19 Billion dollars was targeted for healthcare information technology (HIT) projects to accelerate the adoption of electronic Health Records (EHR)s. Intermountain Healthcare facilities and providers are eligible for approximately $93 million in incentives from HITECH, if we use a "certified EHR" in a "meaningful way". This paper describes the current state of our EHR functions and EHR adoption compared to those required by the HITECH act. We describe the method used to determine the gaps between our EHR functions and EHR adoption. Our analysis identified 17 significant EHR enhancements needed to become certified and identified 42 meaningful use workflow gaps.

The use of antenatal magnesium sulfate for neuroprotection for infants born prematurely.

Cerebral palsy occurs in three to four infants per 1000 live births. Preterm birth prior to 34 weeks' gestation is a major risk factor. Five randomized controlled trials of antenatal magnesium sulfate (MgSO(4)) found a trend of reduced risk of cerebral palsy and mortality in preterm infants. Three meta-analyses using the data from the five randomized controlled trials, which included a total of 5235 prospectively evaluated fetuses, found that MgSO(4) given to women at risk of premature birth significantly reduced the risk of cerebral palsy by 30% without increasing the risk of perinatal or infant death. The implication for clinical practice is that MgSO(4) should be considered for use in patients at high risk of delivery before 34 weeks' gestation.

The role of antibiotics in the prevention of preterm birth.

THERE ARE MULTIPLE USES FOR ANTIBIOTICS DURING PREGNANCY: treatment of intercurrent bacterial infections such as urinary or respiratory tract infections, maternal treatment to prevent fetal or neonatal disease, prophylactic treatment for surgical procedures, and treatment of puerperal infections. This report will summarize the current recommendations for the use of antibiotics specifically to reduce the risk of preterm birth.

Physician use of outpatient electronic health records to improve care.

We applied a model of usage categories of electronic health records for outpatient physicians to a large population of physicians, using an established electronic health record. This model categorizes physician users according to how extensively they adopt the various capabilities of electronic health records. We identified representative indicators from usage statistics for outpatient physician use of the HELP-2 outpatient electronic medical record, in use at Intermountain Healthcare. Using these indicators, we calculated the relative proportion of users in each category. These proportions are useful for predicting the expected benefits of electronic health record adoption.

Use of ambulatory physician group clinical information by hospital-based users within an integrated delivery network.

At Intermountain Healthcare, as part of a broad information system transition plan, a proposal was made to replace the integrated ambulatory EHR, used by 550 physicians, with a new stand alone EHR. The notion leading to the proposal was that ambulatory data was infrequently accessed outside of the ambulatory setting. To test this notion, retrospective analysis was done to determine the number of ambulatory patient events accessed by hospital based users. 399 Departments from the Hospital-based group accessed 1, 984, 785 patient events that originated from within the ambulatory group in a 90 day period. This study showed that a significant number of ambulatory patient records were viewed by a wide range of hospital-based users. The decision to replace the legacy ambulatory system with a new, stand alone system was postponed. This analysis was critical in planning the road map for a new integrated clinical information system.

Physician use of electronic medical records: issues and successes with direct data entry and physician productivity.

At Intermountain Health Care, we evaluated whether physicians in an ambulatory setting will voluntarily choose to enter data directly into an electronic health record (EHR). In this paper we describe the benefits of an EHR, as they exist in the current IHC application and the ways in which we have sought to minimize obstacles to physician data entry. Currently, of 472 IHC employed physicians, 321 (68%) routinely enter some data directly into the EHR without coercion. Twenty-five percent (80/321) of the physicians use voice recognition for some data entry. Twelve of our 95 ambulatory clinics have voluntarily adopted measures to eliminate paper charts. Of the 212 physicians who entered data in 2004, sixty-nine physicians (22%) increased their level of data entry, while 12 (6%) decreased. We conclude that physicians will voluntarily adopt an EHR system, and will continue and even increase use after implementation barriers are addressed.

Using natural language processing to analyze physician modifications to data entry templates.

Efficient data entry by clinicians remains a significant challenge for electronic medical records. Current approaches have largely focused on either structured data entry, which can be limiting in expressive power, or free-text entry, which restricts the use of the data for automated decision support. Text-based templates are a semi-structured data entry method that has been used to assist physicians in manually entering clinical notes, by allowing them to edit predefined example notes. We analyzed changes made to 18,726 sentences from text templates, using a natural language processor. The most common changes were addition or deletion of normal observations, or changes in certainty. We identified common modifications that could be captured in structured form by a graphical user interface.