Effectiveness of an electronic health record-based intervention to improve follow-up of abnormal pathology results: a retrospective record analysis.

BACKGROUND AND OBJECTIVE: On March 11, 2009, the Veterans Health Administration (VA) implemented an electronic health record (EHR)-based intervention that required all pathology results to be transmitted to ordering providers by mandatory automated notifications. We examined the impact of this intervention on improving follow-up of abnormal outpatient pathology results. RESEARCH DESIGN AND SUBJECTS: We extracted pathology reports from the EHR of 2 VA sites. From 16,738 preintervention and 17,305 postintervention reports between 09/01/2008 and 09/30/2009, we randomly selected about 5% and evaluated follow-up outcomes using a standardized chart review instrument. Documented responses to the alerted report (eg, ordering follow-up tests or referrals, notifying patients, and prescribing/changing treatment) were recorded. MEASURES: Primary outcome measures included proportion of timely follow-up responses (within 30 d) and median time to direct response for abnormal reports. RESULTS: Of 816 preintervention and 798 postintervention reports reviewed, 666 (81.6%) and 688 (86.2%) were abnormal. Overall, there was no apparent intervention effect on timely follow-up (69% vs. 67.1%; P=0.4) or median time to direct response (8 vs. 8 d; P=0.7). However, logistic regression uncovered a significant intervention effect (preintervention odds ratio, 0.7; 95% confidence interval, 0.5-1.0) after accounting for site-specific differences in follow-up, with a lower likelihood of timely follow-up at one site (odds ratio, 0.4; 95% confidence interval, 0.2-0.7). CONCLUSIONS: An electronic intervention to improve test result follow-up at 2 VA institutions using the same EHR was found effective only after accounting for certain local contextual factors. Aggregating the effect of EHR interventions across different institutions and EHRs without controlling for contextual factors might underestimate their potential benefits.

The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care.

Several studies have shown that there is information loss during interruptions, and that multitasking creates higher memory load, both of which contribute to medical error. Nowhere is this more critical than in the emergency department (ED), where the emphasis of clinical decision is on the timely evaluation and stabilization of patients. This paper reports on the nature of multitasking and shift change and its implications for patient safety in an adult ED, using the methods of ethnographic observation and interviews. Data were analyzed using grounded theory to study cognition in the context of the work environment. Analysis revealed that interruptions within the ED were prevalent and diverse in nature. On average, there was an interruption every 9 and 14 min for the attending physicians and the residents, respectively. In addition, the workflow analysis showed gaps in information flow due to multitasking and shift changes. Transfer of information began at the point of hand-offs/shift changes and continued through various other activities, such as documentation, consultation, teaching activities and utilization of computer resources. The results show that the nature of the communication process in the ED is complex and cognitively taxing for the clinicians, which can compromise patient safety. The need to tailor existing generic electronic tools to support adaptive processes like multitasking and handoffs in a time-constrained environment is discussed.

Electronic health record-based triggers to detect potential delays in cancer diagnosis.

BACKGROUND: Delayed diagnosis of cancer can lead to patient harm, and strategies are needed to proactively and efficiently detect such delays in care. We aimed to develop and evaluate 'trigger' algorithms to electronically flag medical records of patients with potential delays in prostate and colorectal cancer (CRC) diagnosis. METHODS: We mined retrospective data from two large integrated health systems with comprehensive electronic health records (EHR) to iteratively develop triggers. Data mining algorithms identified all patient records with specific demographics and a lack of appropriate and timely follow-up actions on four diagnostic clues that were newly documented in the EHR: abnormal prostate-specific antigen (PSA), positive faecal occult blood test (FOBT), iron-deficiency anaemia (IDA), and haematochezia. Triggers subsequently excluded patients not needing follow-up (eg, terminal illness) or who had already received appropriate and timely care. Each of the four final triggers was applied to a test cohort, and chart reviews of randomly selected records identified by the triggers were used to calculate positive predictive values (PPV). RESULTS: The PSA trigger was applied to records of 292 587 patients seen between 1 January 2009 and 31 December 2009, and the CRC triggers were applied to 291 773 patients seen between 1 March 2009 and 28 February 2010. Overall, 1564 trigger positive patients were identified (426 PSA, 355 FOBT, 610 IDA and 173 haematochezia). Record reviews revealed PPVs of 70.2%, 66.7%, 67.5%, and 58.3% for the PSA, FOBT, IDA and haematochezia triggers, respectively. Use of all four triggers at the study sites could detect an estimated 1048 instances of delayed or missed follow-up of abnormal findings annually and 47 high-grade cancers. CONCLUSIONS: EHR-based triggers can be used successfully to flag patient records lacking follow-up of abnormal clinical findings suspicious for cancer.

Resilient actions in the diagnostic process and system performance.

OBJECTIVES: Systemic issues can adversely affect the diagnostic process. Many system-related barriers can be masked by 'resilient' actions of frontline providers (ie, actions supporting the safe delivery of care in the presence of pressures that the system cannot readily adapt to). We explored system barriers and resilient actions of primary care providers (PCPs) in the diagnostic evaluation of cancer. METHODS: We conducted a secondary data analysis of interviews of PCPs involved in diagnostic evaluation of 29 lung and colorectal cancer cases. Cases covered a range of diagnostic timeliness and were analysed to identify barriers for rapid diagnostic evaluation, and PCPs' actions involving elements of resilience addressing those barriers. We rated these actions according to whether they were usual or extraordinary for typical PCP work. RESULTS: Resilient actions and associated barriers were found in 59% of the cases, in all ranges of timeliness, with 40% involving actions rated as beyond typical. Most of the barriers were related to access to specialty services and coordination with patients. Many of the resilient actions involved using additional communication channels to solicit cooperation from other participants in the diagnostic process. DISCUSSION: Diagnostic evaluation of cancer involves several resilient actions by PCPs targeted at system deficiencies. PCPs' actions can sometimes mitigate system barriers to diagnosis, and thereby impact the sensitivity of 'downstream' measures (eg, delays) in detecting barriers. While resilient actions might enable providers to mitigate system deficiencies in the short run, they can be resource intensive and potentially unsustainable. They complement, rather than substitute for, structural remedies to improve system performance. Measures to detect and fix system performance issues targeted by these resilient actions could facilitate diagnostic safety.

Primary care practitioners' views on test result management in EHR-enabled health systems: a national survey.

CONTEXT: Failure to notify patients of test results is common even when electronic health records (EHRs) are used to report results to practitioners. We sought to understand the broad range of social and technical factors that affect test result management in an integrated EHR-based health system. METHODS: Between June and November 2010, we conducted a cross-sectional, web-based survey of all primary care practitioners (PCPs) within the Department of Veterans Affairs nationwide. Survey development was guided by a socio-technical model describing multiple inter-related dimensions of EHR use. FINDINGS: Of 5001 PCPs invited, 2590 (51.8%) responded. 55.5% believed that the EHRs did not have convenient features for notifying patients of test results. Over a third (37.9%) reported having staff support needed for notifying patients of test results. Many relied on the patient's next visit to notify them for normal (46.1%) and abnormal results (20.1%). Only 45.7% reported receiving adequate training on using the EHR notification system and 35.1% reported having an assigned contact for technical assistance with the EHR; most received help from colleagues (60.4%). A majority (85.6%) stayed after hours or came in on weekends to address notifications; less than a third reported receiving protected time (30.1%). PCPs strongly endorsed several new features to improve test result management, including better tracking and visualization of result notifications. CONCLUSIONS: Despite an advanced EHR, both social and technical challenges exist in ensuring notification of test results to practitioners and patients. Current EHR technology requires significant improvement in order to avoid similar challenges elsewhere.

Clinical Summarization Capabilities of Commercially-available and Internally-developed Electronic Health Records.

OBJECTIVE: Clinical summarization, the process by which relevant patient information is electronically summarized and presented at the point of care, is of increasing importance given the increasing volume of clinical data in electronic health record systems (EHRs). There is a paucity of research on electronic clinical summarization, including the capabilities of currently available EHR systems. METHODS: We compared different aspects of general clinical summary screens used in twelve different EHR systems using a previously described conceptual model: AORTIS (Aggregation, Organization, Reduction, Interpretation and Synthesis). RESULTS: We found a wide variation in the EHRs' summarization capabilities: all systems were capable of simple aggregation and organization of limited clinical content, but only one demonstrated an ability to synthesize information from the data. CONCLUSION: Improvement of the clinical summary screen functionality for currently available EHRs is necessary. Further research should identify strategies and methods for creating easy to use, well-designed clinical summary screens that aggregate, organize and reduce all pertinent patient information as well as provide clinical interpretations and synthesis as required.

Development and evaluation of a crowdsourcing methodology for knowledge base construction: identifying relationships between clinical problems and medications.

OBJECTIVE: We describe a novel, crowdsourcing method for generating a knowledge base of problem-medication pairs that takes advantage of manually asserted links between medications and problems. METHODS: Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem-medication links for inclusion in a knowledge base that can be used to infer previously unasserted links between problems and medications. RESULTS: Clinicians manually linked 231,223 medications (55.30% of prescribed medications) to problems within the electronic health record, generating 41,203 distinct problem-medication pairs, although not all were accurate. We developed methods to evaluate the accuracy of the pairs, and after limiting the pairs to those meeting an estimated 95% appropriateness threshold, 11,166 pairs remained. The pairs in the knowledge base accounted for 183,127 total links asserted (76.47% of all links). Retrospective application of the knowledge base linked 68,316 medications not previously linked by a clinician to an indicated problem (36.53% of unlinked medications). Expert review of the combined knowledge base, including inferred and manually linked problem-medication pairs, found a sensitivity of 65.8% and a specificity of 97.9%. CONCLUSION: Crowdsourcing is an effective, inexpensive method for generating a knowledge base of problem-medication pairs that is automatically mapped to local terminologies, up-to-date, and reflective of local prescribing practices and trends.

Repeated hemoglobin A1C ordering in the VA Health System.

BACKGROUND: Hemoglobin A1c (HbA1c) is used to assess glycemic control in patients with diabetes. While underuse of HbA1c testing has been well studied, potential overuse is poorly characterized. METHODS: Our objective was to examine the frequency of HbA1c testing in an integrated delivery system. We conducted a retrospective study of administrative data of 130,538 patients with newly diagnosed diabetes receiving care in the Veterans Administration Healthcare System during 2006 and 2007 (mean age 64.1 years, 97.3% male). Our main outcome measures were the proportion of patients receiving repeat HbA1c testing within 30 and 90 days and the proportion of patients receiving more than 4 repeat tests within 12 months of their initial HbA1c. RESULTS: Overall 8.4% of patients (N = 11,003) received at least one repeat HbA1c within 30 days of their initial test and 30.8% (N = 40,162) within 90 days. A significantly higher proportion of patients with poor diabetes control received a repeat test within 30 days (14.7%) than patients with intermediate control (9.1%) or good control (6.8%) (P < 0.01). Overall, 4.2% of patients (N = 5,468) received more than 4 repeat HbA1c tests and 0.4% received more than 6 (N = 479). In logistic regression models, receipt of more than 4 repeat HbA1c tests was more common among patients age 50-70 years (compared to younger and older patients), whites (compared to blacks and Hispanics), and patients manifesting complications of diabetes (P < 0.01 for all). CONCLUSION: Repeat HbA1c testing appears to occur somewhat more frequently than is warranted.

A prototype knowledge base and SMART app to facilitate organization of patient medications by clinical problems.

Increasing use of electronic health records requires comprehensive patient-centered views of clinical data. We describe a prototype knowledge base and SMART app that facilitates organization of patient medications by clinical problems, comprising a preliminary step in building such patient-centered views. The knowledge base includes 7,164,444 distinct problem-medication links, generated from RxNorm, SNOMED CT, and NDF-RT within the UMLS Metathesaurus. In an evaluation of the knowledge base applied to 5000 de-identified patient records, 22.4% of medications linked to an entry in the patient's active problem list, compared to 32.6% of medications manually linked by providers; 46.5% of total links were unique to the knowledge base, not added by providers. Expert review of a random patient subset estimated a sensitivity of 37.1% and specificity of 98.9%. The SMART API successfully utilized the knowledge base to generate problem-medication links for test patients. Future work is necessary to improve knowledge base sensitivity and efficiency.

Designing the design phase of critical care devices: a cognitive approach.

In this study, we show how medical devices used for patient care can be made safer if various cognitive factors involved in patient management are taken into consideration during the design phase. The objective of this paper is to describe a methodology for obtaining insights into patient safety features--derived from investigations of institutional decision making--that could be incorporated into medical devices by their designers. The design cycle of a product, be it a medical device, software, or any kind of equipment, is similar in concept, and course. Through a series of steps we obtained information related to medical errors and patient safety. These were then utilized to customize the generic device design cycle in ways that would improve the production of critical care devices. First, we provided individuals with different levels of expertise in the clinical, administrative, and engineering domains of a large hospital setting with hypothetical clinical scenarios, each of which described a medical error event involving health professionals and medical devices. Then, we asked our subjects to "think-aloud" as they read through each scenario. Using a set of questions as probes, we then asked our subjects to identify key errors and attribute them to various players. We recorded and transcribed the responses and conducted a cognitive task analysis of each scenario to identify different entities as "constant," "partially modifiable," or "modifiable." We compared our subjects' responses to the results of the task analysis and then mapped them to the modifiable entities. Lastly, we coded the relationships of these entities to the errors in medical devices. We propose that the incorporation of these modifiable entities into the device design cycle could improve the device end product for better patient safety management.

Decisions about critical events in device-related scenarios as a function of expertise.

This paper presents the perspectives of personnel involved in decision-making about devices in critical care. We use the concept of "sharp and blunt ends" of practice to describe the performance of health care professionals. The "sharp end" is physically and temporally close to the system; the "blunt end" is removed from the system in time and space and yet affects the system through indirect influence on the sharp end. In this study, the sharp end is represented by the clinicians (nurses and doctors) and the blunt end by the administrators and biomedical engineers. These subjects represent the professionals involved in the decision-making process for purchasing biomedical equipment for the hospital. They were asked to "think aloud" while evaluating three error scenarios based on real events. The responses were recorded and transcribed for analysis. The results show differences in interpretation of critical events as a function of professional expertise. The clinicians (sharp-end practitioners) focused on clinical and human aspect of errors while the biomedical engineers focused on device-related errors. The administrators focused on documentation and training. These different interpretations mean that the problems are represented differently by these groups of subjects, and these representations result in variable decisions about devices. These results are discussed within a systems approach framework to help us assess the completeness of the problem representations of the subjects, their awareness of critical events, and how these events would collectively contribute to the occurrence of error.