Extracting information from textual documents in the electronic health record: a review of recent research.

OBJECTIVES: We examine recent published research on the extraction of information from textual documents in the Electronic Health Record (EHR). METHODS: Literature review of the research published after 1995, based on PubMed, conference proceedings, and the ACM Digital Library, as well as on relevant publications referenced in papers already included. RESULTS: 174 publications were selected and are discussed in this review in terms of methods used, pre-processing of textual documents, contextual features detection and analysis, extraction of information in general, extraction of codes and of information for decision-support and enrichment of the EHR, information extraction for surveillance, research, automated terminology management, and data mining, and de-identification of clinical text. CONCLUSIONS: Performance of information extraction systems with clinical text has improved since the last systematic review in 1995, but they are still rarely applied outside of the laboratory they have been developed in. Competitive challenges for information extraction from clinical text, along with the availability of annotated clinical text corpora, and further improvements in system performance are important factors to stimulate advances in this field and to increase the acceptance and usage of these systems in concrete clinical and biomedical research contexts.

Assessing the difficulty and time cost of de-identification in clinical narratives.

OBJECTIVE: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. METHODS: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the "easy" PHI that are number- or date-based. RESULTS: From 262 notes, 2074 PHI, or 7.9 +/- 6.1 per note, were found. The average recall (or sensitivity) was 95.9% while precision was 99.6% for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5%) but worse precision (88.4%) for its subset of identifiers. Manually de-identifying a note took 87.3 +/- 61 seconds on average. CONCLUSIONS: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.

Use of verbal protocol analysis for identification of ADE signals.

We describe the use of verbal protocol analysis for evaluating the textual signals used by pharmacists for detection of adverse drug events (ADEs). "Think aloud" technique was used to gain insight into how pharmacists reason about ADE occurrence, when reading patient progress notes. We used case-scenarios for five ADEs consisting of information regarding patient history, medications, laboratory results, vital signs and patient progress notes. Pharmacists extensively used information present in the progress notes to make inferences about ADEs.

Direct text entry in electronic progress notes. An evaluation of input errors.

OBJECTIVES: It is not uncommon that the introduction of a new technology fixes old problems while introducing new ones. The Veterans Administration recently implemented a comprehensive electronic medical record system (CPRS) to support provider order entry. Progress notes are entered directly by clinicians, primarily through keyboard input. Due to concerns that there may be significant, invisible disruptions to information flow, this study was conducted to formally examine the incidence and characteristics of input errors in the electronic patient record. METHODS: Sixty patient charts were randomly selected from all 2,301 inpatient admissions during a 5-month period. A panel of clinicians with informatics backgrounds developed the review criteria. After establishing inter-rater reliability, two raters independently reviewed 1,891 notes for copying, copying errors, inconsistent text, inappropriate object insertion and signature issues. RESULTS: Overall, 60% of patients reviewed had one or more input-related errors averaging 7.8 errors per patient. About 20% of notes showed evidence of copying, with an average of 1.01 error per copied note. Copying another clinician's note and making changes had the highest risk of error. Templating resulted in large amounts of blank spaces. Overall, MDs make more errors than other clinicians even after controlling for the number of notes. CONCLUSIONS: Moving towards a more progressive model for the electronic medical record, where actions are recorded only once, history and physical information is encoded for use later, and note generation is organized around problems, would greatly minimize the potential for error.

Developing a taxonomy for research in adverse drug events: potholes and signposts.

Computerized decision support and order entry shows great promise for reducing adverse drug events (ADEs). The evaluation of these solutions depends on a framework of definitions and classifications that is clear and practical. Unfortunately the literature does not always provide a clear path to defining and classifying adverse drug events. While not a systematic review, this paper uses examples from the literature to illustrate problems that investigators will confront as they develop a conceptual framework for their research. It also proposes a targeted taxonomy that can facilitate a clear and consistent approach to the research of ADEs and aid in the comparison to results of past and future studies. The taxonomy addresses the definition of ADE, types, seriousness, error, and causality.

Interventions for disruptive behaviors. Use and success.

Health care providers deal with disruptions from geriatric patients routinely. Despite the negative impact on provider efficiency, provider-patient relations, and patient well-being, there have been no systematic clinical studies of the impact of disruptive behaviors on geriatric inpatient care. This article presents a taxonomy for these behaviors, applying them to a study of disruptive behaviors and concomitant nursing interventions on a geriatric evaluation and management (GEM) unit. The sample, consisting of 23 nursing staff (16 RNs, 4 LPNs, and 3 nurse aides), was followed over 8 weeks (five shifts per week, distributed randomly over day, evening, and night shifts). An experienced pair of RN observers logged all disruptive behaviors and the associated interventions employed by the nursing providers. The taxonomy was validated on 97 disruptive events (113 disruptive behaviors) initiated by 87 patients. The major findings of the study were: (a) disruptive behaviors are common on a GEM unit; (b) behaviors that disrupt care are recognized only 50% of the time by nursing staff; (c) interventions, when used singly, were found successful 45% of the time; (d) multiple simultaneous interventions may be more successful than single interventions but were used in only 16% of cases; and (e) selection of interventions may be associated with staff education level.

Lightweight fuzzy processes in clinical computing.

In spite of advances in computing hardware, many hospitals still have a hard time finding extra capacity in their production clinical information system to run artificial intelligence (AI) modules, for example: to support real-time drug-drug or drug-lab interactions; to track infection trends; to monitor compliance with case specific clinical guidelines; or to monitor/ control biomedical devices like an intelligent ventilator. Historically, adding AI functionality was not a major design concern when a typical clinical system is originally specified. AI technology is usually retrofitted 'on top of the old system' or 'run off line' in tandem with the old system to ensure that the routine work load would still get done (with as little impact from the AI side as possible). To compound the burden on system performance, most institutions have witnessed a long and increasing trend for intramural and extramural reporting, (e.g. the collection of data for a quality-control report in microbiology, or a meta-analysis of a suite of coronary artery bypass grafts techniques, etc.) and these place an ever-growing burden on typical the computer system's performance. We discuss a promising approach to adding extra AI processing power to a heavily-used system based on the notion 'lightweight fuzzy processing (LFP)', that is, fuzzy modules designed from the outset to impose a small computational load. A formal model for a useful subclass of fuzzy systems is defined below and is used as a framework for the automated generation of LFPs. By seeking to reduce the arithmetic complexity of the model (a hand-crafted process) and the data complexity of the model (an automated process), we show how LFPs can be generated for three sample datasets of clinical relevance.

Inter-rater reliability and review of the VA unresolved narratives.

To better understand how VA clinicians use medical vocabulary in every day practice, we set out to characterize terms generated in the Problem List module of the VA's DHCP system that were not mapped to terms in the controlled-vocabulary lexicon of DHCP. When entered terms fail to match those in the lexicon, a note is sent to a central repository. When our study started, the volume in that repository had reached 16,783 terms. We wished to characterize the potential reasons why these terms failed to match terms in the lexicon. After examining two small samples of randomly selected terms, we used group consensus to develop a set of rating criteria and a rating form. To be sure that the results of multiple reviewers could be confidently compared, we analyzed the inter-rater agreement of our rating process. Two rates used this form to rate the same 400 terms. We found that modifiers and numeric data were common and consistent reasons for failure to match, while others such as use of synonyms and absence of the concept from the lexicon were common but less consistently selected.