Familiality of diabetes mellitus.

The aims of this study were to estimate relative risk for type 1 and type 2 diabetes in relatives of diabetic patients, and to test for excess relatedness among diabetic patients. Additionally, the difference in parental transmission of diabetes was investigated. This study used a unique Utah genealogical resource, linked to electronic medical records of the largest health provider in Utah. We identified 19,640 patients with a diagnosis of type 1 or type 2 diabetes. Relative Risks (RRs) for type 1 and type 2 diabetes were assessed for first-, second- and third-degree relatives of diabetic patients. The observed average relatedness of diabetic patients was compared to the expected relatedness using the Genealogical Index of Familiality (GIF). We observed significantly elevated RRs for type 1 diabetes in first-degree (RR=8.68; P<0.0001), second-degree (RR=1.93; P<0.0001) and third-degree relatives (RR=1.74; P<0.0001) of type 1 diabetic patients. RRs for type 2 diabetes were significantly increased in first-degree (RR=2.24; P<0.0001), second-degree (RR=1.36; P<0.0001) and third-degree relatives (RR=1.14; P<0.0001) of type 2 diabetic patients. Significantly increased RRs for type 1 diabetes were observed in the relatives of type 2 diabetic patients, and vice versa. The GIF analysis showed significant excess relatedness for type 1 diabetes cases, and independently for type 2 diabetes cases. Offspring of diabetic fathers were at significantly higher risk for type 1 diabetes than offspring of diabetic mothers (RR=9.73; P<0.0001 compared to RR=4.99; P<0.0001). No significant difference in parental transmission was observed for type 2 diabetes. Our results strongly support the existence of a genetic contribution to both type 1 and type 2 diabetes, and additionally suggest a relationship between both types of diabetes. Furthermore, our results suggest a significant difference in parental transmission of type 1 diabetes.

Building a comprehensive clinical information system from components. The approach at Intermountain Health Care.

OBJECTIVES: To discuss the advantages and disadvantages of an interfaced approach to clinical information systems architecture. METHODS: After many years of internally building almost all components of a hospital clinical information system (HELP) at Intermountain Health Care, we changed our architectural approach as we chose to encompass ambulatory as well as acute care. We now seek to interface applications from a variety of sources (including some that we build ourselves) to a clinical data repository that contains a longitudinal electronic patient record. RESULTS: We have a total of 820 instances of interfaces to 51 different applications. We process nearly 2 million transactions per day via our interface engine and feel that the reliability of the approach is acceptable. Interface costs constitute about four percent of our total information systems budget. The clinical database currently contains records for 1.45 m patients and the response time for a query is 0.19 sec. DISCUSSION: Based upon our experience with both integrated (monolithic) and interfaced approaches, we conclude that for those with the expertise and resources to do so, the interfaced approach offers an attractive alternative to systems provided by a single vendor. We expect the advantages of this approach to increase as the costs of interfaces are reduced in the future as standards for vocabulary and messaging become increasingly mature and functional.

A comprehensive set of coded chief complaints for the emergency department.

OBJECTIVE: To develop a generally applicable set of coded chief complaints for the computerized patient records of emergency departments (EDs). METHODS: At an urban teaching ED the chief complaints of more than 50,000 patients were analyzed retrospectively during a 29-month period (June 1995-October 1997). Applying continuous quality improvement methods, a multidisciplinary team examined the current process documenting the patient's chief complaint. During two prospective periods (November 1997-December 1998; January 1999-June 1999), more than 34,000 chief complaints were analyzed. To reduce free-text charting practices, a variety of interventions on individual and team level were applied. Quantitative analysis was performed with statistical process control charts, and a qualitative evaluation was performed with a questionnaire. RESULTS: The charting of chief complaint in free-text format decreased from 23% to 1%. The range among individual ED staff members narrowed from 45% to 9%. During the refinement of the set of coded chief complaints, six infrequently charted items were removed. Five new chief complaints identified by analysis of free-text entries during the second study period were added. The current set of chief complaints consists of 54 codable and the three original free-text items. The ED staff members perceived all the interventions beneficial. A poster displaying all available terms as a visual aid, however, had the largest impact on charting the patient's chief complaint in coded format. CONCLUSIONS: Applying continuous quality improvement methods, the authors created a clinically developed and applicable set of codable chief complaints that can be easily integrated into a computerized patient record of an ED.

Automatic identification of patients eligible for a pneumonia guideline: comparing the diagnostic accuracy of two decision support models.

BACKGROUND: In busy clinical settings, physicians often do not have enough time to identify patients for specific therapeutic guidelines. As a solution, decision support systems could automatically identify eligible patients and trigger computerized guidelines for specific diseases. Applying this idea to community-acquired pneumonia (CAP), we developed a Bayesian network (BN) and an artificial neural network (ANN) for identifying patients who have CAP and are eligible for a pneumonia guideline. OBJECTIVE: The aim of this study was to determine whether the diagnostic accuracy of these two decision support models differs in terms of identifying CAP patients. METHODS: We trained and tested the networks with a data set of 32,662 adult patients. For each network, we (1) calculated the specificity, the positive predictive value (PPV), and the negative predictive value (NPV) at a sensitivity of 95%, and (2) determined the area under the receiver operating characteristic curve (AUC) as a measure of overall accuracy. We tested for statistical difference between the AUCs using the correlated area z statistic. RESULTS: At a sensitivity of 95%, the respective values for specificity, PPV, and NPV were: 92.3%, 15.1%, and 99.9% for the BN, and 94.0%, 18.6%, and 99.9% for the ANN. The BN had an AUC of 0.9795 (95% CI: 0.9736, 0.9843), and the ANN had an AUC of 0.9855 (95% CI: 0.9805, 0.9894). The difference between the AUCs was statistically significant (p=0.0044). CONCLUSIONS: The networks achieved high overall accuracies on the testing data set. Because the difference in accuracies is statistically significant but not clinically significant, both networks are equally suited to drive a guideline.

Evaluation of a computerized diagnostic decision support system for patients with pneumonia: study design considerations.

Planning the clinical evaluation of a computerized decision support system requires a strategy that encompasses the different aspects of the clinical problem, the technical difficulties of software and hardware integration and implementation, the behavioral aspects of the targeted users, and the discipline of study design. Although clinical information systems are becoming more widely available, only a few decision support systems have been formally evaluated in clinical environments. Published accounts of difficulties associated with the clinical evaluation of decision support systems remain scarce. The authors report on a variety of behavioral, logistical, technical, clinical, cost, and work flow issues that they had to address when choosing a study design for a clinical trial for the evaluation of an integrated, real-time decision support system for the automatic identification of patients likely to have pneumonia in an emergency department. In the absence of a true gold standard, they show how they created a credible, clinically acceptable, and economical reference standard for the diagnosis of pneumonia, to determine the overall accuracy of the system. For the creation of a reference standard, they describe the importance of recognizing verification bias and avoiding it. Finally, advantages and disadvantages of different study designs are explored with respect to the targeted users and the clinical setting.

A comparison of classification algorithms to automatically identify chest X-ray reports that support pneumonia.

We compared the performance of expert-crafted rules, a Bayesian network, and a decision tree at automatically identifying chest X-ray reports that support acute bacterial pneumonia. We randomly selected 292 chest X-ray reports, 75 (25%) of which were from patients with a hospital discharge diagnosis of bacterial pneumonia. The reports were encoded by our natural language processor and then manually corrected for mistakes. The encoded observations were analyzed by three expert systems to determine whether the reports supported pneumonia. The reference standard for radiologic support of pneumonia was the majority vote of three physicians. We compared (a) the performance of the expert systems against each other and (b) the performance of the expert systems against that of four physicians who were not part of the gold standard. Output from the expert systems and the physicians was transformed so that comparisons could be made with both binary and probabilistic output. Metrics of comparison for binary output were sensitivity (sens), precision (prec), and specificity (spec). The metric of comparison for probabilistic output was the area under the receiver operator characteristic (ROC) curve. We used McNemar's test to determine statistical significance for binary output and univariate z-tests for probabilistic output. Measures of performance of the expert systems for binary (probabilistic) output were as follows: Rules--sens, 0.92; prec, 0.80; spec, 0.86 (Az, 0.960); Bayesian network--sens, 0.90; prec, 0.72; spec, 0.78 (Az, 0.945); decision tree--sens, 0.86; prec, 0.85; spec, 0.91 (Az, 0.940). Comparisons of the expert systems against each other using binary output showed a significant difference between the rules and the Bayesian network and between the decision tree and the Bayesian network. Comparisons of expert systems using probabilistic output showed no significant differences. Comparisons of binary output against physicians showed differences between the Bayesian network and two physicians. Comparisons of probabilistic output against physicians showed a difference between the decision tree and one physician. The expert systems performed similarly for the probabilistic output but differed in measures of sensitivity, precision, and specificity produced by the binary output. All three expert systems performed similarly to physicians.

Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia.

RATIONALE AND OBJECTIVES: The purpose of this study was to statistically identify some characteristics of unambiguous (ie, clear) chest radiography reports in the context of acute bacterial pneumonia. MATERIALS AND METHODS: Seven physicians individually read 292 chest radiography reports to determine if they contained radiologic evidence of pneumonia. Unambiguous reports were defined as those that physicians unanimously classified as supporting or not supporting the diagnosis of pneumonia. Ambiguous reports were assigned degrees of ambiguity on the basis of how much disagreement they caused among the physicians. Characteristics of unambiguous reports as described in the literature were manually quantified and assigned to every report. To identify characteristics that statistically distinguished unambiguous from ambiguous reports, the authors performed an ordinal logistic regression analysis for which the dependent variable was the number of dissenting votes the report received and the independent variables were the quantified characteristics of the report. RESULTS: Six independent variables were statistically significantly associated with unambiguous reports (P < .05). Three were positively associated: an interpretation of whether findings supported the diagnosis of pneumonia in reports with pneumonia-related observations, short sentences, and redundancy of pneumonia-related observations. Three were negatively associated: high use of uncertainty modifiers for pneumonia-related observations, use of only descriptive terms to describe pneumonia-related observations, and insufficient amount of pneumonia-related information. CONCLUSION: The most influential characteristic of an unambiguous chest radiography report was an interpretation of whether the radiograph supported the diagnosis of pneumonia when findings could be indicative.

Combining decision support methodologies to diagnose pneumonia.

OBJECTIVE: To evaluate the performance of a computerized decision support system that combines two different decision support methodologies (a Bayesian network and a natural language understanding system) for the diagnosis of patients with pneumonia. DESIGN: Evaluation study using data from a prospective, clinical study. PATIENTS: All patients 18 years and older who presented to the emergency department of a tertiary care setting and whose chest x-ray report was available during the encounter. METHODS: The computerized decision support system calculated a probability of pneumonia using information provided by the two systems. Outcome measures were the area under the receiver operating characteristic curve, sensitivity, specificity, predictive values, likelihood ratios, and test effectiveness. RESULTS: During the 3-month study period there were 742 patients (45 with pneumonia). The area under the receiver operating characteristic curve was 0.881 (95% CI: 0.822, 0.925) for the Bayesian network alone and 0.916 (95% CI: 0.869, 0.949) for the Bayesian network combined with the natural language understanding system (p=0.01). CONCLUSION: Combining decision support methodologies that process information stored in different data formats can increase the performance of a computerized decision support system.

Automatic detection of acute bacterial pneumonia from chest X-ray reports.

OBJECTIVE: To evaluate the performance of a natural language processing system in extracting pneumonia-related concepts from chest x-ray reports. DESIGN: Four physicians, three lay persons, a natural language processing system, and two keyword searches (designated AAKS and KS) detected the presence or absence of three pneumonia-related concepts and inferred the presence or absence of acute bacterial pneumonia from 292 chest x-ray reports. Gold standard: Majority vote of three independent physicians. Reliability of the gold standard was measured. OUTCOME MEASURES: Recall, precision, specificity, and agreement (using Finn's R: statistic) with respect to the gold standard. Differences between the physicians and the other subjects were tested using the McNemar test for each pneumonia concept and for the disease inference of acute bacterial pneumonia. RESULTS: Reliability of the reference standard ranged from 0.86 to 0.96. Recall, precision, specificity, and agreement (Finn R:) for the inference on acute bacterial pneumonia were, respectively, 0.94, 0.87, 0.91, and 0.84 for physicians; 0.95, 0.78, 0.85, and 0.75 for natural language processing system; 0.46, 0.89, 0.95, and 0.54 for lay persons; 0.79, 0.63, 0.71, and 0.49 for AAKS; and 0.87, 0.70, 0.77, and 0.62 for KS. The McNemar pairwise comparisons showed differences between one physician and the natural language processing system for the infiltrate concept and between another physician and the natural language processing system for the inference on acute bacterial pneumonia. The comparisons also showed that most physicians were significantly different from the other subjects in all pneumonia concepts and the disease inference. CONCLUSION: In extracting pneumonia related concepts from chest x-ray reports, the performance of the natural language processing system was similar to that of physicians and better than that of lay persons and keyword searches. The encoded pneumonia information has the potential to support several pneumonia-related applications used in our institution. The applications include a decision support system called the antibiotic assistant, a computerized clinical protocol for pneumonia, and a quality assurance application in the radiology department.

Automatic identification of patients eligible for a pneumonia guideline.

OBJECTIVE: To assess the ability of an integrated, real-time diagnostic system (Bayesian network) to identify patients with community-acquired pneumonia who are eligible for a computerized pneumonia guideline without requiring clinicians to enter additional data. DESIGN: Prospective validation study. PATIENTS: All patients 18 years and older who presented to the emergency department of a tertiary care hospital. METHODS: The diagnostic system computed a probability of pneumonia for every patient. The final diagnosis was established using ICD-9 discharge diagnoses. Outcome measures were sensitivity, specificity, predictive values, likelihood ratios, area under the receiver operating characteristic curve, and test effectiveness. RESULTS: During the 9-week study period there were 4,361 patients (112 pneumonia patients). The area under the receiver operating characteristic curve was 0.930 (CI: 0.907, 0.948). At a fixed sensitivity of 95%, the specificity was 68.5%, the positive predictive value 7.3%, the negative predictive value 99.8%, the positive likelihood ratio 3.0, the negative likelihood ratio 0.08, and the test effectiveness 2.05. CONCLUSION: The diagnostic system was able to detect patients who are eligible for a pneumonia guideline. The detection of eligible patients can be applied to automatically initiate and evaluate computerized guidelines.

Contribution of a speech recognition system to a computerized pneumonia guideline in the emergency department.

OBJECTIVE: Evaluate the effect of a radiology speech recognition system on a real-time computerized guideline in the emergency department. METHODS: We collected all chest x-ray reports (n = 727) generated for patients in the emergency department during a six-week period. We divided the concurrently generated reports into those generated with speech recognition and those generated by traditional dictation. We compared the two sets of reports for availability during the patient's emergency department encounter and for readability. RESULTS: Reports generated by speech recognition were available seven times more often during the patients' encounters than reports generated by traditional dictation. Using speech recognition reduced the turnover time of reports from 12 hours 33 minutes to 2 hours 13 minutes. Readability scores were identical for both kinds of reports. CONCLUSION: Using speech recognition to generate chest x-ray reports reduces turnover time so reports are available while patients are in the emergency department.

Using medical language processing to support real-time evaluation of pneumonia guidelines.

OBJECTIVE: To evaluate if a medical language processing (MLP) system is able to support real-time computerization of community-acquired pneumonia (CAP) guidelines. METHODS: Prospective validation study in the emergency department of a tertiary care facility. All the chest x-ray reports available in real-time for an admission decision during a five-week period were included. The MLP system was compared to a physician for the automatic selection of eligible patients and on the extraction of radiographic findings required by five different CAP guidelines. The gold standard comprised of three independent physicians and reliability measures were calculated. The outcome measures were the area under the receiver operated characteristic curve (AUC) for selecting eligible patients, sensitivity, positive predictive value (PPV), and specificity for the extraction of radiographic findings. RESULTS: During the five-week period, 243 reports were available in real-time. The AUCs on selecting eligible CAP patients were 89.7% (CI: 84.2%, 93.7%) for the MLP system, and 93.3% (CI: 83.9%, 97.8%) for the physician. The average sensitivity, PPV, and specificity for radiographic findings that assessed localization and extension of CAP were respectively: 94%, 87%, 96% (physician); and 34%, 90%, 95% (MLP system). Both, the MLP system and the physician had average sensitivity, PPV, and specificity of 97%, 97%, and 99%, respectively, when localization was not an issue. Reliability measures for the gold standard were above 70%. CONCLUSION: The MLP system was able to support real-time computerization of guidelines by selecting eligible patients and extracting radiographic findings that do not assess localization and extension of CAP.

Assessing the quality of clinical data in a computer-based record for calculating the pneumonia severity index.

OBJECTIVE: This study examined whether clinical data routinely available in a computerized patient record (CPR) can be used to drive a complex guideline that supports physicians in real time and at the point of care in assessing the risk of mortality for patients with community-acquired pneumonia. SETTING: Emergency department of a tertiary-care hospital. DESIGN: Retrospective analysis with medical chart review. PATIENTS: All 241 inpatients during a 17-month period (Jun 1995 to Nov 1996) who presented to the emergency department and had a primary discharge diagnosis of community-acquired pneumonia. METHODS/MAIN OUTCOME MEASURES: The 20 guideline variables were extracted from the CPR (HELP System) and the paper chart. The risk score and the risk class of the Pneumonia Severity Index were computed using data from the CPR alone and from a reference standard of all data available in the paper chart and the CPR at the time of the emergency department encounters. Availability and concordance were quantified to determine data quality. The type and cause of errors were analyzed depending on the source and format of the clinical variables. RESULTS: Of the 20 guideline variables, 12 variables were required to be present for every computer-charted emergency department patient, seven variables were required for selected patients only, and one variable was not typically available in the HELP System during a patient's encounter. The risk class was identical for 86.7 percent of the patients. The majority of patients with different risk classes were assigned too low a risk class. The risk scores were identical for 72.1 percent of the patients. The average availability was 0.99 for the data elements that were required to be present and 0.79 for the data elements that were not required to be present. The average concordance was 0.98 when all a patient's variables were taken into account. The cause of error was attributed to the nurse charting in 77 percent of the cases and to the computerized evaluation in 23 percent. The type of error originated from the free-text fields in 64 percent, from coded fields in 21 percent, from vital signs in 14 percent, and from laboratory results in 1 percent. CONCLUSION: From a clinical perspective, the current level of data quality in the HELP System supports the automation and the prospective evaluation of the Pneumonia Severity Index as a computerized decision support tool.

Automatic identification of pneumonia related concepts on chest x-ray reports.

A medical language processing system called SymText, two other automated methods, and a lay person were compared against an internal medicine resident for their ability to identify pneumonia related concepts on chest x-ray reports. Sensitivity (recall), specificity, and positive predictive value (precision) are reported with respect to an independent panel of physicians. Overall the performance of SymText was similar to the physician and superior to the other methods. The automatic encoding of pneumonia concepts will support clinical research, decision making, computerized clinical protocols, and quality assurance in a radiology department.

An integrated decision support system for diagnosing and managing patients with community-acquired pneumonia.

Decision support systems that integrate guidelines have become popular applications to reduce variation and deliver cost-effective care. However, adverse characteristics of decision support systems, such as additional and time-consuming data entry or manually identifying eligible patients, result in a "behavioral bottleneck" that prevents decision support systems to become part of the clinical routine. This paper describes the design and the implementation of an integrated decision support system that explores a novel approach for bypassing the behavioral bottleneck. The real-time decision support system does not require health care providers to enter additional data and consists of a diagnostic and a management component.

Comparing expert systems for identifying chest x-ray reports that support pneumonia.

We compare the performance of four computerized methods in identifying chest x-ray reports that support acute bacterial pneumonia. Two of the computerized techniques are constructed from expert knowledge, and two learn rules and structure from data. The two machine learning systems perform as well as the expert constructed systems. All of the computerized techniques perform better than a baseline keyword search and a lay person, and perform as well as a physician. We conclude that machine learning can be used to identify chest x-ray reports that support pneumonia.

Evaluation of a semantic data model for chest radiology: application of a new methodology.

An essential step toward the effective processing of the medical language is the development of representational models that formalize the language semantics. These models, also known as semantic data models, help to unlock the meaning of descriptive expressions, making them accessible to computer systems. The present study tries to determine the quality of a semantic data model created to encode chest radiology findings. The evaluation methodology relied on the ability of physicians to extract information from textual and encoded representations of chest X-ray reports, whilst answering questions associated with each report. The evaluation demonstrated that the encoded reports seemed to have the same information content of the original textual reports. The methodology generated useful data regarding the quality of the data model, demonstrating that certain segments were creating ambiguous representations and that some details were not being represented.

Bayesian modeling for linking causally related observations in chest X-ray reports.

Our natural language understanding system outputs a list of diseases, findings, and appliances found in a chest x-ray report. The system described in this paper links those diseases and findings that are causally related. Using Bayesian networks to model the conceptual and diagnostic information found in a chest x-ray we are able to infer more specific information about the findings that are linked to diseases.

Diagnosing community-acquired pneumonia with a Bayesian network.

We present the development and the evaluation of a Bayesian network for the diagnosis of community-acquired pneumonia. The Bayesian network is intended to be part of a larger decision support system which assists emergency room physicians in the management of pneumonia patients. Minimal data entry from the nurse or the physician, timely availability of clinical parameters, and high accuracy were requirements we tried to meet. Data from more than 32,000 emergency room patients over a period of 2 years (June 1995-June 1997) were extracted from the clinical information system to train and test the Bayesian network. The network performed well in discriminating patients with pneumonia from patients with other diseases. The Bayesian network achieved a sensitivity of 95%, a specificity of 96.5%, an area under the receiver operating characteristic of 0.98, and a predictive value positive of 26.8%. Our feasibility study demonstrates that the proposed Bayesian network is an appropriate method to detect pneumonia patients with high accuracy. The study suggests that the proposed Bayesian network may represent a successful component within a larger decision support system for the management of community-acquired pneumonia.

Automatic extraction of PIOPED interpretations from ventilation/perfusion lung scan reports.

Free-text documents are the main type of data produced by a radiology department in a hospital information system. While this type of data is readily accessible for clinical data review it can not be accessed by other applications to perform medical decision support, quality assurance, and outcome studies. In an attempt to solve this problem, natural language processing systems have been developed and tested against chest x-rays reports to extract relevant clinical information and make it accessible to other computer applications. We have used a natural language processing tool called SymText to extract relevant clinical information from a different type of radiology report, the Ventilation/Perfusion lung scan report. Results of this effort can be analyzed in terms of precision and recall. The overall precision was 0.88 and recall was 0.92. In addition, the natural language processing system functions differently in reports with and without an impression section. If this type of information can be successfully extracted from radiology reports, one can develop quality monitors for the diagnostic performance of the radiologist by correlating the impressions with gold standard data present in a hospital information system. Avoiding the manual effort previously necessary to create quality assurance data, can lead to a higher frequency of quality review in a radiology department.