Toward enhanced pharmacovigilance using patient-generated data on the internet.

The promise of augmenting pharmacovigilance with patient-generated data drawn from the Internet was called out by a scientific committee charged with conducting a review of the current and planned pharmacovigilance practices of the US Food and Drug Administration (FDA). To this end, we present a study on harnessing behavioral data drawn from Internet search logs to detect adverse drug reactions (ADRs). By analyzing search queries collected from 80 million consenting users and by using a widely recognized benchmark of ADRs, we found that the performance of ADR detection via search logs is comparable and complementary to detection based on the FDA's adverse event reporting system (AERS). We show that by jointly leveraging data from the AERS and search logs, the accuracy of ADR detection can be improved by 19% relative to the use of each data source independently. The results suggest that leveraging nontraditional sources such as online search logs could supplement existing pharmacovigilance approaches.

Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system.

Signal-detection algorithms (SDAs) are recognized as vital tools in pharmacovigilance. However, their performance characteristics are generally unknown. By leveraging a unique gold standard recently made public by the Observational Medical Outcomes Partnership (OMOP) and by conducting a unique systematic evaluation, we provide new insights into the diagnostic potential and characteristics of SDAs that are routinely applied to the US Food and Drug Administration (FDA) Adverse Event Reporting System (AERS). We find that SDAs can attain reasonable predictive accuracy in signaling adverse events. Two performance classes emerge, indicating that the class of approaches that address confounding and masking effects benefits safety surveillance. Our study shows that not all events are equally detectable, suggesting that specific events might be monitored more effectively using other data sources. We provide performance guidelines for several operating scenarios to inform the trade-off between sensitivity and specificity for specific use cases. We also propose an approach and demonstrate its application in identifying optimal signaling thresholds, given specific misclassification tolerances.

Novel data-mining methodologies for adverse drug event discovery and analysis.

An important goal of the health system is to identify new adverse drug events (ADEs) in the postapproval period. Datamining methods that can transform data into meaningful knowledge to inform patient safety have proven essential for this purpose. New opportunities have emerged to harness data sources that have not been used within the traditional framework. This article provides an overview of recent methodological innovations and data sources used to support ADE discovery and analysis.

Novel statistical tools for monitoring the safety of marketed drugs.

Robust tools for monitoring the safety of marketed therapeutic products are of paramount importance to public health. In recent years, innovative statistical approaches have been developed to screen large post-marketing safety databases for adverse events (AEs) that occur with disproportionate frequency. These methods, known variously as quantitative signal detection, disproportionality analysis, or safety data mining, facilitate the identification of new safety issues or possible harmful effects of a product. In this article, we describe the statistical concepts behind these methods, as well as their practical application to monitoring the safety of pharmaceutical products using spontaneous AE reports. We also provide examples of how these tools can be used to identify novel drug interactions and demographic risk factors for adverse drug reactions. Challenges, controversies, and frontiers for future research are discussed.

Transplantation of embryonic dopamine neurons for severe Parkinson's disease.

BACKGROUND: Transplantation of human embryonic dopamine neurons into the brains of patients with Parkinson's disease has proved beneficial in open clinical trials. However, whether this intervention would be more effective than sham surgery in a controlled trial is not known. METHODS: We randomly assigned 40 patients who were 34 to 75 years of age and had severe Parkinson's disease (mean duration, 14 years) to receive a transplant of nerve cells or sham surgery; all were to be followed in a double-blind manner for one year. In the transplant recipients, cultured mesencephalic tissue from four embryos was implanted into the putamen bilaterally. In the patients who received sham surgery, holes were drilled in the skull but the dura was not penetrated. The primary outcome was a subjective global rating of the change in the severity of disease, scored on a scale of -3.0 to 3.0 at one year, with negative scores indicating a worsening of symptoms and positive scores an improvement. RESULTS: The mean (+/-SD) scores on the global rating scale for improvement or deterioration at one year were 0.0+/-2.1 in the transplantation group and -0.4+/-1.7 in the sham-surgery group. Among younger patients (60 years old or younger), standardized tests of Parkinson's disease revealed significant improvement in the transplantation group as compared with the sham-surgery group when patients were tested in the morning before receiving medication (P=0.01 for scores on the Unified Parkinson's Disease Rating Scale; P=0.006 for the Schwab and England score). There was no significant improvement in older patients in the transplantation group. Fiber outgrowth from the transplanted neurons was detected in 17 of the 20 patients in the transplantation group, as indicated by an increase in 18F-fluorodopa uptake on positron-emission tomography or postmortem examination. After improvement in the first year, dystonia and dyskinesias recurred in 15 percent of the patients who received transplants, even after reduction or discontinuation of the dose of levodopa. CONCLUSIONS: Human embryonic dopamine-neuron transplants survive in patients with severe Parkinson's disease and result in some clinical benefit in younger but not in older patients.

A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting.

OBJECTIVE: Computer-based reminder systems have the potential to change physician and patient behaviors and to improve patient outcomes. We performed a meta-analysis of published randomized controlled trials to assess the overall effectiveness of computer-based reminder systems in ambulatory settings directed at preventive care. DESIGN: Meta-analysis. SEARCH STRATEGY: Searches of the Medline (1966-1994), Nursing and Allied Health (1982-1994), and Health Planning and Administration (1975-1994) databases identified 16 randomized, controlled trials of computer-based reminder systems in ambulatory settings. STATISTICAL METHODS: A weighted mixed effects model regression analysis was used to estimate intervention effects for computer and manual reminder systems for six classes of preventive practices. MAIN OUTCOME MEASURE: Adjusted odds ratio for preventive practices. RESULTS: Computer reminders improved preventive practices compared with the control condition for vaccinations (adjusted odds ratio [OR] 3.09; 95% confidence interval [CI] 2.39-4.00), breast cancer screening (OR 1.88; 95% CI 1.44-2.45), colorectal cancer screening (OR 2.25; 95% CI 1.74-2.91), and cardiovascular risk reduction (OR 2.01; 95% CI 1.55-2.61) but not cervical cancer screening (OR 1.15; 95% CI 0.89-1.49) or other preventive care (OR 1.02; 95% CI 0.79-1.32). For all six classes of preventive practices combined the adjusted OR was 1.77 (95% CI 1.38-2.27). CONCLUSION: Evidence from randomized controlled studies supports the effectiveness of data-driven computer-based reminder systems to improve prevention services in the ambulatory care setting.

Evaluation of Medicaid managed care. Satisfaction, access, and use.

OBJECTIVE: To evaluate the effects of managed care on Medicaid beneficiaries' satisfaction with, access to, and use of medical care during early implementation of an enrollment initiative. DESIGN: Cross-sectional survey of a random sample of Medicaid beneficiaries in 5 managed care plans and in the conventional Medicaid program. SETTING: New York, NY. PARTICIPANTS: Adults aged 18 to 64 years who received Medicaid insurance benefits through Aid to Families With Dependent Children or State Home Relief and had been enrolled in a managed care plan or receiving benefits under conventional Medicaid for at least 6 months. Of the 2500 enrollees in managed care plans and the 600 other beneficiaries in conventional Medicaid whom we surveyed, 1038 enrollees and 410 nonenrollees responded. OUTCOME MEASURES: Beneficiaries' ratings of overall satisfaction and 13 dimensions of satisfaction related to access, interpersonal and technical quality, and cost; reports of access, including regular source (location) of care, waiting time for appointment, waiting time in office, and ability to obtain care; and reports of use, including inpatient, emergency department, and ambulatory visits. RESULTS: Compared with beneficiaries in conventional Medicaid, managed care enrollees in general gave higher ratings of satisfaction. The results were not consistent, however, between the proportion who were extremely satisfied and the proportion who were extremely dissatisfied. Managed care enrollees had significantly greater odds of being extremely satisfied (excellent and very good ratings), but fewer differences were statistically significant for levels of extreme dissatisfaction (fair and poor ratings). With regard to access, managed care enrollees had significantly greater odds of having a usual source of care (odds ratio [OR], 2.33) and seeing the same clinician there (OR, 2.72) and had significantly shorter appointment and office waiting times. Managed care and conventional Medicaid beneficiaries reported no significant differences in obtaining or delays in getting needed care and in inpatient or emergency department use. CONCLUSIONS: Medicaid managed care enrollees in New York City reported better access to care and higher levels of satisfaction compared with conventional Medicaid beneficiaries. Differences between these findings and those for privately insured populations highlight the pitfalls of generalizing from other groups to Medicaid for policy purposes. Given growing reliance on consumer satisfaction surveys for clinical and public policy, future research should focus on factors that explain extreme satisfaction vs extreme dissatisfaction. New York State's initiative, which has been associated with careful state and local monitoring, merits continuing evaluation as managed care enrollment grows and may become mandatory.

Unlocking clinical data from narrative reports: a study of natural language processing.

OBJECTIVE: To evaluate the automated detection of clinical conditions described in narrative reports. DESIGN: Automated methods and human experts detected the presence or absence of six clinical conditions in 200 admission chest radiograph reports. STUDY SUBJECTS: A computerized, general-purpose natural language processor; 6 internists; 6 radiologists; 6 lay persons; and 3 other computer methods. MAIN OUTCOME MEASURES: Intersubject disagreement was quantified by "distance" (the average number of clinical conditions per report on which two subjects disagreed) and by sensitivity and specificity with respect to the physicians. RESULTS: Using a majority vote, physicians detected 101 conditions in the 200 reports (0.51 per report); the most common condition was acute bacterial pneumonia (prevalence, 0.14), and the least common was chronic obstructive pulmonary disease (prevalence, 0.03). Pairs of physicians disagreed on the presence of at least 1 condition for an average of 20% of reports. The average intersubject distance among physicians was 0.24 (95% Cl, 0.19 to 0.29) out of a maximum possible distance of 6. No physician had a significantly greater distance than the average. The average distance of the natural language processor from the physicians was 0.26 (Cl, 0.21 to 0.32; not significantly greater than the average among physicians). Lay persons and alternative computer methods had significantly greater distance from the physicians (all > 0.5). The natural language processor had a sensitivity of 81% (Cl, 73% to 87%) and a specificity of 98% (Cl, 97% to 99%); physicians had an average sensitivity of 85% and an average specificity of 98%. CONCLUSIONS: Physicians disagreed on the interpretation of narrative reports, but this was not caused by outlier physicians or a consistent difference in the way internists and radiologists read reports. The natural language processor was not distinguishable from the physicians and was superior to all other comparison subjects. Although the domain of this study was restricted (six clinical conditions in chest radiographs), natural language processing seems to have the potential to extract clinical information from narrative reports in a manner that will support automated decision-support and clinical research.

Medical decision support: experience with implementing the Arden Syntax at the Columbia-Presbyterian Medical Center.

We began implementation of a medical decision support system (MDSS) at the Columbia-Presbyterian Medical Center (CPMC) using the Arden Syntax in 1992. The Clinical Event Monitor which executes the Medical Logic Modules (MLMs) runs on a mainframe computer. Data are stored in a relational database and accessed via PL/I programs known as Data Access Modules (DAMs). Currently we have 18 clinical, 12 research and 10 administrative MLMs. On average, the clinical MLMs generate 50357 simple interpretations of laboratory data and 1080 alerts each month. The number of alerts actually read varies by subject of the MLM from 32.4% to 73.5%. Most simple interpretations are not read at all. A significant problem of MLMs is maintenance, and changes in laboratory testing and message output can impair MLM execution significantly. We are now using relational database technology and coded MLM output to study the process outcome of our MDSS.

Computer-generated informational messages directed to physicians: effect on length of hospital stay.

OBJECTIVE: With the advent of hospital payment by diagnosis-related group (DRG), length of stay (LOS) has become a major issue in hospital efforts to control costs. Because the Columbia-Presbyterian Medical Center (CPMC) has had above-average LOSs for many DRGs, the authors tested the hypothesis that a computer-generated informational message directed to physicians would shorten LOS. DESIGN: Randomized clinical trial with the patient as the unit of randomization. SETTING AND STUDY POPULATION: From June 1991 to April 1993, at CPMC in New York, 7,109 patient admissions were randomly assigned to an intervention (informational message) group and 6,990 to a control (no message) group. INTERVENTION: A message giving the average LOS for the patient's admission or provisional DRG, as assigned by hospital utilization review, and the current LOS, in days, was included in the main menu for review of test results in the hospital's clinical information system, available at all nursing stations in the hospital. MAIN OUTCOME MEASURE: Hospital LOS. RESULTS: The median LOS for study patients was 7 days. After adjustment for covariates including age, sex, payor, patient care unit, and time trends, the mean LOS in the intervention group was 3.2% shorter than that in the control group (p = 0.022). CONCLUSION: Computer-generated patient-specific LOS information directed to physicians was associated with a reduction in hospital LOS.

A Bayesian methodology for scaling radiation studies from animals to man.

This paper describes a Bayesian methodology for integrating studies in experimental animals and humans to obtain a risk estimate for a radionuclide for which no data or very limited human data are available. The method is quite general and is not limited to radiation studies. In fact, it was first developed for chemical toxicants. The methodology is illustrated using studies with rats, beagles, and humans exposed to isotopes of Ra and Pu. The goal is a quantitative risk estimate for bone cancer in humans exposed to internally deposited Pu. The choice of bone cancer as an end point and of Pu as the source of exposure was made partially because of its inherent interest but also because of issues of data availability and suitability. We performed Poisson regression analyses on 13 of 15 data sets. These analyses form the basis for the unifying method of interpreting the entire ensemble of studies. Each of the studies is summarized by the estimated dose-response slope and its estimated standard error. These summary statistics are combined with other available biological and physical information about species differences, physical and metabolic characteristics of isotopes, disease mechanisms, and the like. This information enters the analysis in the form of prior assumptions about the parameters of the Bayesian model combining the studies. The posterior distribution for the bone cancer rate in man from the Bayesian analysis of the 13 studies is updated with the limited data on Pu in humans. This update gives the final probability density for the bone cancer rate in humans exposed to internally deposited Pu. This density has a median of about three cancers per 100 Gy and has a 95% probability interval from 0.8 to 11 bone cancers per 100 Gy.