Association Between Implementation of the Severe Sepsis and Septic Shock Early Management Bundle Performance Measure and Outcomes in Patients With Suspected Sepsis in US Hospitals.

Importance: In October 2015, the Centers for Medicare & Medicaid Services began requiring US hospitals to report adherence to the Severe Sepsis and Septic Shock Early Management Bundle (SEP-1). Objective: To evaluate the association of SEP-1 implementation with sepsis treatment patterns and outcomes in diverse hospitals. Design, Setting, and Participants: This retrospective cohort study with interrupted time-series analysis and logistic regression models was conducted among adults admitted to 114 hospitals from October 2013 to December 2017 with suspected sepsis (blood culture orders, ≥2 systemic inflammatory response syndrome criteria, and acute organ dysfunction) within 24 hours of hospital arrival. Data analysis was conducted from September 2020 to September 2021. Exposures: SEP-1 implementation in the fourth quarter (Q4) of 2015. Main Outcomes and Measures: The primary outcome was quarterly rates of risk-adjusted short-term mortality (in-hospital death or discharge to hospice). Secondary outcomes included lactate testing and administration of anti-methicillin-resistant Staphylococcus aureus (MRSA) or antipseudomonal ß-lactam antibiotics within 24 hours of hospital arrival. Generalized estimating equations with robust sandwich variances were used to fit logistic regression models to assess for changes in level or trends in these outcomes, adjusting for baseline characteristics and severity of illness. Results: The cohort included 117 510 patients (median [IQR] age, 67 years [55-78] years; 60 530 [51.5%] men and 56 980 [48.5%] women) with suspected sepsis. Lactate testing rates increased from 55.1% (95% CI, 53.9%-56.2%) in Q4 of 2013 to 76.7% (95% CI, 75.4%-78.0%) in Q4 of 2017, with a significant level change following SEP-1 implementation (odds ratio [OR], 1.34; 95% CI, 1.04-1.74). There were increases in use of anti-MRSA antibiotics (19.8% [95% CI, 18.9%-20.7%] in Q4 of 2013 to 26.3% [95% CI, 24.9%-27.7%] in Q4 of 2017) and antipseudomonal antibiotics (27.7% [95% CI, 26.7%-28.8%] in Q4 of 2013 to 40.5% [95% CI, 38.9%-42.0%] in Q4 of 2017), but these trends preceded SEP-1 and did not change with SEP-1 implementation. Unadjusted short-term mortality rates were similar in the pre-SEP-1 period (Q4 of 2013 through Q3 of 2015) vs the post-SEP-1 period (Q1 of 2016 through Q4 of 2017) (20.3% [95% CI, 20.0%-20.6%] vs 20.4% [95% CI, 20.1%-20.7%]), and SEP-1 implementation was not associated with changes in level (OR, 0.94; 95% CI, 0.68-1.29) or trend (OR, 1.00; 95% CI, 0.97-1.04) for risk-adjusted short-term mortality rates. Conclusions and Relevance: In this cohort study, SEP-1 implementation was associated with an immediate increase in lactate testing rates, no change in already-increasing rates of broad-spectrum antibiotic use, and no change in short-term mortality rates for patients with suspected sepsis. Other approaches to decrease sepsis mortality may be warranted.

Impact of Risk Adjustment Using Clinical vs Administrative Data on Hospital Sepsis Mortality Comparisons.

BACKGROUND: A reliable risk-adjusted sepsis outcome measure could complement current national process metrics by identifying outlier hospitals and catalyzing additional improvements in care. However, it is unclear whether integrating clinical data into risk adjustment models identifies similar high- and low-performing hospitals compared with administrative data alone, which are simpler to acquire and analyze. METHODS: We ranked 200 US hospitals by their Centers for Disease Control and Prevention Adult Sepsis Event (ASE) mortality rates and assessed how rankings changed after applying (1) an administrative risk adjustment model incorporating demographics, comorbidities, and codes for severe illness and (2) an integrated clinical and administrative model replacing severity-of-illness codes with laboratory results, vasopressors, and mechanical ventilation. We assessed agreement between hospitals' risk-adjusted ASE mortality rates when ranked into quartiles using weighted kappa statistics (к). RESULTS: The cohort included 4 009 631 hospitalizations, of which 245 808 met ASE criteria. Risk-adjustment had a large effect on rankings: 22/50 hospitals (44%) in the worst quartile using crude mortality rates shifted into better quartiles after administrative risk adjustment, and a further 21/50 (42%) of hospitals in the worst quartile using administrative risk adjustment shifted to better quartiles after incorporating clinical data. Conversely, 14/50 (28%) hospitals in the best quartile using administrative risk adjustment shifted to worse quartiles with clinical data. Overall agreement between hospital quartile rankings when risk-adjusted using administrative vs clinical data was moderate (к = 0.55). CONCLUSIONS: Incorporating clinical data into risk adjustment substantially changes rankings of hospitals' sepsis mortality rates compared with using administrative data alone. Comprehensive risk adjustment using both administrative and clinical data is necessary before comparing hospitals by sepsis mortality rates.

Prevalence of Antibiotic-Resistant Pathogens in Culture-Proven Sepsis and Outcomes Associated With Inadequate and Broad-Spectrum Empiric Antibiotic Use.

Importance: Broad-spectrum antibiotics are recommended for all patients with suspected sepsis to minimize the risk of undertreatment. However, little is known regarding the net prevalence of antibiotic-resistant pathogens across all patients with community-onset sepsis or the outcomes associated with unnecessarily broad empiric treatment. Objective: To elucidate the epidemiology of antibiotic-resistant pathogens and the outcomes associated with both undertreatment and overtreatment in patients with culture-positive community-onset sepsis. Design, Setting, and Participants: This cohort study included 17 430 adults admitted to 104 US hospitals between January 2009 and December 2015 with sepsis and positive clinical cultures within 2 days of admission. Data analysis took place from January 2018 to December 2019. Exposures: Inadequate empiric antibiotic therapy (ie, ≥1 pathogen nonsusceptible to all antibiotics administered on the first or second day of treatment) and unnecessarily broad empiric therapy (ie, active against methicillin-resistant Staphylococcus aureus [MRSA]; vancomycin-resistant Enterococcus [VRE]; ceftriaxone-resistant gram-negative [CTX-RO] organisms, including Pseudomonas aeruginosa; or extended-spectrum ß-lactamase [ESBL] gram-negative organisms when none of these were isolated). Main Outcomes and Measures: Prevalence and empiric treatment rates for antibiotic-resistant organisms and associations of inadequate and unnecessarily broad empiric therapy with in-hospital mortality were assessed, adjusting for baseline characteristics and severity of illness. Results: Of 17 430 patients with culture-positive community-onset sepsis (median [interquartile range] age, 69 [57-81] years; 9737 [55.9%] women), 2865 (16.4%) died in the hospital. The most common culture-positive sites were urine (9077 [52.1%]), blood (6968 [40.0%]), and the respiratory tract (2912 [16.7%]). The most common pathogens were Escherichia coli (5873 [33.7%]), S aureus (3706 [21.3%]), and Streptococcus species (2361 [13.5%]). Among 15 183 cases in which all antibiotic-pathogen susceptibility combinations could be calculated, most (12 398 [81.6%]) received adequate empiric antibiotics. Empiric therapy targeted resistant organisms in 11 683 of 17 430 cases (67.0%; primarily vancomycin and anti-Pseudomonal ß-lactams), but resistant organisms were uncommon (MRSA, 2045 [11.7%]; CTX-RO, 2278 [13.1%]; VRE, 360 [2.1%]; ESBLs, 133 [0.8%]). The net prevalence for at least 1 resistant gram-positive organism (ie, MRSA or VRE) was 13.6% (2376 patients), and for at least 1 resistant gram-negative organism (ie, CTX-RO, ESBL, or CRE), it was 13.2% (2297 patients). Both inadequate and unnecessarily broad empiric antibiotics were associated with higher mortality after detailed risk adjustment (inadequate empiric antibiotics: odds ratio, 1.19; 95% CI, 1.03-1.37; P = .02; unnecessarily broad empiric antibiotics: odds ratio, 1.22; 95% CI, 1.06-1.40; P = .007). Conclusions and Relevance: In this study, most patients with community-onset sepsis did not have resistant pathogens, yet broad-spectrum antibiotics were frequently administered. Both inadequate and unnecessarily broad empiric antibiotics were associated with higher mortality. These findings underscore the need for better tests to rapidly identify patients with resistant pathogens and for more judicious use of broad-spectrum antibiotics for empiric sepsis treatment.

A Novel Approach to Visualize Risk Minimization Effectiveness: Peeping at the 2012 UK Proton Pump Inhibitor Label Change Using a Rapid Cycle Analysis Tool.

INTRODUCTION: Evaluation of risk minimization (RM) actions is an emerging area of regulatory science, often without tools to rapidly and systematically assess their effectiveness. PURPOSE: The aim of this study was to evaluate whether chronographs, typically used for rapid signal detection in observational longitudinal databases, could be used to visualize RM effectiveness. We evaluated the UK Medicines and Healthcare products Regulatory Agency (MHRA) 2012 proton-pump inhibitors (PPIs) class-wide label change that warned of increased risk of bone fracture, advocated to limit duration of use, and recommended to treat those at risk for osteoporosis according to clinical guidelines. METHODS: The cohort consisted of adults aged 18 years and above prescribed one of the five PPIs available in the UK The Health Improvement Network (THIN) database through September 2015. Four chronographs were compared using drug episodes that started before (PRE) and after (POST) the 20 April 2012 MHRA warning; fracture and osteoporosis were evaluated separately. Chronographs show a measure of observed/expected events, the Information Component (IC) and 95% credibility interval (CI), calculated at monthly time intervals relative to the start date of a prescription, and summed to estimate IC over a 3-year period; IC > 0 indicates observed > expected events. We hypothesized that chronographs may assess RM effectiveness if stratified by PRE/POST an RM intervention such as a label change. RESULTS: There were 1,588,973 and 664,601 PPI users in the PRE and POST periods, respectively. We observed a 4.6% reduction in the proportion of long-term PPI episodes and a 4.1% reduction in the overall proportion of the THIN population using PPIs. Compared with the PRE chronographs, when both visually comparing and when examining the summed ICs for fracture in the POST period, a significant reduction was observed overall (IC = 0.024 [95% CI 0.015 to 0.33] PRE vs - 0.141 [95% CI - 0.162 to - 0.120] POST), suggesting less observed events than expected, and prior to PPI start, suggestive of strong channeling (IC = - 0.027 [95% CI - 0.037 to - 0.017] PRE vs - 0.291 [95% CI - 0.308 to - 0.274] POST). Results were qualitatively similar for osteoporosis. CONCLUSIONS: This pilot demonstrated a novel application of a visual, rapid analysis technique to assess RM effectiveness, and supported a hypothesis that prescribers altered some behaviors after the MHRA label change, such as channeling patients at risk of fracture or osteoporosis away from PPI use and potentially reducing fracture outcomes. Limitations include lack of confounding control and outcomes defined only by diagnosis code. Results demonstrate the potential to use large healthcare databases with chronographs to rapidly assess RM effectiveness, similar to signal detection in pharmacovigilance, and may help design more comprehensive RM evaluation studies.

Epidemiology of Hospital-Onset Versus Community-Onset Sepsis in U.S. Hospitals and Association With Mortality: A Retrospective Analysis Using Electronic Clinical Data.

OBJECTIVES: Prior studies have reported that hospital-onset sepsis is associated with higher mortality rates than community-onset sepsis. Most studies, however, have used inconsistent case-finding methods and applied limited risk-adjustment for potential confounders. We used consistent sepsis criteria and detailed electronic clinical data to elucidate the epidemiology and mortality associated with hospital-onset sepsis. DESIGN: Retrospective cohort study. SETTING: 136 U.S. hospitals in the Cerner HealthFacts dataset. PATIENTS: Adults hospitalized in 2009-2015. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We identified sepsis using Centers for Disease Control and Prevention Adult Sepsis Event criteria and estimated the risk of in-hospital death for hospital-onset sepsis versus community-onset sepsis using logistic regression models. In patients admitted without community-onset sepsis, we estimated risk of death associated with hospital-onset sepsis using Cox regression models with sepsis as a time-varying covariate. Models were adjusted for baseline characteristics and severity of illness. Among 2.2 million hospitalizations, there were 95,154 sepsis cases: 83,620 (87.9%) community-onset sepsis and 11,534 (12.1%) hospital-onset sepsis (0.5% of hospitalized cohort). Compared to community-onset sepsis, hospital-onset sepsis patients were younger (median 66 vs 68 yr) but had more comorbidities (median Elixhauser score 14 vs 11), higher Sequential Organ Failure Assessment scores (median 4 vs 3), higher ICU admission rates (61% vs 44%), longer hospital length of stay (median 19 vs 8 d), and higher in-hospital mortality (33% vs 17%) (p < 0.001 for all comparisons). On multivariate analysis, hospital-onset sepsis was associated with higher mortality versus community-onset sepsis (odds ratio, 2.1; 95% CI, 2.0-2.2) and patients admitted without sepsis (hazard ratio, 3.0; 95% CI, 2.9-3.2). CONCLUSIONS: Hospital-onset sepsis complicated one in 200 hospitalizations and accounted for one in eight sepsis cases, with one in three patients dying in-hospital. Hospital-onset sepsis preferentially afflicted ill patients but even after risk-adjustment, they were twice as likely to die as community-onset sepsis patients; in patients admitted without sepsis, hospital-onset sepsis tripled the risk of death. Hospital-onset sepsis is an important target for surveillance, prevention, and quality improvement initiatives.

Does Obesity Protect Against Death in Sepsis? A Retrospective Cohort Study of 55,038 Adult Patients.

OBJECTIVES: Observational studies suggest obesity is associated with sepsis survival, but these studies are small, fail to adjust for key confounders, measure body mass index at inconsistent time points, and/or use administrative data to define sepsis. To estimate the relationship between body mass index and sepsis mortality using detailed clinical data for case detection and risk adjustment. DESIGN: Retrospective cohort analysis of a large clinical data repository. SETTING: One-hundred thirty-nine hospitals in the United States. PATIENTS: Adult inpatients with sepsis meeting Sepsis-3 criteria. EXPOSURE: Body mass index in six categories: underweight (body mass index < 18.5 kg/m), normal weight (body mass index = 18.5-24.9 kg/m), overweight (body mass index = 25.0-29.9 kg/m), obese class I (body mass index = 30.0-34.9 kg/m), obese class II (body mass index = 35.0-39.9 kg/m), and obese class III (body mass index ≥ 40 kg/m). MEASUREMENTS: Multivariate logistic regression with generalized estimating equations to estimate the effect of body mass index category on short-term mortality (in-hospital death or discharge to hospice) adjusting for patient, infection, and hospital-level factors. Sensitivity analyses were conducted in subgroups of age, gender, Elixhauser comorbidity index, Sequential Organ Failure Assessment quartiles, bacteremic sepsis, and ICU admission. MAIN RESULTS: From 2009 to 2015, we identified 55,038 adults with sepsis and assessable body mass index measurements: 6% underweight, 33% normal weight, 28% overweight, and 33% obese. Crude mortality was inversely proportional to body mass index category: underweight (31%), normal weight (24%), overweight (19%), obese class I (16%), obese class II (16%), and obese class III (14%). Compared with normal weight, the adjusted odds ratio (95% CI) of mortality was 1.62 (1.50-1.74) for underweight, 0.73 (0.70-0.77) for overweight, 0.61 (0.57-0.66) for obese class I, 0.61 (0.55-0.67) for obese class II, and 0.65 (0.59-0.71) for obese class III. Results were consistent in sensitivity analyses. CONCLUSIONS: In adults with clinically defined sepsis, we demonstrate lower short-term mortality in patients with higher body mass indices compared with those with normal body mass indices (both unadjusted and adjusted analyses) and higher short-term mortality in those with low body mass indices. Understanding how obesity improves survival in sepsis would inform prognostic and therapeutic strategies.

Sepsis Surveillance Using Adult Sepsis Events Simplified eSOFA Criteria Versus Sepsis-3 Sequential Organ Failure Assessment Criteria.

OBJECTIVES: Sepsis-3 defines organ dysfunction as an increase in the Sequential Organ Failure Assessment score by greater than or equal to 2 points. However, some Sequential Organ Failure Assessment score components are not routinely recorded in all hospitals' electronic health record systems, limiting its utility for wide-scale sepsis surveillance. The Centers for Disease Control and Prevention recently released the Adult Sepsis Event surveillance definition that includes simplified organ dysfunction criteria optimized for electronic health records (eSOFA). We compared eSOFA versus Sequential Organ Failure Assessment with regard to sepsis prevalence, overlap, and outcomes. DESIGN: Retrospective cohort study. SETTING: One hundred eleven U.S. hospitals in the Cerner HealthFacts dataset. PATIENTS: Adults hospitalized in 2013-2015. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We identified clinical indicators of presumed infection (blood cultures and antibiotics) concurrent with either: 1) an increase in Sequential Organ Failure Assessment score by 2 or more points (Sepsis-3) or 2) 1 or more eSOFA criteria: vasopressor initiation, mechanical ventilation initiation, lactate greater than or equal to 2.0 mmol/L, doubling in creatinine, doubling in bilirubin to greater than or equal to 2.0 mg/dL, or greater than or equal to 50% decrease in platelet count to less than 100 cells/µL (Centers for Disease Control and Prevention Adult Sepsis Event). We compared area under the receiver operating characteristic curves for discriminating in-hospital mortality, adjusting for baseline characteristics. Of 942,360 patients in the cohort, 57,242 (6.1%) had sepsis by Sequential Organ Failure Assessment versus 41,618 (4.4%) by eSOFA. Agreement between sepsis by Sequential Organ Failure Assessment and eSOFA was good (Cronbach's alpha 0.81). Baseline characteristics and infectious diagnoses were similar, but mortality was higher with eSOFA (17.1%) versus Sequential Organ Failure Assessment (14.4%; p < 0.001) as was discrimination for mortality (area under the receiver operating characteristic curve, 0.774 vs 0.759; p < 0.001). Comparisons were consistent across subgroups of age, infectious diagnoses, and comorbidities. CONCLUSIONS: The Adult Sepsis Event's eSOFA organ dysfunction criteria identify a smaller, more severely ill sepsis cohort compared with the Sequential Organ Failure Assessment score, but with good overlap and similar clinical characteristics. Adult Sepsis Events may facilitate wide-scale automated sepsis surveillance that tracks closely with the more complex Sepsis-3 criteria.

Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons.

OBJECTIVES: Administrative claims data are commonly used for sepsis surveillance, research, and quality improvement. However, variations in diagnosis, documentation, and coding practices for sepsis and organ dysfunction may confound efforts to estimate sepsis rates, compare outcomes, and perform risk adjustment. We evaluated hospital variation in the sensitivity of claims data relative to clinical data from electronic health records and its impact on outcome comparisons. DESIGN, SETTING, AND PATIENTS: Retrospective cohort study of 4.3 million adult encounters at 193 U.S. hospitals in 2013-2014. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Sepsis was defined using electronic health record-derived clinical indicators of presumed infection (blood culture draws and antibiotic administrations) and concurrent organ dysfunction (vasopressors, mechanical ventilation, doubling in creatinine, doubling in bilirubin to ≥ 2.0 mg/dL, decrease in platelets to < 100 cells/µL, or lactate ≥ 2.0 mmol/L). We compared claims for sepsis prevalence and mortality rates between both methods. All estimates were reliability adjusted to account for random variation using hierarchical logistic regression modeling. The sensitivity of hospitals' claims data was low and variable: median 30% (range, 5-54%) for sepsis, 66% (range, 26-84%) for acute kidney injury, 39% (range, 16-60%) for thrombocytopenia, 36% (range, 29-44%) for hepatic injury, and 66% (range, 29-84%) for shock. Correlation between claims and clinical data was moderate for sepsis prevalence (Pearson coefficient, 0.64) and mortality (0.61). Among hospitals in the lowest sepsis mortality quartile by claims, 46% shifted to higher mortality quartiles using clinical data. Using implicit sepsis criteria based on infection and organ dysfunction codes also yielded major differences versus clinical data. CONCLUSIONS: Variation in the accuracy of claims data for identifying sepsis and organ dysfunction limits their use for comparing hospitals' sepsis rates and outcomes. Using objective clinical data may facilitate more meaningful hospital comparisons.

Risk Adjustment for Sepsis Mortality to Facilitate Hospital Comparisons Using Centers for Disease Control and Prevention's Adult Sepsis Event Criteria and Routine Electronic Clinical Data.

Variability in hospital-level sepsis mortality rates may be due to differences in case mix, quality of care, or diagnosis and coding practices. Centers for Disease Control and Prevention's Adult Sepsis Event definition could facilitate objective comparisons of sepsis mortality rates between hospitals but requires rigorous risk-adjustment tools. We developed risk-adjustment models for Adult Sepsis Events using administrative and electronic health record data. DESIGN: Retrospective cohort study. SETTING: One hundred thirty-six U.S. hospitals in Cerner HealthFacts (derivation dataset) and 137 HCA Healthcare hospitals (validation dataset). PATIENTS: A total of 95,154 hospitalized adult patients (derivation) and 201,997 patients (validation) meeting Centers for Disease Control and Prevention Adult Sepsis Event criteria. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We created logistic regression models of increasing complexity using administrative and electronic health record data to predict in-hospital mortality. An administrative model using demographics, comorbidities, and coded markers of severity of illness at admission achieved an area under the receiver operating curve of 0.776 (95% CI, 0.770-0.783) in the Cerner cohort, with diminishing calibration at higher baseline risk deciles. An electronic health record-based model that integrated administrative data with laboratory results, vasopressors, and mechanical ventilation achieved an area under the receiver operating curve of 0.826 (95% CI, 0.820-0.831) in the derivation cohort and 0.827 (95% CI, 0.824-0.829) in the validation cohort, with better calibration than the administrative model. Adding vital signs and Glasgow Coma Score minimally improved performance. CONCLUSIONS: Models incorporating electronic health record data accurately predict hospital mortality for patients with Adult Sepsis Events and outperform models using administrative data alone. Utilizing laboratory test results, vasopressors, and mechanical ventilation without vital signs may achieve a good balance between data collection needs and model performance, but electronic health record-based models must be attentive to potential variability in data quality and availability. With ongoing testing and refinement of these risk-adjustment models, Adult Sepsis Event surveillance may enable more meaningful comparisons of hospital sepsis outcomes and provide an important window into quality of care.

Development of a mother-child database for drug exposure and adverse event detection in the Military Health System.

PURPOSE: The aim of this study was to develop a mother-child linked database consisting of all eligible active duty military personnel, retirees, and their dependents in order to conduct medication-related analyses to improve the safety and quality of care in the Military Health System (MHS). METHODS: Eligible women of reproductive age with at least one pregnancy-related encounter between January 2005 and December 2013 receiving care in the MHS were included in the study population. Building on previously published algorithms, we used pregnancy-related diagnostic and procedure codes, parameterized temporal constraints, and data elements unique to the MHS to identify pregnancies ending in live births, stillbirth, spontaneous abortion, or ectopic pregnancy. Pregnancies ending in live births were matched to presumptive offspring using birth dates and family-based sponsorship identification. Antidepressant and antiepileptic use during pregnancy was evaluated using electronic pharmacy data. RESULTS: Algorithms identified 755,232 women who experienced 1,099,648 complete pregnancies with both pregnancy care encounter and pregnancy outcome. Of the 924,320 live birth pregnancies, 827,753 (90.0%) were matched to offspring. Algorithms also identified 5,663 stillbirths, 11,358 ectopic pregnancies, and 169,665 spontaneous abortions. Among the matched singleton live birth pregnancies, 7.1% of mothers were dispensed an antidepressant at any point during pregnancy, usually a selective serotonin reuptake inhibitor, (75.3%), whereas 1.3% of mothers were dispensed an antiepileptic drug.

Automated method for detecting increases in frequency of spontaneous adverse event reports over time.

A statistical methodology--focused on temporal change detection--was developed to highlight excursions from baseline spontaneous adverse event (AE) reporting. We used regression (both smooth trend and seasonal components) to model the time course of a drug's reports containing an AE, and then compared the sum of counts in the past 2 months with the fitted trend. The signaling threshold was tuned, using retrospective analysis, to yield acceptable sensitivity and specificity. The method may enhance pharmacovigilance by providing effective automated alerting of reporting aberrations when databases are small, when drugs have established safety profiles, and/or when product quality issues are of concern.

Molecular clinical safety intelligence: a system for bridging clinically focused safety knowledge to early-stage drug discovery - the GSK experience.

Drug toxicity is a major cause of late-stage product attrition. During lead identification and optimization phases little information is typically available about which molecules might have safety concerns. A system was built linking chemistry, preclinical and human safety information, enabling scientists to lever safety knowledge across multiple disciplines. The system consists of a data warehouse with chemical structures and chemical and biological properties for ∼80000 compounds and tools to access and analyze clinical data, toxicology, in vitro pharmacology and drug metabolism data. Tapping into this safety knowledge enables rapid clinically focused risk assessments of drug candidates. Use of this strategy adds value to the drug discovery process at GSK via efficient triage of compounds based on their potential for toxicity.

Adaptation of Bayesian data mining algorithms to longitudinal claims data: coxib safety as an example.

INTRODUCTION: Bayesian data mining methods have been used to evaluate drug safety signals from adverse event reporting systems and allow for evaluation of multiple endpoints that are not prespecified. Their adaptation for use with longitudinal data such as administrative claims has not been previously evaluated or validated. METHODS: In this pilot study, we evaluated the feasibility of adapting data mining methods using the empirical Bayes Multi-item Gamma Poisson Shrinkage (MGPS) algorithm to longitudinal administrative claims data. The Medicare Current Beneficiary Survey was used to identify a cohort of Medicare enrollees who were exposed to cyclooxygenase selective (coxib) or nonselective nonsteroidal anti-inflammatory drugs (NS-NSAIDs) from 1999 to 2003. Empirical Bayes MGPS algorithm was used to simultaneously evaluate 259 outcomes associated with current use of coxibs versus NS-NSAIDs while adjusting for key covariates and multiple comparisons. For comparison, a parallel analysis used traditional epidemiologic methods to evaluate the relationship between coxib versus NS-NSAID use and acute myocardial infarction, with the goal of establishing the concurrent validity of the data mining approach. RESULTS: Among 9431 Medicare beneficiaries using NSAIDs and considering all 259 possible outcomes, empirical Bayes MGPS identified an association between current celecoxib use and acute myocardial infarction (Empirical Bayes Geometric Mean ratio 1.91) but not other outcomes. Rofecoxib use was associated with acute cerebrovascular events (Empirical Bayes Geometric Mean ratio 1.85) and several other diagnoses that likely represented indications for the drug. Results from the analyses using traditional epidemiologic methods were similar and indicated that the data mining results were valid. DISCUSSION: Bayesian data mining methods seem useful to evaluate drug safety using administrative data. Further work will be needed to extend these findings to different types of drug exposures and to other claims databases.

Antipsychotics, glycemic disorders, and life-threatening diabetic events: a Bayesian data-mining analysis of the FDA adverse event reporting system (1968-2004).

BACKGROUND: This analysis compared diabetes-related adverse events associated with use of different antipsychotic agents. A disproportionality analysis of the US Food and Drug Administration (FDA) Adverse Event Reporting System (AERS) was performed. METHODS: Data from the FDA postmarketing AERS database (1968 through first quarter 2004) were evaluated. Drugs studied included aripiprazole, clozapine, haloperidol, olanzapine, quetiapine, risperidone, and ziprasidone. Fourteen Medical Dictionary for Regulatory Activities (MedDRA) Primary Terms (MPTs) were chosen to identify diabetes-related adverse events; 3 groupings into higher-level descriptive categories were also studied. Three methods of measuring drug-event associations were used: proportional reporting ratio, the empirical Bayes data-mining algorithm known as the Multi-Item Gamma Poisson Shrinker, and logistic regression (LR) analysis. Quantitative measures of association strength, with corresponding confidence intervals, between drugs and specified adverse events were computed and graphed. Some of the LR analyses were repeated separately for reports from patients under and over 45 years of age. Differences in association strength were declared statistically significant if the corresponding 90% confidence intervals did not overlap. RESULTS: Association with various glycemic events differed for different drugs. On average, the rankings of association strength agreed with the following ordering: low association, ziprasidone, aripiprazole, haloperidol, and risperidone; medium association, quetiapine; and strong association, clozapine and olanzapine. The median rank correlation between the above ordering and the 17 sets of LR coefficients (1 set for each glycemic event) was 93%. Many of the disproportionality measures were significantly different across drugs, and ratios of disproportionality factors of 5 or more were frequently observed. CONCLUSIONS: There are consistent and substantial differences between atypical antipsychotic drugs in the disproportionality reporting ratios relating to glycemic effects, especially life-threatening events, in the AERS database. The relative associational rankings of drugs are similar in reports from younger and older patients. These results agree with several other reports in the literature, do not support a "class effect" hypothesis, and provide a strong rationale for further studies to clarify the issue.

Psychogenic camptocormia.

Camptocormia describes a severe forward-flexion at the waist. Originally used in reference to a conversion disorder seen in military personel (Souques and Rosanoff-Saloff, Rev Neurol 1915, 22, 937; Rosen and Frymoyer, Spine 1985, 10, 325; Miller and Forbes, Br J Psychiatry 1990, 157, 765; Perez-Sales, Arch Phys Med Rehabil 1990, 71, 1078; Sinel and Eisenberg, Rev Rhum Mal Osteoartic 1992, 59, 169; Miller and Forbes, Mil Med 1990, 155, 561; Belgrano and Giordano, Rev Neurol 1947, 79, 25-35), the term has been adapted to describe severely flexed postures observed in Parkinson disease (Djaldetti et al., Mov Disord 1999, 14, 443), other basal ganglia disorders (Nieves et al., Mov Disord 2001, 16, 177; Reichel et al., Nervenarzt 2001, 72, 281), and muscular disease (Delcey et al., Rev Med Intern 2002, 23, 144; Van Gerpen, Mov Disord 2001, 16, 358). Although rare, psychogenic camptocormia is seen in civilian populations, presenting diagnostic challenges. We describe a patient whose initial history and examination suggested a psychogenic etiology for his camptocormic posture. A multidisciplinary approach elucidated the patient's motivations and emotional state, and has been helpful in management of this patient.

Signal detection methodologies to support effective safety management.

The increased focus on the safety of medical products, as well as the growing volume of available safety information, has created a need for objective quantitative approaches to supplement the medical review of individual case safety reports. Statistical algorithms can be used to identify trends and relationships in both clinical and postmarketing safety databases in support of safety signal detection. Powerful data visualization tools facilitate the medical review of the complex information generated by these methods. In addition, all these approaches need to be integrated into the daily practice of clinical safety and postmarketing pharmacovigilance.

Comparative performance of two quantitative safety signalling methods: implications for use in a pharmacovigilance department.

BACKGROUND AND OBJECTIVES: There is increasing interest in using disproportionality-based signal detection methods to support postmarketing safety surveillance activities. Two commonly used methods, empirical Bayes multi-item gamma Poisson shrinker (MGPS) and proportional reporting ratio (PRR), perform differently with respect to the number and types of signals detected. The goal of this study was to compare and analyse the performance characteristics of these two methods, to understand why they differ and to consider the practical implications of these differences for a large, industry-based pharmacovigilance department. METHODS: We compared the numbers and types of signals of disproportionate reporting (SDRs) obtained with MGPS and PRR using two postmarketing safety databases and a simulated database. We recorded signal counts and performed a qualitative comparison of the drug-event combinations signalled by the two methods as well as a sensitivity analysis to better understand how the thresholds commonly used for these methods impact their performance. RESULTS: PRR detected more SDRs than MGPS. We observed that MGPS is less subject to confounding by demographic factors because it employs stratification and is more stable than PRR when report counts are low. Simulation experiments performed using published empirical thresholds demonstrated that PRR detected false-positive signals at a rate of 1.1%, while MGPS did not detect any statistical false positives. In an attempt to separate the effect of choice of signal threshold from more fundamental methodological differences, we performed a series of experiments in which we modified the conventional threshold values for each method so that each method detected the same number of SDRs for the example drugs studied. This analysis, which provided quantitative examples of the relationship between the published thresholds for the two methods, demonstrates that the signalling criterion published for PRR has a higher signalling frequency than that published for MGPS. DISCUSSION AND CONCLUSION: The performance differences between the PRR and MGPS methods are related to (i) greater confounding by demographic factors with PRR; (ii) a higher tendency of PRR to detect false-positive signals when the number of reports is small; and (iii) the conventional thresholds that have been adapted for each method. PRR tends to be more 'sensitive' and less 'specific' than MGPS. A high-specificity disproportionality method, when used in conjunction with medical triage and investigation of critical medical events, may provide an efficient and robust approach to applying quantitative methods in routine postmarketing pharmacovigilance.

Using simulation to assess the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data.

The objective of the work described in this paper is to develop a means for characterizing the validity of an empirical methodology for detecting signals potentially related to complicated adverse event (AE) coding terms in multidimensional public health surveillance data. The signal detection tool under evaluation is the multi-item gamma Poisson shrinkage (MGPS) estimation program. We were interested in its potential application to passive surveillance system monitoring, to screen for 'signals' of complicated adverse event coding terms (AE terms) in complex and noisy data. The research was to design and produce a flexible and user-friendly utility for probabilistically defining complicated signals in a database, iterating large numbers of applications of the MGPS detection algorithm and establishing proportions of correct detection events. We sought to establish the specificity of the MGPS by developing a random background using a gradient that ranged from rigorous (but not very relevant) to relevant (but noisy). To establish the sensitivity, signals were defined based on recognized public health issues of interest (such as the introduction of a new vaccine into the population). Methods of representing a signal included a simple pair-wise association consisting of a new vaccine and one AE term, as well as a more realistic complex of multiple AE terms comprising a 'syndrome'. A web application has been developed to create and insert signals with user-defined probabilities in multiple iterations of simulated random background data. Three forms of simulated data based on the vaccine adverse event reporting system (VAERS) cumulative spontaneous database were defined to serve as background noise against which to contrast introduced vaccine adverse event signals: (1) completely random associations between vaccines and AE terms, (2) random associations of vaccine sets and AE term sets preserving naturally observed vaccine co-occurrences and AE term co-occurrences and (3) samples from the actual VAERS data as reported. Rates of detection by the MGPS algorithm can be established for specific signal patterns at varying probabilistic intensities in a choice of random background data forms. Knowing these rates is important for determining the degree of response to an MGPS signal detection event in 'live' data.

Association of asthma therapy and Churg-Strauss syndrome: an analysis of postmarketing surveillance data.

BACKGROUND: Churg-Strauss syndrome (CSS), also known as allergic granulomatous angiitis (AGA), is a rare vasculitis that occurs in patients with bronchial asthma. The nature of the association of CSS with various asthma therapies is unclear. OBJECTIVE: This study investigated the associations of different multidrug asthma therapy regimens and the reporting of AGA (the preferred code for CSS in the coding dictionary for the Adverse Event Reporting System [AERS]) by applying an iterative method of disproportionally analysis to th AERS database maintained by the US Food and Drug Administration. METHODS: The public-release version of the AERS database was used to identify reports of AGA in patients receiving asthma therapy. Reporting of AGA was examined using iterative disproportionality methods in patients receiving > or =1 of the following drug classes: inhaled corticosteroid (ICS), leukotriene receptor antagonist (LTRA), short-acting beta(2)-agonist (SABA), or long-acting beta(2)-agonist (LABA). The Bayesian data-mining algorithm known as the multi-item gamma poisson shrinker was used to determine the relative reporting rates by calculation of the empirical Bayes geometric mean (EBGM) and its 90% CI (EB05 = lower limit and EB95 = upper limit) for each drug. Subset analyses were performed for each drug with different medication combinations to differentiate the relative reporting of AGA for each. RESULTS: A strong association was found between LTRA use and AGA (EBGM = 104.0, EB05 = 95.0, EB95 = 113.8) that persisted with all combinations of therapy studied. AGA was also associated with the ICS, SABA and LABA classes (EBGM values of 27.8, 14.6 and 40.4, respectively). However, the latter associations were mostly dependent on the presence of concurrent LTRA and, to a lesser extemt, oral corticosteroid therapy and became negligible (ie, EB05 < 2) for patients who were not receiving these concurrent treatments. CONCLUSIONS: Differences based on relative reporting were observed in the patterns of association of AGA with LTRA, ICS, and beta(2)-agonist therapies. A strong association between LTRA use and AGA was present regardless of the use of other asthma drugs.