GRADE guidance 23: considering cost-effectiveness evidence in moving from evidence to health-related recommendations.

BACKGROUND: This is the 23rd in a series of articles describing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the certainty of evidence and strength of recommendations for systematic reviews, health technology assessments, and clinical guideline development. OBJECTIVES: We outline how resource utilization and cost-effectiveness analyses are integrated into health-related recommendations, using the GRADE Evidence to Decision (EtD) frameworks. STUDY DESIGN AND SETTING: Through iterative discussions and refinement, in-person, and online meetings, and through e-mail communication, we developed draft guidance to incorporate economic evidence in the formulation of health-related recommendations. We developed scenarios to operationalize the guidance. We presented a summary of the results to members of the GRADE Economic Evaluation Project Group. RESULTS: We describe how to estimate the cost of preventing (or achieving) an event to inform assessments of cost-effectiveness of alternative treatments, when there are no published economic evaluations. Evidence profiles and Summary of Findings tables based on systematic reviews of cost-effectiveness analyses can be created to provide top-level summaries of results and quality of multiple published economic evaluations. We also describe how this information could be integrated in GRADE's EtD frameworks to inform health-related recommendations. Three scenarios representing various levels of available cost-effectiveness evidence were used to illustrate the integration process. CONCLUSION: This GRADE guidance provides practical information for presenting cost-effectiveness data and its integration in the development of health-related recommendations, using the EtD frameworks.

Drug-Related Harms in Hospitalized Medicare Beneficiaries: Results From the Healthcare Cost and Utilization Project, 2000-2008.

OBJECTIVES: We examined drug harms in Medicare beneficiaries using the 2000-2008 data from the Nationwide Inpatient Sample Healthcare Cost and Utilization Project. RESEARCH DESIGN: This is a cross-sectional study to examine the national estimates of potentially preventable drug-related hospitalizations and in-hospital adverse drug events identified with the International Classification of Diseases codes excluding intentional harms. RESULTS: Hospitalizations related to drug poisoning constituted 0.8% of all Medicare hospitalizations. Annual hospital charges increased from $1.6 billion in 2000 to $4 billion in 2008. In-hospital adverse drug events were noted in 5.3% of all Medicare hospitalizations and increased by 90% from 2000 to 2008. Patients with extreme versus minor loss of function (odds ratio [OR], 2.96; 95% confidence interval [CI], 2.93-2.99) and with extreme versus minor likelihood of dying (OR, 2.30; 95% CI, 2.29-2.33) had increased odds of in-hospital adverse drug events after adjustment for age, sex, and race. The Medicare beneficiaries with more than 5 versus fewer than 5 listed diagnoses had greater odds of in-hospital adverse drug events (OR, 3.79; 95% CI, 3.76-3.82). Each additional diagnosis at discharge was associated with a 13% increase in odds of in-hospital death in the Medicare beneficiaries hospitalized with drug-related diagnoses and with 12% increase in odds of in-hospital death in the Medicare beneficiaries with in-hospital adverse events. CONCLUSIONS: Potentially preventable drug harms are a growing clinical and financial burden. Comparative outpatient drug safety should be analyzed using Medicare claim databases. In-hospital management of drug safety should target patients with multimorbidity and functional decline.

Availability of results from clinical research: failing policy efforts.

OBJECTIVES: Trial registration has a great potential to increase research transparency and public access to research results. This study examined the availability of results either in journal publications or in the trial registry from all studies registered at ClinicalTrials.gov. METHODS: All 137,612 records from ClinicalTrials.gov in December 2012 were merged with all 19,158 PubMed records containing registration numbers in the indexing field or in the abstracts. A multivariate analysis was conducted to examine the association between the availability of the results with study and participant characteristics available in registration records. RESULTS: Fewer than 10% of the registered studies and 15% of the registered and completed studies had published results. The highest publication rate of 22.4% was for randomized trials completed between 2005 (starting year for structured indexing in PubMed of study registration) and 2010. For 86% of overall and 78% of completed registered studies, no results were available in ClinicalTrials.gov or in journal publications. Studies funded by industry vs. other funding sources and drug studies vs. all studies of other interventions were published less often after adjustment for study type, subject characteristics, or posting of results in ClinicalTrials.gov. CONCLUSION: Existing policy does not ensure availability of results from clinical research. International policy revisions should charge principal investigators with ensuring that the approved protocols and posted data elements are aligned and that results are available from all conducted studies.

Clinical research involving children: registration, completeness, and publication.

BACKGROUND AND OBJECTIVE: Effective health care for children must be based on thorough analyses of the best research evidence. The objective of this study was to examine registration, completeness, and publication of studies involving children. METHODS: We searched the ClinicalTrials.gov registry to identify all closed studies involving children and examined them for completeness and availability of results. We examined publication in peer-reviewed journals for 160 randomly selected National Institutes of Health (NIH)-funded studies from 2000 through 2010 and for 758 randomly selected completed studies. RESULTS: Of 3428 closed studies involving children identified in ClinicalTrials.gov, 2385 (70%) were completed, 28 (0.8%) suspended, 152 (4.4%) terminated, and 38 (1.1%) withdrawn. The proportion of non-completed studies (terminated and suspended) increased linearly by 186% between 2001 and 2009, from 1.9% to 8.4%. Of the 152 terminated studies, 48 did not report reasons for termination, 21 cited safety concerns, and 83 cited poor recruitment or other administrative reasons. Only 29% of completed studies were published. Publication that did occur was an average of 2 years after study completion. Completed interventional studies were published more often than observational studies. Completed industry-funded studies were published less often than studies funded by the NIH. Registered NIH-funded trials were published more often than unregistered. CONCLUSIONS: Results are unavailable for more than half of the studies involving children, revealing a substantial publication bias. Registration and posting of results on ClinicalTrials.gov should be mandatory for all studies involving children.

Network meta-analysis of margin threshold for women with ductal carcinoma in situ.

BACKGROUND: Negative margins are associated with reduced risk of ipsilateral breast tumor recurrence (IBTR) for women with ductal carcinoma in situ (DCIS) treated with breast-conserving surgery (BCS). However, there is no consensus about the best minimum margin width. METHODS: We searched the PubMed database for studies of DCIS published in English between January 1970 and July 2010 and examined the relationship between IBTR and margin status after BCS for DCIS. Women with DCIS were stratified into two groups, BCS with or without radiotherapy. We used frequentist and Bayesian approaches to estimate the odds ratios (OR) of IBTR for groups with negative margins and positive margins. We further examined specific margin thresholds using mixed treatment comparisons and meta-regression techniques. All statistical tests were two-sided. RESULTS: We identified 21 studies published in 24 articles. A total of 1066 IBTR events occurred in 7564 patients, including BCS alone (565 IBTR events in 3098 patients) and BCS with radiotherapy (501 IBTR events in 4466 patients). Compared with positive margins, negative margins were associated with reduced risk of IBTR in patients with radiotherapy (OR = 0.46, 95% credible interval [CrI] = 0.35 to 0.59), and in patients without radiotherapy (OR = 0.34, 95% CrI = 0.24 to 0.47). Compared with patients with positive margins, the risk of IBTR for patients with negative margins was smaller (negative margin >0 mm, OR = 0.45, 95% CrI = 0.38 to 0.53; >2 mm, OR = 0.38, 95% CrI = 0.28 to 0.51; >5 mm, OR = 0.55, 95% CrI = 0.15 to 1.30; and >10 mm, OR = 0.17, 95% CrI = 0.12 to 0.24). Compared with a negative margin greater than 2 mm, a negative margin of at least 10 mm was associated with a lower risk of IBTR (OR = 0.46, 95% CrI = 0.29 to 0.69). We found a probability of .96 that a negative margin threshold greater than 10 mm is the best option compared with other margin thresholds. CONCLUSIONS: Negative surgical margins should be obtained for DCIS patients after BCS regardless of radiotherapy. Within cosmetic constraint, surgeons should attempt to achieve negative margins as wide as possible in their first attempt. More studies are needed to understand whether margin thresholds greater than 10 mm are warranted.

Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases.

OBJECTIVE: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases. STUDY DESIGN AND SETTING: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies. RESULTS: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment. CONCLUSION: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.

Quality of systematic reviews of observational nontherapeutic studies.

INTRODUCTION: High-quality epidemiologic research is essential in reducing chronic diseases. We analyzed the quality of systematic reviews of observational nontherapeutic studies. METHODS: We searched several databases for systematic reviews of observational nontherapeutic studies that examined the prevalence of or risk factors for chronic diseases and were published in core clinical journals from 1966 through June 2008. We analyzed the quality of such reviews by using prespecified criteria and internal quality evaluation of the included studies. RESULTS: Of the 145 systematic reviews we found, fewer than half met each quality criterion; 49% reported study flow, 27% assessed gray literature, 2% abstracted sponsorship of individual studies, and none abstracted the disclosure of conflict of interest by the authors of individual studies. Planned, formal internal quality evaluation of included studies was reported in 37% of systematic reviews. The journal of publication, topic of review, sponsorship, and conflict of interest were not associated with better quality. Odds of formal internal quality evaluation (odds ratio [OR], 1.10 per year; 95% confidence interval [CI], 1.02-1.19) and either planned, formal internal quality evaluation or abstraction of quality criteria of included studies (OR, 1.17 per year; 95% CI, 1.08-1.26) increased over time, without positive trends in other quality criteria from 1990 through June 2008. Systematic reviews with internal quality evaluation did not meet other quality criteria more often than those that ignored the quality of included studies. CONCLUSION: Collaborative efforts from investigators and journal editors are needed to improve the quality of systematic reviews.

Adverse drug effects in hospitalized elderly: data from the healthcare cost and utilization project.

We aimed to analyze trends in hospital admissions due to adverse drug effects between the years 2000 to 2007 among the elderly using the National Inpatient Sample (NIS) of the Healthcare Cost and Utilization Project. We identified the discharges with the principal and all listed diagnoses related to adverse drug effects and associated hospital charges using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9) codes. Between 2000 and 2007, 321,057 patients over 65 years were discharged with a principal diagnosis related to an adverse drug effect. Hospital charges were $5,329,276,300 or $666,159,537 annual cost. The number of discharges and total hospital charges did not change over the examined years, while mean charge per discharge increased on average by $1064 ± 384 per year. Total hospital charges for drug-induced gastritis with hemorrhage increased the most by $11,206,555 per year among those 66-84 years old and by $8,646,456 per year among those older than 85 years. During 2007, 791,931 elderly had adverse treatment effects among all listed diagnoses with hospital charges of $937,795,690. Effective drug management interventions are needed to improve safety of treatments in the elderly.

Cost savings associated with increased RN staffing in acute care hospitals: simulation exercise.

Increasing nurse-to-patient staffing has been recommended as a means to improve patient safety. In this study, researchers analyzed the savings-cost ratio of increased RN-to-patient ratios for patients in ICUs and patients in surgical and medical units based on a meta-analysis of published observational studies. Increased RN staffing was associated with lower hospital-related mortality and adverse patient events and generates societal net savings from avoided patient adverse events. This finding appears to hold in ICUs and, to some extent, in surgical units, but not in medical units. Hospitals do not experience sufficient monetary benefit from reduced length of stay corresponding to an increased RN staffing. Policy decisions about RN staffing should include cost-utility analyses.