Reporting of context and implementation in studies of global health interventions: a pilot study.

BACKGROUND: There is an increasing push for 'evidence-based' decision making in global health policy circles. However, at present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. We performed a pilot study to assess the current state of reporting of context and implementation in studies of global health interventions. METHODS: We identified three existing criteria sets for implementation reporting and selected from them 10 criteria potentially relevant to the needs of policy makers in global health contexts. We applied these 10 criteria to 15 articles included in the evidence base for three global health interventions chosen to represent a diverse set of advocated global health programs or interventions: household water chlorination, prevention of mother-to-child transmission of HIV, and lay community health workers to reduce child mortality. We used a good-fair-poor/none scale for the ratings. RESULTS: The proportion of criteria for which reporting was poor/none ranged from 11% to 54% with an average of 30%. Eight articles had 'good' or 'fair' documentation for greater than 75% of criteria, while five articles had 'poor or none' documentation for 50% of criteria or more. Examples of good reporting were identified. CONCLUSIONS: Reporting of context and implementation information in studies of global health interventions is mostly fair or poor, and highly variable. The idiosyncratic variability in reporting indicates that global health investigators need more guidance about what aspects of context and implementation to measure and how to report them. This lack of context and implementation information is a major gap in the evidence needed by global health policy makers to reach decisions.

Assessment of a method to detect signals for updating systematic reviews.

BACKGROUND: Systematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed. METHODS: The AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority. RESULTS: The 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74. CONCLUSIONS: These results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating.

Hospital fall prevention: a systematic review of implementation, components, adherence, and effectiveness.

OBJECTIVES: To systematically document the implementation, components, comparators, adherence, and effectiveness of published fall prevention approaches in U.S. acute care hospitals. DESIGN: Systematic review. Studies were identified through existing reviews, searching five electronic databases, screening reference lists, and contacting topic experts for studies published through August 2011. SETTING: U.S. acute care hospitals. PARTICIPANTS: Studies reporting in-hospital falls for intervention groups and concurrent (e.g., controlled trials) or historic comparators (e.g., before-after studies). INTERVENTION: Fall prevention interventions. MEASUREMENTS: Incidence rate ratios (IRR, ratio of fall rate postintervention or treatment group to the fall rate preintervention or control group) and ratings of study details. RESULTS: Fifty-nine studies met inclusion criteria. Implementation strategies were sparsely documented (17% not at all) and included staff education, establishing committees, seeking leadership support, and occasionally continuous quality improvement techniques. Most interventions (81%) included multiple components (e.g., risk assessments (often not validated), visual risk alerts, patient education, care rounds, bed-exit alarms, and postfall evaluations). Fifty-four percent did not report on fall prevention measures applied in the comparison group, and 39% neither reported fidelity data nor described adherence strategies such as regular audits and feedback to ensure completion of care processes. Only 45% of concurrent and 15% of historic control studies reported sufficient data to compare fall rates. The pooled postintervention incidence rate ratio (IRR) was 0.77 (95% confidence interval = 0.52-1.12, P = .17; eight studies; I(2) : 94%). Meta-regressions showed no systematic association between implementation intensity, intervention complexity, comparator information, or adherence levels and IRR. CONCLUSION: Promising approaches exist, but better reporting of outcomes, implementation, adherence, intervention components, and comparison group information is necessary to establish evidence on how hospitals can successfully prevent falls.

Probiotics for the prevention and treatment of antibiotic-associated diarrhea: a systematic review and meta-analysis.

CONTEXT: Probiotics are live microorganisms intended to confer a health benefit when consumed. One condition for which probiotics have been advocated is the diarrhea that is a common adverse effect of antibiotic use. OBJECTIVE: To evaluate the evidence for probiotic use in the prevention and treatment of antibiotic-associated diarrhea (AAD). DATA SOURCES: Twelve electronic databases were searched (DARE, Cochrane Library of Systematic Reviews, CENTRAL, PubMed, EMBASE, CINAHL, AMED, MANTIS, TOXLINE, ToxFILE, NTIS, and AGRICOLA) and references of included studies and reviews were screened from database inception to February 2012, without language restriction. STUDY SELECTION: Two independent reviewers identified parallel randomized controlled trials (RCTs) of probiotics (Lactobacillus, Bifidobacterium, Saccharomyces, Streptococcus, Enterococcus, and/or Bacillus) for the prevention or treatment of AAD. DATA EXTRACTION: Two independent reviewers extracted the data and assessed trial quality. RESULTS: A total of 82 RCTs met inclusion criteria. The majority used Lactobacillus-based interventions alone or in combination with other genera; strains were poorly documented. The pooled relative risk in a DerSimonian-Laird random-effects meta-analysis of 63 RCTs, which included 11 811 participants, indicated a statistically significant association of probiotic administration with reduction in AAD (relative risk, 0.58; 95% CI, 0.50 to 0.68; P < .001; I(2), 54%; [risk difference, -0.07; 95% CI, -0.10 to -0.05], [number needed to treat, 13; 95% CI, 10.3 to 19.1]) in trials reporting on the number of patients with AAD. This result was relatively insensitive to numerous subgroup analyses. However, there exists significant heterogeneity in pooled results and the evidence is insufficient to determine whether this association varies systematically by population, antibiotic characteristic, or probiotic preparation. CONCLUSIONS: The pooled evidence suggests that probiotics are associated with a reduction in AAD. More research is needed to determine which probiotics are associated with the greatest efficacy and for which patients receiving which specific antibiotics.

Safety of probiotics used to reduce risk and prevent or treat disease.

OBJECTIVES: To catalog what is known about the safety of interventions containing Lactobacillus, Bifidobacterium, Saccharomyces, Streptococcus, Enterococcus, and/or Bacillus strains used as probiotic agents in research to reduce the risk of, prevent, or treat disease. DATA SOURCES: We searched 12 electronic databases, references of included studies, and pertinent reviews for studies addressing the safety of probiotics from database inception to August 2010 without language restriction. REVIEW METHODS: We identified intervention studies on probiotics that reported the presence or absence of adverse health outcomes in human participants, without restriction by study design, participant type, or clinical field. We investigated the quantity, quality, and nature of adverse events. RESULTS: The search identified 11,977 publications, of which 622 studies were included in the review. In 235 studies, only nonspecific safety statements were made ("well tolerated"); the remaining 387 studies reported the presence or absence of specific adverse events. Interventions and adverse events were poorly documented. A number of case studies described fungemia and some bacteremia potentially associated with administered probiotic organisms. Controlled trials did not monitor routinely for such infections and primarily reported on gastrointestinal adverse events. Based on reported adverse events, randomized controlled trials (RCTs) showed no statistically significantly increased relative risk (RR) of the overall number of experienced adverse events (RR 1.00; 95% confidence interval [CI]: 0.93, 1.07, p=0.999); gastrointestinal; infections; or other adverse events, including serious adverse events (RR 1.06; 95% CI: 0.97, 1.16; p=0.201), associated with short-term probiotic use compared to control group participants; long-term effects are largely unknown. Existing studies primarily examined Lactobacillus alone or in combination with other genera, often Bifidobacterium. Few studies directly compared the safety among different intervention or participant characteristics. Indirect comparisons indicated that effects of delivery vehicles (e.g., yogurt, dairy) should be investigated further. Case studies suggested that participants with compromised health are most likely to experience adverse events associated with probiotics. However, RCTs in medium-risk and critically ill participants did not report a statistically significantly increased risk of adverse events compared to control group participants. CONCLUSIONS: There is a lack of assessment and systematic reporting of adverse events in probiotic intervention studies, and interventions are poorly documented. The available evidence in RCTs does not indicate an increased risk; however, rare adverse events are difficult to assess, and despite the substantial number of publications, the current literature is not well equipped to answer questions on the safety of probiotic interventions with confidence.

Management of Acute Otitis Media: update.

CONTEXT: Acute Otitis Media (AOM), a viral or bacterial infection of the ear, is the most common childhood infection for which antibiotics are prescribed in the United States. In 2001, the Southern California Evidence-based Practice Center conducted a systematic review of the evidence comparing treatments of AOM. OBJECTIVES: This review updates the 2001 review findings on diagnosis and treatment of uncomplicated AOM, assesses the evidence for treatment of recurrent AOM, and assesses the impact of the heptavalent pneumococcal conjugate (PCV7) vaccine on the microbiology of AOM. DATA SOURCES AND STUDY SELECTION: Searches of PubMed® and the Cochrane databases were conducted from January 1998 to July 2010 using the same search strategies used for the 2001 report, with the addition of terms not considered in the 2001 review. The Web of Science was also searched for citations of the 2001 report and its peer-reviewed publications. DATA EXTRACTION: After review by two investigators against pre-determined inclusion/exclusion criteria, we included existing systematic reviews and randomized controlled clinical trials for assessment of treatment efficacy and safety. Pooled analysis was performed for comparisons with three or more trials. RESULTS AND CONCLUSIONS: Few studies were found that examined the accuracy and precision of the diagnosis of AOM. Since PCV7's introduction, AOM microbiology has shifted significantly, with Streptococcus pneumoniae becoming less prevalent and Haemophilus influenzae (HF) increasing in importance. For uncomplicated AOM, pooled analysis indicates that nine children (95% CI: 6, 20) would need to be treated with ampicillin or amoxicillin rather than placebo to note a difference in the rate of clinical success. However, in four studies of delayed treatment approaches for uncomplicated AOM, two had higher rates of clinical success with immediate antibiotic therapy while two did not, and in three studies, a marked decrease in antibiotic utilization was noted. We are unable to draw definitive conclusions regarding the comparative effectiveness of different antibiotics for AOM in children with recurrent otitis media (ROM). For ROM, long-term antibiotic administration will decrease AOM episodes from 3 to 1.5 for every 12 months of treatment per otitis prone child during active treatment (95% CI: 1.2, 2.1); however, potential consequences of long-term treatment need to be considered. Data were insufficient to draw conclusions about comparative effectiveness of different treatment strategies in subgroups of children with uncomplicated AOM. Adverse events were generally more frequent for amoxicillin-clavulanate than for cefdinir, ceftriaxone, or azithromycin. Higher quality studies and improved reporting of study characteristics related to quality are needed to provide definitive conclusions for AOM and ROM treatment options.

Systematic review: comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis.

BACKGROUND: Although several agents are available to treat osteoporosis, the relative efficacy and toxicity of these agents when used to prevent fractures has not been well described. PURPOSE: To compare the benefits in fracture reduction and the harms from adverse events of various therapies for osteoporosis. DATA SOURCES: MEDLINE (1966 to November 2007) and other selected databases were searched for English-language studies. STUDY SELECTION: For the efficacy analysis, investigators selected studies that reported the rate of or risk for fractures. For the adverse event analysis, they selected studies that reported the relationship between an agent and cardiovascular, thromboembolic, or upper gastrointestinal events; malignant conditions; and osteonecrosis. DATA EXTRACTION: Using a standardized protocol, investigators abstracted data on fractures and adverse events, agents and comparators, study design, and variables of methodological quality. DATA SYNTHESIS: Good evidence suggests that alendronate, etidronate, ibandronate, risedronate, zoledronic acid, estrogen, parathyroid hormone (1-34), and raloxifene prevent vertebral fractures more than placebo; the evidence for calcitonin was fair. Good evidence suggests that alendronate, risedronate, and estrogen prevent hip fractures more than placebo; the evidence for zoledronic acid was fair. The effects of vitamin D varied with dose, analogue, and study population for both vertebral and hip fractures. Raloxifene, estrogen, and estrogen-progestin increased the risk for thromboembolic events, and etidronate increased the risk for esophageal ulcerations and gastrointestinal perforations, ulcerations, and bleeding. LIMITATION: Few studies have directly compared different agents or classes of agents used to treat osteoporosis. CONCLUSION: Although good evidence suggests that many agents are effective in preventing osteoporotic fractures, the data are insufficient to determine the relative efficacy or safety of these agents.