Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study.

OBJECTIVES: To describe how the methodological quality of primary studies is assessed in systematic reviews and whether the quality assessment is taken into account in the interpretation of results. DATA SOURCES: Cochrane systematic reviews and systematic reviews in paper based journals. STUDY SELECTION: 965 systematic reviews (809 Cochrane reviews and 156 paper based reviews) published between 1995 and 2002. DATA SYNTHESIS: The methodological quality of primary studies was assessed in 854 of the 965 systematic reviews (88.5%). This occurred more often in Cochrane reviews than in paper based reviews (93.9% v 60.3%, P < 0.0001). Overall, only 496 (51.4%) used the quality assessment in the analysis and interpretation of the results or in their discussion, with no significant differences between Cochrane reviews and paper based reviews (52% v 49%, P = 0.58). The tools and methods used for quality assessment varied widely. CONCLUSIONS: Cochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results. Methods for assessment of methodological quality by systematic reviews are still in their infancy and there is substantial room for improvement.

Sensory stimulation of brain-injured individuals in coma or vegetative state: results of a Cochrane systematic review.

OBJECTIVES: To assess the effectiveness of sensory stimulation programmes in patients in coma or vegetative state. DESIGN: Systematic review of randomized control trials (RCT) and nonrandomized controlled clinical trials (CCT) comparing any type of stimulation programmes with standard rehabilitation in patients in coma or vegetative state. The Injuries Group specialized register, the Cochrane Controlled trial register, EMBASE, MEDLINE, CINAHL, PSYCHLIT from 1966 to January 2002 were searched without language restriction. Reference lists of articles were scanned and experts in the area contacted to find other relevant studies. Abstracts and papers found were initially screened by one reviewer. Three reviewers independently identified relevant studies, extracted data and assessed study quality, resolving disagreement by consensus. OUTCOME MEASURES: Duration of unconsciousness (including coma and vegetative state) defined as the time between trauma and objective recovery of the ability to respond to verbal commands; level of consciousness, as measured by the Glasgow Coma Scale (GCS); level of cognitive functioning (LCF); functional outcomes, as measured by Glasgow Outcome Scale (GOS) or by Disability Rating Scale; negative effects (e.g. increased intracranial pressure). RESULTS: Three studies (one RCT and two CCTs) with 68 traumatic brain-injured patients in total, most of whom were road accident victims, met the inclusion criteria. The overall methodological quality was poor and studies differed widely in terms of study design and conduct. Moreover, due to the diversity in reporting of outcome measures, a quantitative metanalysis was not possible. None of the three studies provided useful and valid results on outcomes of clinical relevance for coma patients. CONCLUSIONS: This systematic review indicates that there is no reliable evidence to support the effectiveness of multisensory stimulation programmes in patients in coma or the vegetative state.