Making health care safer II: an updated critical analysis of the evidence for patient safety practices.

OBJECTIVES: To review important patient safety practices for evidence of effectiveness, implementation, and adoption. DATA SOURCES: Searches of multiple computerized databases, gray literature, and the judgments of a 20-member panel of patient safety stakeholders. REVIEW METHODS: The judgments of the stakeholders were used to prioritize patient safety practices for review, and to select which practices received in-depth reviews and which received brief reviews. In-depth reviews consisted of a formal literature search, usually of multiple databases, and included gray literature, where applicable. In-depth reviews assessed practices on the following domains: • How important is the problem? • What is the patient safety practice? • Why should this practice work? • What are the beneficial effects of the practice? • What are the harms of the practice? • How has the practice been implemented, and in what contexts? • Are there any data about costs? • Are there data about the effect of context on effectiveness? We assessed individual studies for risk of bias using tools appropriate to specific study designs. We assessed the strength of evidence of effectiveness using a system developed for this project. Brief reviews had focused literature searches for focused questions. All practices were then summarized on the following domains: scope of the problem, strength of evidence for effectiveness, evidence on potential for harmful unintended consequences, estimate of costs, how much is known about implementation and how difficult the practice is to implement. Stakeholder judgment was then used to identify practices that were "strongly encouraged" for adoption, and those practices that were "encouraged" for adoption. RESULTS: From an initial list of over 100 patient safety practices, the stakeholders identified 41 practices as a priority for this review: 18 in-depth reviews and 23 brief reviews. Of these, 20 practices had their strength of evidence of effectiveness rated as at least "moderate," and 25 practices had at least "moderate" evidence of how to implement them. Ten practices were classified by the stakeholders as having sufficient evidence of effectiveness and implementation and should be "strongly encouraged" for adoption, and an additional 12 practices were classified as those that should be "encouraged" for adoption. CONCLUSIONS: The evidence supporting the effectiveness of many patient safety practices has improved substantially over the past decade. Evidence about implementation and context has also improved, but continues to lag behind evidence of effectiveness. Twenty-two patient safety practices are sufficiently well understood, and health care providers can consider adopting them now.

Is social support associated with greater weight loss after bariatric surgery?: a systematic review.

Social support may be associated with increased weight loss after bariatric surgery. The objective of this article is to determine impact of post-operative support groups and other forms of social support on weight loss after bariatric surgery. MEDLINE search (1988-2009) was completed using MeSH terms including bariatric procedures and a spectrum of patient factors with potential relationship to weight loss outcomes. Of the 934 screened studies, 10 reported on social support and weight loss outcomes. Five studies reported on support groups and five studies reported on other forms of social support (such as perceived family support or number of confidants) and degree of post-operative weight loss (total n = 735 patients). All studies found a positive association between post-operative support groups and weight loss. One study found a positive association between marital status (being single) and weight loss, while three studies found a non-significant positive trend and one study was inconclusive. Support group attendance after bariatric surgery is associated with greater post-operative weight loss. Further research is necessary to determine the impact of other forms of social support. These factors should be addressed in prospective studies of weight loss following bariatric surgery, as they may represent ways to improve post-operative outcomes.

Identifying quality improvement intervention evaluations: is consensus achievable?

BACKGROUND: The diversity of quality improvement interventions (QIIs) has impeded the use of evidence review to advance quality improvement activities. An agreed-upon framework for identifying QII articles would facilitate evidence review and consensus around best practices. AIM: To adapt and test evidence review methods for identifying empirical QII evaluations that would be suitable for assessing QII effectiveness, impact or success. DESIGN: Literature search with measurement of multilevel inter-rater agreement and review of disagreement. METHODS: Ten journals (2005-2007) were searched electronically and the output was screened based on title and abstract. Three pairs of reviewers then independently rated 22 articles, randomly selected from the screened list. Kappa statistics and percentage agreement were assessed. 12 stakeholders in quality improvement, including QII experts and journal editors, rated and discussed publications about which reviewers disagreed. RESULTS: The level of agreement among reviewers for identifying empirical evaluations of QII development, implementation or results was 73% (with a paradoxically low kappa of 0.041). Discussion by raters and stakeholders regarding how to improve agreement focused on three controversial article selection issues: no data on patient health, provider behaviour or process of care outcomes; no evidence for adaptation of an intervention to a local context; and a design using only observational methods, as correlational analyses, with no comparison group. CONCLUSION: The level of reviewer agreement was only moderate. Reliable identification of relevant articles is an initial step in assessing published evidence. Advancement in quality improvement will depend on the theory- and consensus-based development and testing of a generalizable framework for identifying QII evaluations.

Developing quality indicators for older adults: transfer from the USA to the UK is feasible.

BACKGROUND: Measurement of the quality of health care is essential for quality improvement, and patients are an underused source of data about quality of care. We describe the adaptation of a set of USA quality indicators for use in patient interview surveys in England, to measure the extent to which older patients receive a broad range of effective health care interventions in both primary and secondary care. METHOD: One hundred and nineteen quality indicators covering 16 clinical areas, based on a set of indicators for the care of vulnerable elderly patients in the USA, were reviewed by a panel of 10 clinical experts in England. A modified version of the RAND/UCLA appropriateness method was used and panel members were supplied with literature reviews summarising the evidence base for each quality indicator. The indicators were sent for comment before the panel meeting to UK charitable organisations for older people. RESULTS: The panel rated 102 of the 119 indicators (86%) as valid for use in England; 17 (14%) were rejected as invalid. All 58 indicators about treatment or continuity and follow up were rated as valid compared with just over half (13 of 24) of the indicators about screening. CONCLUSIONS: These 102 indicators are suitable for use in patient interview surveys, including the English Longitudinal Study of Ageing (ELSA). The systematic measurement of quality of care at the population level and identification of gaps in quality is essential for quality improvement. There is potential for transfer of quality indicators between countries, at least for the health care of older people.

Spinal manipulative therapy for low back pain.

BACKGROUND: Low-back pain is a costly illness for which spinal manipulative therapy is commonly recommended. Previous systematic reviews and practice guidelines have reached discordant results on the effectiveness of this therapy for low-back pain. OBJECTIVES: To resolve the discrepancies related to the use of spinal manipulative therapy and to update previous estimates of effectiveness, by comparing spinal manipulative therapy with other therapies and then incorporating data from recent high-quality randomized, controlled trials (RCTs) into the analysis. SEARCH STRATEGY: The Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE and CINAHL were electronically searched from their respective beginning to January 2000, using the Back Group search strategy; references from previous systematic reviews were also screened. SELECTION CRITERIA: Randomized, controlled trials (RCT) that evaluated spinal manipulative therapy for patients with low-back pain, with at least one day of follow-up, and at least one clinically-relevant outcome measure. DATA COLLECTION AND ANALYSIS: Two authors, who served as the reviewers for all stages of the meta-analysis, independently extracted data from unmasked articles. Comparison treatments were classified into the following seven categories: sham, conventional general practitioner care, analgesics, physical therapy, exercises, back school, or a collection of therapies judged to be ineffective or even harmful (traction, corset, bed rest, home care, topical gel, no treatment, diathermy, and minimal massage). MAIN RESULTS: Thirty-nine RCTs were identified. Meta-regression models were developed for acute or chronic pain and short-term and long-term pain and function. For patients with acute low-back pain, spinal manipulative therapy was superior only to sham therapy (10-mm difference [95% CI, 2 to 17 mm] on a 100-mm visual analogue scale) or therapies judged to be ineffective or even harmful. Spinal manipulative therapy had no statistically or clinically significant advantage over general practitioner care, analgesics, physical therapy, exercises, or back school. Results for patients with chronic low-back pain were similar. Radiation of pain, study quality, profession of manipulator, and use of manipulation alone or in combination with other therapies did not affect these results. REVIEWER'S CONCLUSIONS: There is no evidence that spinal manipulative therapy is superior to other standard treatments for patients with acute or chronic low-back pain.

Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently.

OBJECTIVE: To determine whether North American guidelines published subsequent to and in the same topic areas as those developed by the US Agency for Health Care Policy and Research (AHCPR) meet the same methodological criteria. STUDY DESIGN: A guideline appraisal instrument containing 30 criteria was used to evaluate the methodological quality of the AHCPR guidelines, "updates" of the AHCPR guidelines authored by others, and guidelines that referenced or were adapted from the AHCPR guidelines. The frequency with which the criteria appeared in each guideline was compared and an analysis was performed to determine guidelines with two key features of the ACHPR guidelines-multidisciplinary guideline development panels and systematic reviews of the literature. Data were extracted from the guidelines by one investigator and then checked for accuracy by the other. RESULTS: Fifty two guidelines identified by broad based searches were evaluated. 50% of the criteria were present in every AHCPR guideline. The AHCPR guidelines scored 80% or more on 24 of the 30 criteria compared with 14 for the "updates" and 11 for those that referenced/adapted the AHCPR guidelines. All of the 17 AHCPR guidelines had both multidisciplinary development panels and systematic reviews of the literature compared with five from the other two categories (p<0.05). CONCLUSIONS: North American guidelines developed subsequent to and in the same topic areas as the AHCPR guidelines are of substantially worse methodological quality and ignore key features important to guideline development. This finding contrasts with previously published conclusions that guideline methodological quality is improving over time.

Can health care quality indicators be transferred between countries?

OBJECTIVE: To evaluate the transferability of primary care quality indicators by comparing indicators for common clinical problems developed using the same method in the UK and the USA. METHOD: Quality indicators developed in the USA for a range of common conditions using the RAND-UCLA appropriateness method were applied to 19 common primary care conditions in the UK. The US indicators for the selected conditions were used as a starting point, but the literature reviews were updated and panels of UK primary care practitioners were convened to develop quality indicators applicable to British general practice. RESULTS: Of 174 indicators covering 18 conditions in the US set for which a direct comparison could be made, 98 (56.3%) had indicators in the UK set which were exactly or nearly equivalent. Some of the differences may have related to differences in the process of developing the indicators, but many appeared to relate to differences in clinical practice or norms of professional behaviour in the two countries. There was a small but non-significant relationship between the strength of evidence for an indicator and the probability of it appearing in both sets of indicators. CONCLUSION: There are considerable benefits in using work from other settings in developing measures of quality of care. However, indicators cannot simply be transferred directly between countries without an intermediate process to allow for variation in professional culture or clinical practice.

Evaluating quality indicators for patients with community-acquired pneumonia.

BACKGROUND: Several organizations have published evidence-based quality indicators for community-acquired pneumonia (CAP). However, there is variability in the types of indicators presented between organizations and the level of supporting evidence for each of the indicators. A systematic review of the literature and relevant Internet Web sites was performed to identify quality indicators for CAP that have been proposed or recommended by organizations, and each of the indicators was then critically appraised, using a well-defined set of criteria. METHODOLOGY: The MEDLINE, EMBASE, Best Evidence, and Cochrane Systematic Review databases and Internet Web sites were searched for articles and guidelines published between January 1980 and May 2001 to identify quality indicators for CAP and relevant evidence. Experts in the area of health services research were contacted to identify additional sources. A well-defined set of criteria was applied to evaluate each of the quality indicators. RESULTS: The systematic review of the literature and Internet Web sites yielded 44 CAP-specific quality indicators. The critical appraisal of these indicators yielded 16 indicators that were supported by a study that identified an association between quality of care and the process of care or outcome measure, were applied to enough patients to be able to detect clinically meaningful differences, were clinically and/or economically relevant, were measurable in a clinical practice setting, and were precise in their specifications. CONCLUSIONS: Many organizations recommend indicators for CAP. Indicators may serve as measures of clinical performance for clinicians and hospitals, may help in benchmarking, and may ultimately facilitate improvements in quality of care and cost reductions. However, CAP indicators often vary in their meaningfulness, scientific soundness, and interpretability of results. A set of five critical appraisal questions may assist in the evaluation of which quality indicators are most valid.

Sensitivity and specificity of the RAND/UCLA Appropriateness Method to identify the overuse and underuse of coronary revascularization and hysterectomy.

There is no empirical evidence on the sensitivity and specificity of methods to identify the possible overuse and underuse of medical procedures. To estimate the sensitivity and specificity of the RAND/UCLA Appropriateness Method. Parallel three-way replication of the RAND/UCLA Appropriateness Method for each of two procedures, coronary revascularization and hysterectomy. Maximum likelihood estimates of the sensitivity and specificity of the method for each procedure. These values were then used to re-calculate past estimates of overuse and underuse, correcting for the error rate in the appropriateness method. The sensitivity of detecting overuse of coronary revascularization was 68% (95% confidence interval 60-76%) and the specificity was 99% (98-100%). The corresponding values for hysterectomy were 89% (85-94%) and 86% (83-89%). The sensitivity and specificity of detecting the underuse of coronary revascularization were 94% (92-95%) and 97% (96-98%), respectively. Past applications of the appropriateness method have overestimated the prevalence of the overuse of hysterectomy, underestimated the prevalence of the overuse of the coronary revascularization, and provided true estimates of the underuse of revascularization. The sensitivity and specificity of the RAND/UCLA Appropriateness Method vary according to the procedure assessed and appear to estimate the underuse of procedures more accurately than their overuse.

Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated?

CONTEXT: Practice guidelines need to be up-to-date to be useful to clinicians. No published methods are available for assessing whether existing practice guidelines are still valid, nor does any empirical information exist regarding how often such assessments need to be made. OBJECTIVES: To assess the current validity of 17 clinical practice guidelines published by the US Agency for Healthcare Research and Quality (AHRQ) that are still in circulation, and to use this information to estimate how quickly guidelines become obsolete. DESIGN, SETTING, AND PARTICIPANTS: We developed criteria for defining when a guideline needs updating, mailed surveys to members of the original AHRQ guideline panels (n = 170; response rate, 71%), and searched the literature for evidence through March 2000 (n = 6994 titles yielding 173 articles plus 159 new guidelines on the same topics). MAIN OUTCOME MEASURES: Identification of new evidence calling for a major, minor, or no update of the 17 guidelines; survival analysis of the rate at which guidelines became outdated. RESULTS: For 7 guidelines, new evidence and expert judgment indicated that a major update is required; 6 were found to be in need of a minor update; 3 were judged as still valid; and for 1 guideline, we could reach no conclusion. Survival analysis indicated that about half the guidelines were outdated in 5.8 years (95% confidence interval [CI], 5.0-6.6 years). The point at which no more than 90% of the guidelines were still valid was 3.6 years (95% CI, 2.6-4.6 years). CONCLUSIONS: More than three quarters of the AHRQ guidelines need updating. As a general rule, guidelines should be reassessed for validity every 3 years.