Building Capacity for Evidence-Informed Priority Setting in the Indian Health System: An International Collaborative Experience.

India's rapid economic growth has been accompanied by slower improvements in population health. Given the need to reconcile the ambitious goal of achieving Universal Coverage with limited resources, a robust priority-setting mechanism is required to ensure that the right trade-offs are made and the impact on health is maximised. Health Technology Assessment (HTA) is endorsed by the World Health Assembly as the gold standard approach to synthesizing evidence systematically for evidence-informed priority setting (EIPS). India is formally committed to institutionalising HTA as an integral component of the EIPS process. The effective conduct and uptake of HTA depends on a well-functioning ecosystem of stakeholders adept at commissioning and generating policy-relevant HTA research, developing and utilising rigorous technical, transparent, and inclusive methods and processes, and a strong multisectoral and transnational appetite for the use of evidence to inform policy. These all require myriad complex and complementary capacities to be built at each level of the health system . In this paper we describe how a framework for targeted and locally-tailored capacity building for EIPS, and specifically HTA, was collaboratively developed and implemented by an international network of priority-setting expertise, and the Government of India.

International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument.

PURPOSE: To describe the quality of oncology guidelines developed in different countries. METHODS: The Appraisal of Guidelines and Research and Evaluation (AGREE) Instrument was used to assess the quality of 100 guidelines (including 32 oncology guidelines) from 13 countries. The criteria of the instrument are grouped into six quality domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence. RESULTS: Oncology guidelines had significantly higher scores on rigor of development than nononcology guidelines (42.2% v 29.4%; P =.02). In particular, systematic methods to search for evidence were more often used (P =.01); the methods for formulating the recommendations were more clearly described (P =.02); and health benefits, risks, and side effects were more often considered in formulating the recommendations (P =.03). Although the standardized scores for the other domains were not significantly different, the oncology guidelines had significantly higher scores for items measuring inclusion of all relevant professional groups (P =.05), consideration of patient views (P =.04), and presentation of different options (P =.05). Only three organizations producing oncology guidelines had standardized scores more than 60% for more than three domains. CONCLUSION: The quality of clinical practice guidelines (CPGs) is modest in general, but for certain domains, oncology guidelines seem to be of better quality than others. The experience of the organization may explain higher scores for some items. Research projects and training aimed at improving the quality of guidelines should be developed. The AGREE instrument could provide a basis for defining steps in a shared development approach to produce high-quality CPGs.

EULAR recommendations for the management of knee osteoarthritis: report of a task force of the Standing Committee for International Clinical Studies Including Therapeutic Trials (ESCISIT).

BACKGROUND: Osteoarthritis (OA) is the most common joint disease encountered throughout Europe. A task force for the EULAR Standing Committee for Clinical Trials met in 1998 to determine the methodological and logistical approach required for the development of evidence based guidelines for treatment of knee OA. The guidelines were restricted to cover all currently available treatments for knee OA diagnosed either clinically and/or radiographically affecting any compartment of the knee. METHODS: The first stage was the selection of treatment modalities to be considered. The second stage comprised a search of the electronic databases Medline and Embase using a combination of subject headings and keywords. All European language publications in the form of systematic reviews, meta-analyses, randomised controlled trials, controlled trials, and observational studies were included. During stage three all the relevant studies were quality scored. The summary statistics for validated outcome measures, when available, were recorded and, where practical, the numbers needed to treat and the effect size for each treatment were calculated. In the fourth stage key clinical propositions were determined by expert consensus employing a Delphi approach. The final stage ranked these propositions according to the available evidence. A second set of propositions relating to a future research agenda was determined by expert consensus using a Delphi approach. RESULTS: Over 2400 English language publications and 400 non-English language publications were identified. Seven hundred and forty four studies presented outcome data of the effects of specific treatments on knee OA. Quantitative analysis of treatment effect was possible in only 61 studies. Recommendations for the management of knee OA based on currently available data and expert opinion are presented. Proposals for a future research agenda are highlighted. CONCLUSIONS: These are the first clinical guidelines on knee OA to combine an evidence based approach and a consensus approach across a wide range of treatment modalities. It is apparent that certain clinical propositions are supported by substantial research based evidence, while others are not. There is thus an urgent need for future well designed trials to consider key clinical questions.

Evaluating "payback" on biomedical research from papers cited in clinical guidelines: applied bibliometric study.

OBJECTIVES: To develop a methodology for evaluating the impact of research on health care, and to characterise the papers cited on clinical guidelines. DESIGN: The bibliographic details of the papers cited in 15 clinical guidelines, developed in and for the United Kingdom, were collated and analysed with applied bibliometric techniques. RESULTS: The median age of papers cited in clinical guidelines was eight years; most papers were published by authors living in either the United States (36%) or the United Kingdom (25%)-this is two and a half times more than expected as about 10% of all biomedical outputs are published in the United Kingdom; and clinical guidelines do not cite basic research papers. CONCLUSION: Analysis of the evidence base of clinical guidelines may be one way of tracking the flow of knowledge from the laboratory to the clinic. Moreover, such analysis provides a useful, clinically relevant method for evaluating research outcomes and different strategies in research and development.

Guidelines for evaluation.

Before implementing a guideline, reassurance is needed that the guideline is valid, reproducible and reliable. A method for assessing guideline validity has now been developed for general use. Guideline producers need to consider suitable strategies for implementing their guidelines. When designing and evaluating a guideline implementation strategy, three key questions need to be addressed: what are the most suitable intervention techniques; what are the desired effects of the guideline; and what will be the relevance of changes achieved? The time scale for the evaluation should also be determined carefully.

Appraising clinical practice guidelines in England and Wales: the development of a methodologic framework and its application to policy.

BACKGROUND: Clinical guidelines are pervading all aspects of health care. Their potential benefits are vast--from decreasing unjustified variation in treatment, to improving outcomes for patient, to containing escalating costs. However, there is increasing concern that many of the guidelines produced may be of low quality and recommend ineffective treatment. A framework to appraise the methodologic quality of clinical guidelines, commissioned by the NHS in 1997, was developed and validated in the United Kingdom (UK) under the auspices of the National Research and Development Programme. THE INDEPENDENT APPRAISAL SERVICE: This framework is now used to assess all national guidelines funded by the National Health Service (NHS) in the UK as part of an independent appraisal service. The appraisal provides a basis for policymakers to decide which guidelines should be commended for use in the NHS. Each guideline is appraised by a minimum of six appraisers. Twenty-one guidelines had been appraised as of July 1999. The mean time for completion of the appraisals, from receipt of the guidelines to dispatch of the reports, was just over ten weeks. There has been a marked improvement in the quality of documentation for national guidelines--including the search strategy and synthesis of evidence--in the past three years, although some areas of development remain inadequately reported. POLICY IMPLICATIONS: Ensuring that the clinical guidelines are sound before recommending their use is essential to policymakers responsible for guideline programs, and a formal appraisal should be an integral part of those programs.

Development and application of a generic methodology to assess the quality of clinical guidelines.

BACKGROUND: Despite clinical guidelines penetrating every aspect of clinical practice and health policy, doubts persist over their ability to improve patient care. We have designed and tested a generic critical appraisal instrument, that assesses whether developers have minimized the biases inherent in creating guidelines, and addressed the requirements for effective implementation. DESIGN: Thirty-seven items describing suggested predictors of guideline quality were grouped into three dimensions covering the rigour of development, clarity of presentation (including the context and content) and implementation issues. The ease of use, reliability and validity of the instrument was tested on a national sample of guidelines for the management of asthma, breast cancer, depression and coronary heart disease, with 120 appraisers. A numerical score was derived to allow comparison of guidelines within and between diseases. RESULTS: The instrument has acceptable reliability (Cronbach's alpha coefficient, 0.68-0.84; intra-class correlation coefficient, (0.82-0.90)). The results provided some evidence of validity (Pearson's correlation coefficient between appraisers' dimension scores and their global assessment was 0.49 for dimension one, 0.63 for dimension two and 0.40 for dimension three). The instrument could differentiate between national and local guidelines and was easy to apply. There was variation in the performance of guidelines with most not achieving a majority of criteria in each dimension. CONCLUSIONS: Use of this instrument should encourage developers to create guidelines that reflect relevant research evidence more accurately. Potential users or groups adapting guidelines for local use could apply the instrument to help decide which one to follow. The National Health Service Executive is using the instrument to assist in deciding which guidelines to recommend to the UK National Health Service. This methodology forms the basis of a common approach to assessing guideline quality in Europe.

The quantity and quality of clinical practice guidelines for the management of depression in primary care in the UK.

BACKGROUND: Despite research evidence that guidelines can improve patient care, concerns remain over their cost-effectiveness. This is particularly so when there is a proliferation of guidelines for the same condition. Faced with differing recommendations, users will wish to make informed decisions on which guideline to follow. In creating a guideline appraisal instrument we have assessed guidelines developed in the United Kingdom (UK) for the management of a range of disorders including depression in primary care. AIM: To identify the number of UK clinical guidelines for the management of depression in primary care and to describe their quality and clinical content. METHOD: A survey was undertaken to identify all depression guidelines developed in the UK between January 1991 and January 1996. All guidelines produced by national organizations and a random sample of local guidelines were appraised using a validated instrument by six assessors: a national expert in the disease area, a general practitioner, a public health physician, a hospital consultant, a nurse specializing in the disease area, and a researcher on guideline methodology. The clinical content of each guideline was then assessed by one of the researchers (RB) according to a defined framework. RESULTS: Forty-five depression guidelines were identified. While there was a considerable range in the quality of the six national and three local guidelines appraised, at a group level their performance was similar to guidelines for other diseases. Clinical recommendations tended to reflect the joint consensus statement produced by the Royal College of General Practitioners and Royal College of Psychiatrists in 1992. The most obvious difference was in the style in which the guidelines were written and presented. CONCLUSION: A 'national template' was the starting place for most guidelines. Steps need to be taken to ensure that these templates are based on the best possible research evidence and professional opinion. Local clinicians should concentrate on effective dissemination and implementation strategies, rather than creating new guidelines.