Illustrating potential efficiency gains from using cost-effectiveness evidence to reallocate Medicare expenditures.

OBJECTIVES: The Centers for Medicare & Medicaid Services does not explicitly use cost-effectiveness information in national coverage determinations. The objective of this study was to illustrate potential efficiency gains from reallocating Medicare expenditures by using cost-effectiveness information, and the consequences for health gains among Medicare beneficiaries. METHODS: We included national coverage determinations from 1999 through 2007. Estimates of cost-effectiveness were identified through a literature review. For coverage decisions with an associated cost-effectiveness estimate, we estimated utilization and size of the "unserved" eligible population by using a Medicare claims database (2007) and diagnostic and reimbursement codes. Technology costs originated from the cost-effectiveness literature or were estimated by using reimbursement codes. We illustrated potential aggregate health gains from increasing utilization of dominant interventions (i.e., cost saving and health increasing) and from reallocating expenditures by decreasing investment in cost-ineffective interventions and increasing investment in relatively cost-effective interventions. RESULTS: Complete information was available for 36 interventions. Increasing investment in dominant interventions alone led to an increase of 270,000 quality-adjusted life-years (QALYs) and savings of $12.9 billion. Reallocation of a broader array of interventions yielded an additional 1.8 million QALYs, approximately 0.17 QALYs per affected Medicare beneficiary. Compared with the distribution of resources prior to reallocation, following reallocation a greater proportion was directed to oncology, diagnostic imaging/tests, and the most prevalent diseases. A smaller proportion of resources went to cardiology, treatments (including drugs, surgeries, and medical devices, as opposed to nontreatments such as preventive services), and the least prevalent diseases. CONCLUSIONS: Using cost-effectiveness information has the potential to increase the aggregate health of Medicare beneficiaries while maintaining existing spending levels.

Factors predicting Medicare national coverage: an empirical analysis.

BACKGROUND: Interventions considered to be particularly controversial or expected to significantly impact the Medicare program in the United States are considered in National Coverage Determinations. Medicare coverage for such interventions is limited to those deemed "reasonable and necessary" for the diagnosis or treatment of an illness or injury. What constitutes reasonable and necessary has not, however, been clearly defined. OBJECTIVE: To determine factors associated with positive National Coverage Determinations. RESEARCH DESIGN: A dataset of coverage decisions from 1999 to 2007 (n=195) was created with the following variables: direction of coverage decision; quality of supporting evidence; availability of alternative interventions; cost-effectiveness of intervention; type of intervention; coverage requestor; and year of decision. Univariate and multivariate logistic regression analysis was used to determine factors associated with positive coverage. RESULTS: The following variables were independently associated with positive Medicare coverage: good or fair quality supporting evidence (adjusted odds ratio, OR=6.04, P<0.01); presence of an alternative intervention (OR=0.130, P<0.01); no associated estimate of cost-effectiveness (OR=0.190, P<0.05). In addition, in comparison with coverage decisions made in the years 1999 to 2001, those made in the years 2002 to 2003, 2004 to 2005, and 2006 to 2007, were associated with positive coverage [ORs of 0.311 (P<0.05), 0.310 (P<0.1), and 0.109 (P<0.01), respectively]. CONCLUSIONS: Findings suggest that good or fair quality supporting evidence is a strong predictor of positive coverage. Availability of alternative interventions, more recent decisions, and lack of an associated estimate of cost-effectiveness are associated with a decreased likelihood of positive coverage. The findings highlight Medicare's move to evidence-based coverage decisions, and suggest that coverage decisions are influenced by the availability of cost-effectiveness evidence.

A review of health utilities using the EQ-5D in studies of cardiovascular disease.

BACKGROUND: The EQ-5D has been extensively used to assess patient utility in trials of new treatments within the cardiovascular field. The aims of this study were to review evidence of the validity and reliability of the EQ-5D, and to summarise utility scores based on the use of the EQ-5D in clinical trials and in studies of patients with cardiovascular disease. METHODS: A structured literature search was conducted using keywords related to cardiovascular disease and EQ-5D. Original research studies of patients with cardiovascular disease that reported EQ-5D results and its measurement properties were included. RESULTS: Of 147 identified papers, 66 met the selection criteria, with 10 studies reporting evidence on validity or reliability and 60 reporting EQ-5D responses (VAS or self-classification). Mean EQ-5D index-based scores ranged from 0.24 (SD 0.39) to 0.90 (SD 0.16), while VAS scores ranged from 37 (SD 21) to 89 (no SD reported). Stratification of EQ-5D index scores by disease severity revealed that scores decreased from a mean of 0.78 (SD 0.18) to 0.51 (SD 0.21) for mild to severe disease in heart failure patients and from 0.80 (SD 0.05) to 0.45 (SD 0.22) for mild to severe disease in angina patients. CONCLUSIONS: The published evidence generally supports the validity and reliability of the EQ-5D as an outcome measure within the cardiovascular area. This review provides utility estimates across a range of cardiovascular subgroups and treatments that may be useful for future modelling of utilities and QALYs in economic evaluations within the cardiovascular area.

Mapping of the EQ-5D index from clinical outcome measures and demographic variables in patients with coronary heart disease.

BACKGROUND: The EuroQoL 5D (EQ-5D) is a questionnaire that provides a measure of utility for cost-effectiveness analysis. The EQ-5D has been widely used in many patient groups, including those with coronary heart disease. Studies often require patients to complete many questionnaires and the EQ-5D may not be gathered. This study aimed to assess whether demographic and clinical outcome variables, including scores from a disease specific measure, the Seattle Angina Questionnaire (SAQ), could be used to predict, or map, the EQ-5D index value where it is not available. METHODS: Patient-level data from 5 studies of cardiac interventions were used. The data were split into two groups - approximately 60% of the data were used as an estimation dataset for building models, and 40% were used as a validation dataset. Forward ordinary least squares linear regression methods and measures of prediction error were used to build a model to map to the EQ-5D index. Age, sex, a proxy measure of disease stage, Canadian Cardiovascular Society (CCS) angina severity class, treadmill exercise time (ETT) and scales of the SAQ were examined. RESULTS: The exertional capacity (ECS), disease perception (DPS) and anginal frequency scales (AFS) of the SAQ were the strongest predictors of the EQ-5D index and gave the smallest root mean square errors. A final model was chosen with age, gender, disease stage and the ECS, DPS and AFS scales of the SAQ. ETT and CCS did not improve prediction in the presence of the SAQ scales. Bland-Altman agreement between predicted and observed EQ-5D index values was reasonable for values greater than 0.4, but below this level predicted values were higher than observed. The 95% limits of agreement were wide (-0.34, 0.33). CONCLUSIONS: Mapping of the EQ-5D index in cardiac patients from demographics and commonly measured cardiac outcome variables is possible; however, prediction for values of the EQ-5D index below 0.4 was not accurate. The newly designed 5-level version of the EQ-5D with its increased ability to discriminate health states may improve prediction of EQ-5D index values.

Relationship between the EQ-5D index and measures of clinical outcomes in selected studies of cardiovascular interventions.

BACKGROUND: The EuroQoL 5D (EQ-5D) has been widely used in studies of cardiac disease, but its measurement properties in this group are not well established. The study aimed to quantify the relationship between measures commonly used in studies of cardiac disease and the EQ-5D index across different levels of disease severity. METHODS: Patient-level data from 7 studies of cardiac interventions were used, which included randomised trials and observational studies. Relationships between the EQ-5D index and commonly used cardiac measures, Canadian Cardiovascular Society (CCS) angina severity class, treadmill exercise time (ETT) and scales of the Seattle Angina Questionnaire (SAQ) were examined. Mixed effects linear regression was used to assess these relationships, with the EQ-5D index as the response. RESULTS: Study sample sizes ranged from 68 to 2419. Mean baseline EQ-5D index ranged from 0.77 in patients at diagnosis (95% CI 0.75, 0.78) to 0.43 in patients with advanced disease (95% CI 0.39, 0.48) and differed significantly across studies (p < 0.001). There was evidence of a ceiling effect in patients at diagnosis. The minimum clinically important difference of a one minute increase in ETT was associated with a 0.019 (95% CI 0.014, 0.025) increase in EQ-5D index. One class increase in CCS was associated with a 0.11 (95% CI 0.09, 0.13) decrease in EQ-5D index. A 10 unit increase in SAQ scales was associated with increases between 0.04 and 0.07 in EQ-5D index (95% CIs 0.03, 0.05 and 0.05, 0.08). Tests of heterogeneity indicated the EQ-5D-covariate relationships were consistent across levels of disease severity for ETT and the treatment satisfaction scale of the SAQ, but heterogeneous for age, gender, CCS angina class and other scales of the SAQ. CONCLUSION: The EQ-5D index varies with coronary disease severity. The relationship between the EQ-5D index and an outcome measure used in cardiac intervention studies, ETT, was consistent across disease severity levels, but the relationship between demographic variables, CCS angina class and most of the SAQ scales and the EQ-5D index was heterogeneous for patients with different levels of coronary disease. Differences in the EQ-5D index associated with clinically important differences in cardiac measures can be quantified and vary between three important examples - angina class, ETT and SAQ.

[Developing and applying the Payback Framework to assess the socioeconomic impact of health research]. / Desarrollo y aplicación del Modelo Payback para la evaluación del impacto socioeconómico de la investigación en salud.

There is increasing pressure for assessments of the wider socioeconomic impacts of health research. Governments are making greater demands to justify the expenditure of public money. However, there is also a belief that assessing how the wider effects or benefits of health research arise should help to inform the management and organization of health research so as to increase future impacts. Since the mid- 1990s, Buxton and Hanney at the Health Economics Research Group, Brunel University, have been developing and applying the Payback Framework to assess the impacts of health research. Together with their colleagues, these researchers have applied this model in a series of studies to assess the payback from research programs in various fields (including diabetes, arthritis and cardiovascular disease) and in various countries (including the United Kingdom, The Netherlands, Ireland, Australia and Canada). Other teams of researchers have applied the Payback Framework in, for example, Spain and Hong Kong. The Payback Framework consists of two elements, the first being the multi-dimensional categorization of the benefits of health research, which covers five main categories ranging from traditional knowledge production and research training and targeting, to impacts on policy and product development through to health and economic gains. The second element is a logic model of how best to assess these impacts. Application of this framework can be resource intensive, but has provided illustrative 'good news' stories on the payback resulting from research and has helped to inform research management.

The role of the national general medical journal: surveys of which journals UK clinicians read to inform their clinical practice.

BACKGROUND: For biomedical research findings to contribute toward health gains they must reach clinicians. Academic journals have historically been considered important information sources. Birken and Parkin found seven journals to most consistently contain the best pediatric evidence and, of these seven, four were general medical journals. METHODS: We surveyed clinicians in three UK medical specialties (psychiatry, surgery and pediatrics), asking which journals they read and which they considered important to inform their clinical practice. RESULTS: The readership of general medical journals, in comparison to specialty and sub-specialty journals, is widespread across the three UK medical specialties, although the importance of general medical journals varies widely. The BMJ is the most prominent general medical journal in terms of readership and importance but a dominant specialty or sub-specialty journal was usually more important for most groups. The Lancet is less widely read and less important, although more academics than non-academics consider it important. CONCLUSIONS: Overall, key general medical journals play an important role. Journal availability and cost, particularly in relation to membership for UK clinicians, and the position of academics and non-academics have to be considered in any analysis. Three of the four general medical journals containing the best pediatric evidence were found to be widely read by UK pediatricians and two UK-based general medical journals, the BMJ and The Lancet, were also considered important in our survey. Further investigation of the reasons for the importance of a journal and studies that would allow international comparisons would provide greater input to the discussion.

Economic returns to medical research funding.

Following the publication of the final paper in a planned series of four studies estimating the economic returns from biomedical and health research, we reflect on what we have learnt from these types of assessment.

The information sources and journals consulted or read by UK paediatricians to inform their clinical practice and those which they consider important: a questionnaire survey.

BACKGROUND: Implementation of health research findings is important for medicine to be evidence-based. Previous studies have found variation in the information sources thought to be of greatest importance to clinicians but publication in peer-reviewed journals is the traditional route for dissemination of research findings. There is debate about whether the impact made on clinicians should be considered as part of the evaluation of research outputs. We aimed to determine first which information sources are generally most consulted by paediatricians to inform their clinical practice, and which sources they considered most important, and second, how many and which peer-reviewed journals they read. METHODS: We inquired, by questionnaire survey, about the information sources and academic journals that UK medical paediatric specialists generally consulted, attended or read and considered important to their clinical practice. RESULTS: The same three information sources--professional meetings & conferences, peer-reviewed journals and medical colleagues--were, overall, the most consulted or attended and ranked the most important. No one information source was found to be of greatest importance to all groups of paediatricians. Journals were widely read by all groups, but the proportion ranking them first in importance as an information source ranged from 10% to 46%. The number of journals read varied between the groups, but Archives of Disease in Childhood and BMJ were the most read journals in all groups. Six out of the seven journals previously identified as containing best paediatric evidence are the most widely read overall by UK paediatricians, however, only the two most prominent are widely read by those based in the community. CONCLUSION: No one information source is dominant, therefore a variety of approaches to Continuing Professional Development and the dissemination of research findings to paediatricians should be used. Journals are an important information source. A small number of key ones can be identified and such analysis could provide valuable additional input into the evaluation of clinical research outputs.

An economic evaluation of pediatric small bowel transplantation in the United Kingdom.

BACKGROUND: Small bowel transplantation (SBTx) offers an alternative to parenteral nutrition (PN) for the treatment of chronic intestinal failure in children: this study estimated its cost-effectiveness in the early phase of a U.K. program. METHODS: Children assessed for SBTx were categorized as: 1) requiring SBTx following PN-related complications (n=23), 2) stable at home not requiring SBTx (n=24), and 3) terminally ill and unsuitable for SBTx (n=6). Costs were estimated from detailed resource-use data. Two comparisons were used for effectiveness: actual survival following transplantation (n=14) compared to: 1) estimated survival without transplantation using a prognostic model, and 2) the waiting list experiences of all patients listed for SBTx (n=23). RESULTS: Mean costs up to 30 months were pounds sterling 207,000 for those transplanted or on the waiting list, pounds sterling 159,000 for those stable on home PN, and pounds sterling 56,000 for those terminally ill. The prognostic model estimated a mean survival gain from transplantation of 0.12 years over 30 months, and suggested that transplantation was cost-saving. The second approach suggested that transplantation reduced survival by 0.24 years at an additional cost of pounds sterling 131,000. CONCLUSIONS: Firm conclusions on cost-effectiveness of SBTx are not possible given the two different estimates. The prognostic model approach (suggesting that pediatric SBTx may provide a small survival benefit at a small reduction in costs) should be less subject to bias, but the model requires external validation. Meanwhile, children at risk of fatal PN-complications should be given the opportunity to receive a SBTx only within a continuing formal assessment of the technology.

Economic evaluation and decision making in the UK.

This article reviews the development of economic evaluation of health technologies in the UK and its impact on decision making. After a long period of limited impact from studies mainly carried out as academic exercises, the advent of the National Institute for Health and Clinical Excellence (NICE) in 1999 provided a transparent decision-making context where economic evaluation plays a central role. This article reviews some of the key characteristics about the way NICE works, for example, the way NICE has defined the form of analysis that it requires, reflecting its objective of maximising health gain (QALYs) from the predetermined and limited UK NHS budget. Two broad areas of widespread concern are noted. The first relates to the cost-effectiveness thresholds that NICE uses and the basis for them. The second is the patchy implementation of NICE guidance and the possible reasons for this. But even within the UK, NICE is the exception in making extensive and explicit use of economic evaluation and this article goes on to suggest that if there is to be a more widespread and consistent use of economic evaluation at both central and local levels, then health economists and others need to address three issues. The first is to be clear about what is the correct conceptual basis for determining the cost-effectiveness threshold and then to ensure that NICE has the empirical evidence to set it appropriately. The second is to recognise that even using the limited view of costs adopted by NICE, economic evaluations imply temporal and cross-service budgetary flexibility that the NHS locally does not in practice enjoy. The third issue is that with academic pressures for ever-increasing sophistication of 'state of the art' economic evaluation analysis, the NHS has more and more precise understanding of the cost effectiveness of just a few new technologies and little or no analysis of most. This limits the value of the former by reducing further the scope for appropriately disinvesting from cost-ineffective technologies to meet the additional costs of investing in cost-effective new ones. Whilst NICE stands out as an example of a context where high-quality economic evaluation plays a major role in decision making, the process is far from perfect and certainly is not representative of the use made of economic evaluation by the NHS as a whole. Health economists need to engage with the public and the health service to better understand their perspectives, rather than focusing on academic concerns relating to details of theory and analytical method.

The journals of importance to UK clinicians: a questionnaire survey of surgeons.

BACKGROUND: Peer-reviewed journals are seen as a major vehicle in the transmission of research findings to clinicians. Perspectives on the importance of individual journals vary and the use of impact factors to assess research is criticised. Other surveys of clinicians suggest a few key journals within a specialty, and sub-specialties, are widely read. Journals with high impact factors are not always widely read or perceived as important. In order to determine whether UK surgeons consider peer-reviewed journals to be important information sources and which journals they read and consider important to inform their clinical practice, we conducted a postal questionnaire survey and then compared the findings with those from a survey of US surgeons. METHODS: A questionnaire survey sent to 2,660 UK surgeons asked which information sources they considered to be important and which peer-reviewed journals they read, and perceived as important, to inform their clinical practice. Comparisons were made with numbers of UK NHS-funded surgery publications, journal impact factors and other similar surveys. RESULTS: Peer-reviewed journals were considered to be the second most important information source for UK surgeons. A mode of four journals read was found with academics reading more than non-academics. Two journals, the BMJ and the Annals of the Royal College of Surgeons of England, are prominent across all sub-specialties and others within sub-specialties. The British Journal of Surgery plays a key role within three sub-specialties. UK journals are generally preferred and readership patterns are influenced by membership journals. Some of the journals viewed by surgeons as being most important, for example the Annals of the Royal College of Surgeons of England, do not have high impact factors. CONCLUSION: Combining the findings from this study with comparable studies highlights the importance of national journals and of membership journals. Our study also illustrates the complexity of the link between the impact factors of journals and the importance of the journals to clinicians. This analysis potentially provides an additional basis on which to assess the role of different journals, and the published output from research.

Estimating utility data from clinical indicators for patients with stable angina.

This study estimated a model from which data routinely collected in clinical trials of angina patients can be mapped to a utility scale and used to estimate quality-adjusted life years (QALYs). Patients with stable angina attending four cardiac out-patient clinics in the UK were included in the study. Data collected included information on patients' health-related quality of life (HRQL) using the EQ-5D, and severity of angina symptoms using two cardiac-specific measures [Breathlessness Grade and Canadian Cardiovascular Society (CCS) classification of angina]. Regression analysis was used to predict EQ-5D index values from the data. Data were obtained from 510 patients. For CCS grades, mean EQ-5D scores ranged from 0.36 (95% confidence interval 0.25-0.48) for grade 4 to 0.81 (0.77-0.85) for grade 0, and for breathlessness grades, EQ-5D scores ranged from 0.31 (0.06-0.55) for grade 0 to 0.84 (0.79-0.88) for grade 5. The final model used data on CCS grades, breathlessness grades, and patients' current medications to predict EQ-5D scores. The model had an R2 value of 0.37, and predictions for less severe angina were considered more reliable than the estimates for severe angina. In the absence of utility data collected as part of a clinical trial it is possible to map HRQL utility data from samples of patients with similar characteristics to those in the original trial. The uncertainty surrounding the estimates should be considered when using the results to estimate QALYs for purposes of economic evaluation.

Trial-based cost-utility comparison of percutaneous myocardial laser revascularisation and continued medical therapy for treatment of refractory angina pectoris.

This study examined from a health service perspective whether percutaneous myocardial laser revascularisation (PMR) plus standard medical management is cost-effective when compared with standard medical management alone in the treatment of refractory angina. This involved a cost-utility analysis using patient-specific data from a single-centre, randomised, controlled trial carried out in the United Kingdom. Of 73 patients diagnosed as having refractory angina and not suitable for conventional forms of revascularisation, 36 were randomised to PMR plus medical management and 37 to medical management alone. We collected costs to the health service of PMR and all secondary sector health care contacts and cardiac-related medication in the 12 months following randomisation. Patient utility, measured using the EuroQol EQ-5D questionnaire, was combined with 12-month survival data to generate quality-adjusted life years (QALYs). The mean 12-month cost per patient for PMR was 8,307 pounds, and that for medical management was 1,888 pounds, giving a cost difference of 6,410 pounds. The mean QALY difference favoured PMR at 0.126, giving an incremental cost per QALY of 50,873 pounds. The cost-effectiveness acceptability curve indicates that the probability of PMR being cost-effective over the first 12 months is quite low. Whilst a longer period of follow-up might indicate continued benefit from PMR, which would make the intervention economically more attractive, PMR could not be considered cost-effective based on 1-year follow-up data.

Survival gain after liver transplantation for patients with alcoholic liver disease: a comparison across models and centers.

BACKGROUND: Prognostic models, which estimate survival probabilities, enable the survival benefit from a procedure to be assessed and allow the comparison of outcomes between centers. We therefore compared three models that estimate survival in the absence of transplantation for patients with alcoholic liver disease (ALD) to assess survival benefit after transplantation and to compare the outcomes in the six liver-transplant centers in England. METHODS: The Béclére and Birmingham models and the Model for End-Stage Liver Disease (MELD) were used to estimate survival in the absence of transplantation in a cohort of 82 patients with end-stage ALD. Posttransplant survival in the same cohort of patients was calculated using conventional survival techniques. Each individual's short-term survival gain after liver transplantation was also calculated and compared across UK liver transplant centers. RESULTS: The expected gain in survival differed substantially depending on the model used. Over the 4-year study period, the survival gain for all ALD patients was 1.70 years (95% confidence interval [CI] 1.37-2.03) using the Béclére model, 0.95 years (CI 0.60-1.30) using MELD, and 0.08 years (CI -0.31-0.47) using the Birmingham model. Two centers consistently had greater estimated survival gains up to 4 years postliver transplant regardless of the model used to estimate nontransplant survival. CONCLUSIONS: These findings suggest that although liver transplantation is associated with an improvement in survival, the gain over 4 years is modest. The three models are poorly correlated and should be applied with caution. Survival gain does, however, appear to vary between centers.

Measures of importance for economic analysis based on decision modeling.

In probabilistic economic analysis, the uncertainty concerning input parameters is quantified, and determines the level of uncertainty over the optimal decision. Researchers from a wide range of disciplines employ mathematical models to simulate complex processes. Common through many such disciplines is the conduct of importance analysis to determine those input parameters that contribute most to the uncertainty over the optimal decision based on the results of the analysis. In this study, we compare a range of potential importance measures to see how they compare with methods used in economic analysis. Techniques were classified as variance/correlation, information, probability, entropy, or elasticity-based measures. A selection of the most commonly used measures were applied to an economic model of treatment for patients with Parkinson's disease. Techniques were evaluated in terms of their ranking of variables, complexity, and interpretation.

Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign'

BACKGROUND: External and internal factors are increasingly encouraging research funding bodies to demonstrate the outcomes of their research. Traditional methods of assessing research are still important, but can be merged into broader multi-dimensional categorisations of research benefits. The onus has hitherto been on public sector funding bodies, but in the UK the role of medical charities in funding research is particularly important and the Arthritis Research Campaign, the leading medical charity in its field in the UK, commissioned a study to identify the outcomes from research that it funds. This article describes the methods to be used. METHODS: A case study approach will enable narratives to be told, illuminating how research funded in the early 1990s was (or was not) translated into practice. Each study will be organised using a common structure, which, with careful selection of cases, should enable cross-case analysis to illustrate the strengths of different modes and categories of research. Three main interdependent methods will be used: documentary and literature review; semi-structured interviews; and bibliometric analysis. The evaluative framework for organising the studies was previously used for assessing the benefits from health services research. Here, it has been specifically amended for a medical charity that funds a wide range of research and is concerned to develop the careers of researchers. It was further refined in three pilot studies. The framework has two main elements. First, a multi-dimensional categorisation of benefits going from the knowledge produced in peer reviewed journal articles through to the health and potential economic gain. The second element is a logic model, which, with various stages, should provide a way of organising the studies. The stock of knowledge is important: much research, especially basic, will feed into it and influence further research rather than directly lead to health gains. The cross-case analysis will look for factors associated with outcomes. CONCLUSIONS: The pilots confirmed the applicability of the methods for a full study which should assist the Arthritis Research Campaign to demonstrate the outcomes from its funding, and provide it with evidence to inform its own policies.

The utilisation of health research in policy-making: concepts, examples and methods of assessment.

The importance of health research utilisation in policy-making, and of understanding the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. Key utilisation issues have been described for at least twenty years, but the growing focus on health research systems creates additional dimensions.The utilisation of health research in policy-making should contribute to policies that may eventually lead to desired outcomes, including health gains. In this article, exploration of these issues is combined with a review of various forms of policy-making. When this is linked to analysis of different types of health research, it assists in building a comprehensive account of the diverse meanings of research utilisation.Previous studies report methods and conceptual frameworks that have been applied, if with varying degrees of success, to record utilisation in policy-making. These studies reveal various examples of research impact within a general picture of underutilisation.Factors potentially enhancing utilisation can be identified by exploration of: priority setting; activities of the health research system at the interface between research and policy-making; and the role of the recipients, or 'receptors', of health research. An interfaces and receptors model provides a framework for analysis.Recommendations about possible methods for assessing health research utilisation follow identification of the purposes of such assessments. Our conclusion is that research utilisation can be better understood, and enhanced, by developing assessment methods informed by conceptual analysis and review of previous studies.

Understanding factors associated with the translation of cardiovascular research: a multinational case study approach.

BACKGROUND: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. METHODS: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. RESULTS: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. CONCLUSIONS: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes.