Evidence gaps in economic analyses of hearing healthcare: A systematic review.

BACKGROUND: Hearing loss is a common and costly medical condition. This systematic review sought to identify evidence gaps in published model-based economic analyses addressing hearing loss to inform model development for an ongoing Lancet Commission. METHODS: We searched the published literature through 14 June 2020 and our inclusion criteria included decision model-based cost-effectiveness analyses that addressed diagnosis, treatment, or prevention of hearing loss. Two investigators screened articles for inclusion at the title, abstract, and full-text levels. Data were abstracted and the studies were assessed for the qualities of model structure, data assumptions, and reporting using a previously published quality scale. FINDINGS: Of 1437 articles identified by our search, 117 unique studies met the inclusion criteria. Most of these model-based analyses were set in high-income countries (n = 96, 82%). The evaluated interventions were hearing screening (n = 35, 30%), cochlear implantation (n = 34, 29%), hearing aid use (n = 28, 24%), vaccination (n = 22, 19%), and other interventions (n = 29, 25%); some studies included multiple interventions. Eighty-six studies reported the main outcome in quality-adjusted or disability-adjusted life-years, 24 of which derived their own utility values. The majority of the studies used decision tree (n = 72, 62%) or Markov (n = 41, 35%) models. Forty-one studies (35%) incorporated indirect economic effects. The median quality rating was 92/100 (IQR:72-100). INTERPRETATION: The review identified a large body of literature exploring the economic efficiency of hearing healthcare interventions. However, gaps in evidence remain in evaluation of hearing healthcare in low- and middle-income countries, as well as in investigating interventions across the lifespan. Additionally, considerable uncertainty remains around productivity benefits of hearing healthcare interventions as well as utility values for hearing-assisted health states. Future economic evaluations could address these limitations. FUNDING: NCATS 3UL1-TR002553-03S3.

The Cost-Effectiveness of Palliative Care: Insights from the PAL-HF Trial.

BACKGROUND: In a randomized control trial, Palliative Care in Heart Failure (PAL-HF) improved heart failure-related quality of life, though cost-effectiveness remains unknown. The aim of this study was to evaluate the cost-effectiveness of the PAL-HF trial, which provided outpatient palliative care to patients with advanced heart failure. METHODS AND RESULTS: Outcomes for usual care and PAL-HF strategies were compared using a Markov cohort model over 36 months from a payer perspective. The model parameters were informed by PAL-HF trial data and supplemented with meta-analyses and Medicare administrative data. Outcomes included hospitalization, place of death, Medicare expenditures, quality-adjusted life years (QALYs), and incremental cost-effectiveness ratios. Simulated mortality rates were the same for PAL-HF and usual care cohorts, at 89.7% at 36 months. In the base case analysis, the PAL-HF intervention resulted in an incremental gain of 0.03 QALYs and an incremental cost of $964 per patient for an incremental cost-effectiveness ratio of $29,041 per QALY. In 1-way sensitivity analyses, an intervention cost of up to $140 per month is cost effective at $50,000 per QALY. Of 1000 simulations, the PC intervention had a 66.1% probability of being cost effective at a $50,000 willingness-to-pay threshold assuming no decrease in hospitalization. In a scenario analysis, PAL-HF decreased payer spending through reductions in noncardiovascular hospitalizations. CONCLUSIONS: These results from this single-center trial are encouraging that palliative care for advanced heart failure is an economically attractive intervention. Confirmation of these findings in larger multicenter trials will be an important part of developing the evidence to support more widespread implementation of the PAL-HF palliative care intervention.

Future Directions for Cost-effectiveness Analyses in Health and Medicine.

OBJECTIVES: In 2016, the Second Panel on Cost-effectiveness in Health and Medicine updated the seminal work of the original panel from 2 decades earlier. The Second Panel had an opportunity to reflect on the evolution of cost-effectiveness analysis (CEA) and to provide guidance for the next generation of practitioners and consumers. In this article, we present key topics for future research and policy. METHODS: During the course of its deliberations, the Second Panel discussed numerous topics for advancing methods and for improving the use of CEA in decision making. We identify and consider 7 areas for which the panel believes that future research would be particularly fruitful. In each of these areas, we highlight outstanding research needs. The list is not intended as an exhaustive inventory but rather a set of key items that surfaced repeatedly in the panel's discussions. In the online Appendix , we also list and expound briefly on 8 other important topics. RESULTS: We highlight 7 key areas: CEA and perspectives (determining, valuing, and summarizing elements for the analysis), modeling (comparative modeling and model transparency), health outcomes (valuing temporary health and path states, as well as health effects on caregivers), costing (a cost catalogue, valuing household production, and productivity effects), evidence synthesis (developing theory on learning across studies and combining data from clinical trials and observational studies), estimating and using cost-effectiveness thresholds (empirically representing 2 broad concepts: opportunity costs and public willingness to pay), and reporting and communicating CEAs (written protocols and a quality scoring system). CONCLUSIONS: Cost-effectiveness analysis remains a flourishing and evolving field with many opportunities for research. More work is needed on many fronts to understand how best to incorporate CEA into policy and practice.

How well does early-career investigators' cardiovascular outcomes research training align with funded outcomes research?

BACKGROUND: Outcomes research training programs should prepare trainees to successfully compete for research funding. We examined how early-career investigators' prior and desired training aligns with recently funded cardiovascular (CV) outcomes research. METHODS: We (1) reviewed literature to identify 13 core competency areas in CV outcomes research; (2) surveyed early-career investigators to understand their prior and desired training in each competency area; (3) examined recently funded grants commonly pursued by early-career outcomes researchers to ascertain available funding in competency areas; and (4) analyzed alignment between investigator training and funded research in each competency area. We evaluated 185 survey responses from early-career investigators (response rate 28%) and 521 funded grants from 2010 to 2014. RESULTS: Respondents' prior training aligned with funded grants in the areas of clinical epidemiology, observational research, randomized controlled trials, and implementation/dissemination research. Funding in community-engaged research and health informatics was more common than prior training in these areas. Respondents' prior training in biostatistics and systematic review was more common than funded grants focusing on these specific areas. Respondents' desired training aligned similarly with funded grants, with some exceptions; for example, desired training in health economics/cost-effectiveness research was more common than funded grants in these areas. Restricting to CV grants (n=132) and National Heart, Lung, and Blood Institute-funded grants (n=170) produced similar results. CONCLUSIONS: Identifying mismatch between funded grants in outcomes research and early-career investigators' prior/desired training may help efforts to harmonize investigator interests, training, and funding. Our findings suggest a need for further consideration of how to best prepare early-career investigators for funding success.

Advancing Value Assessment in the United States: A Multistakeholder Perspective.

Rising costs without perceived proportional improvements in quality and outcomes have motivated fundamental shifts in health care delivery and payment to achieve better value. Aligned with these efforts, several value assessment frameworks have been introduced recently to help providers, patients, and payers better understand the potential value of drugs and other interventions and make informed decisions about their use. Given their early stage of development, it is imperative to evaluate these efforts on an ongoing basis to identify how best to support and improve them moving forward. This article provides a multistakeholder perspective on the key limitations and opportunities posed by the current value assessment frameworks and areas of and actions for improvement. In particular, we outline 10 fundamental guiding principles and associated strategies that should be considered in subsequent iterations of the existing frameworks or by emerging initiatives in the future. Although value assessment frameworks may not be able to meet all the needs and preferences of stakeholders, we contend that there are common elements and potential next steps that can be supported to advance value assessment in the United States.

Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses: Second Panel on Cost-Effectiveness in Health and Medicine.

IMPORTANCE: Since publication of the report by the Panel on Cost-Effectiveness in Health and Medicine in 1996, researchers have advanced the methods of cost-effectiveness analysis, and policy makers have experimented with its application. The need to deliver health care efficiently and the importance of using analytic techniques to understand the clinical and economic consequences of strategies to improve health have increased in recent years. OBJECTIVE: To review the state of the field and provide recommendations to improve the quality of cost-effectiveness analyses. The intended audiences include researchers, government policy makers, public health officials, health care administrators, payers, businesses, clinicians, patients, and consumers. DESIGN: In 2012, the Second Panel on Cost-Effectiveness in Health and Medicine was formed and included 2 co-chairs, 13 members, and 3 additional members of a leadership group. These members were selected on the basis of their experience in the field to provide broad expertise in the design, conduct, and use of cost-effectiveness analyses. Over the next 3.5 years, the panel developed recommendations by consensus. These recommendations were then reviewed by invited external reviewers and through a public posting process. FINDINGS: The concept of a "reference case" and a set of standard methodological practices that all cost-effectiveness analyses should follow to improve quality and comparability are recommended. All cost-effectiveness analyses should report 2 reference case analyses: one based on a health care sector perspective and another based on a societal perspective. The use of an "impact inventory," which is a structured table that contains consequences (both inside and outside the formal health care sector), intended to clarify the scope and boundaries of the 2 reference case analyses is also recommended. This special communication reviews these recommendations and others concerning the estimation of the consequences of interventions, the valuation of health outcomes, and the reporting of cost-effectiveness analyses. CONCLUSIONS AND RELEVANCE: The Second Panel reviewed the current status of the field of cost-effectiveness analysis and developed a new set of recommendations. Major changes include the recommendation to perform analyses from 2 reference case perspectives and to provide an impact inventory to clarify included consequences.

An overview and discussion of the Patient-Centered Outcomes Research Institute's decision aid portfolio.

Decision aids (DAs) help patients make informed healthcare decisions in a manner consistent with their values and preferences. Despite their promise, DAs developed with public research dollars are not being implemented and adopted in real-world patient care settings at a rate consistent with which they are being developed. To appraise the sum of the parts of the portfolio and create a strategic imperative surrounding future funding, the Patient-Centered Outcomes Research Institute (PCORI) tasked the Duke Evidence Synthesis Group with evaluating its DA portfolio. This paper describes PCORI's portfolio of DAs according to the Duke Evidence Synthesis Group's analysis in the context of PCORI's mission and the field of decision science. The results revealed a diversity within PCORI's portfolio of funded DA projects. Findings support the movement toward more rigorous DA development, assessment and maintenance. PCORI's funding priorities related to DAs are clarified and comparative questions of interest are posed.

Comparative Effectiveness of Implantable Cardioverter Defibrillators for Primary Prevention in Women.

BACKGROUND: Clinical trials of implantable cardioverter defibrillators (ICDs) for primary prevention enrolled a limited number of women. We sought to examine clinical practice data to compare survival rates among women with heart failure with or without a primary prevention ICD. METHODS AND RESULTS: We linked data from 264 US hospitals included in the Get With The Guidelines for Heart Failure registry with data from the Centers for Medicare and Medicaid Services. From these sources, we propensity score matched 430 women with heart failure who received a primary prevention ICD to 430 women who did not; we further adjusted using a Cox proportional hazards model. Median follow-up was 3.4 and 3.0 years. For comparison, we matched 859 men receiving an ICD with 859 who did not; median follow-up was 3.9 versus 2.9 years. In the matched cohorts, an ICD was associated with similarly better survival in women (hazard ratio, 0.78; 95% confidence interval, 0.66-0.92; P=0.003) and men (hazard ratio, 0.76; 95% confidence interval, 0.67-0.87 P<0.001). There was no interaction between sex and presence of an ICD with respect to survival (P=0.79). CONCLUSIONS: Among patients with heart failure with reduced left ventricular ejection fraction, a primary prevention ICD was associated with a significant survival advantage among women and among men. These findings support guideline-directed use of primary prevention ICDs in eligible patients.

Survival after primary prevention implantable cardioverter-defibrillator placement among patients with chronic kidney disease.

BACKGROUND: Guidelines recommend that implantable cardioverter-defibrillator (ICD) candidates have an estimated longevity of ≥1 year. Longevity can be affected by chronic kidney disease (CKD). METHODS AND RESULTS: Using the National Cardiovascular Data Registry ICD registry linked with the Social Security Death Master File, we assessed the rate of death after primary prevention ICD placement between January 1, 2006, and December 31, 2007, according to CKD stage. Using Cox models, we identified factors associated with death among patients with CKD. Compared with patients without CKD (n=26,056), those with CKD (n=21,226) were older, less commonly men, more often white, and more frequently had comorbid illness. Compared with patients without CKD, patients with a glomerular filtration rate 30 to 60, glomerular filtration rate <30, and end-stage renal disease on dialysis had a higher risk of death after ICD placement (hazard ratio, 2.08; 95% confidence interval, 1.99-2.18; P<0.0001; hazard ratio, 4.20; 95% confidence interval, 3.92-4.50; P<0.0001; and hazard ratio, 4.80; 95% confidence interval, 4.46-5.17; P<0.0001, respectively). Corresponding 1-year death rates were 4.4%, 9.1%, 20.2%, and 22.4%. Among patients with CKD, factors associated with increased risk of death included CKD severity, age >65 years, heart failure symptoms, diabetes mellitus, lung disease, serum sodium <140 mEq/L, atrial fibrillation or flutter, and a lower ejection fraction. CONCLUSIONS: The risk of death after primary prevention ICD placement is proportional to CKD severity. Among patients with CKD, several factors are prognostically significant and could inform clinical decision making on primary prevention ICD candidacy.

Association between prophylactic implantable cardioverter-defibrillators and survival in patients with left ventricular ejection fraction between 30% and 35%.

IMPORTANCE: Clinical trials of prophylactic implantable cardioverter-defibrillators (ICDs) have included a minority of patients with a left ventricular ejection fraction (LVEF) between 30% and 35%. Because a large number of ICDs in the United States are implanted in such patients, it is important to study survival associated with this therapy. OBJECTIVE: To characterize patients with LVEF between 30% and 35% and compare the survival of those with and without ICDs. DESIGN, SETTING, AND PARTICIPANTS: Retrospective cohort study of Medicare beneficiaries in the National Cardiovascular Data Registry ICD registry (January 1, 2006, through December 31, 2007) with an LVEF between 30% and 35% who received an ICD during a heart failure hospitalization and similar patients in the Get With The Guidelines-Heart Failure (GWTG-HF) database (January 1, 2005, through December 31, 2009) with no ICD. The analysis was repeated in patients with an LVEF less than 30%. There were 3120 patients with an LVEF between 30% and 35% (816 in matched cohorts) and 4578 with an LVEF less than 30% (2176 in matched cohorts). Propensity score matching and Cox models were applied. MAIN OUTCOMES AND MEASURES: The primary outcome was all-cause mortality; data were obtained from Medicare claims through December 31, 2011. RESULTS: There were no significant differences in the baseline characteristics of the matched groups (n = 408 for both groups). Among patients with an LVEF between 30% and 35%, there were 248 deaths in the ICD Registry group, within a median follow-up of 4.4 years (interquartile range, 2.7-4.9) and 249 deaths in the GWTG HF group, within a median follow-up of 2.9 years (interquartile range, 2.1-4.4). The risk of all-cause mortality in patients with an LVEF between 30% and 35% and an ICD was significantly lower than that in matched patients without an ICD (3-year mortality rates: 51.4% vs 55.0%; hazard ratio, 0.83 [95% CI, 0.69-0.99]; P = .04). Presence of an ICD also was associated with better survival in patients with an LVEF less than 30% (3-year mortality rates: 45.0% vs 57.6%; 634 and 660 total deaths; hazard ratio, 0.72 [95% CI, 0.65-0.81]; P < .001) (P = .20 for interaction). CONCLUSIONS AND RELEVANCE: Among Medicare beneficiaries hospitalized for heart failure and with an LVEF between 30% and 35% and less than 30%, survival at 3 years was better in patients who received a prophylactic ICD than in comparable patients with no ICD. These findings support guideline recommendations to implant prophylactic ICDs in eligible patients with an LVEF of 35% or less.

Cost-effectiveness of newer antiretroviral drugs in treatment-experienced patients with multidrug-resistant HIV disease.

OBJECTIVE: Newer antiretroviral drugs provide substantial benefits but are expensive. The cost-effectiveness of using antiretroviral drugs in combination for patients with multidrug-resistant HIV disease was determined. DESIGN: A cohort state-transition model was built representing treatment-experienced patients with low CD4 counts, high viral load levels, and multidrug-resistant virus. The effectiveness of newer drugs (those approved in 2005 or later) was estimated from published randomized trials. Other parameters were estimated from a randomized trial and from the literature. The model had a lifetime time horizon and used the perspective of an ideal insurer in the United States. The interventions were combination antiretroviral therapy, consisting of 2 newer drugs and 1 conventional drug, compared with 3 conventional drugs. Outcome measures were life-years, quality-adjusted life-years (QALYs), costs, and incremental cost-effectiveness. RESULTS: Substituting newer antiretroviral drugs increased expected survival by 3.9 years in advanced HIV disease. The incremental cost-effectiveness ratio of newer, compared with conventional, antiretroviral drugs was $75,556/QALY gained. Sensitivity analyses showed that substituting only one newer antiretroviral drug cost $54,559 to $68,732/QALY, depending on assumptions about efficacy. Substituting 3 newer drugs cost $105,956 to $117,477/QALY. Cost-effectiveness ratios were higher if conventional drugs were not discontinued. CONCLUSIONS: In treatment-experienced patients with advanced HIV disease, use of newer antiretroviral agents can be cost-effective, given a cost-effectiveness threshold in the range of $50,000 to $75,000 per QALY gained. Newer antiretroviral agents should be used in carefully selected patients for whom less expensive options are clearly inferior.

Evaluating cough assessment tools: a systematic review.

BACKGROUND: Little is known about the comparative validity, reliability, or responsiveness of instruments for assessing cough frequency or impact, where the term impact encompasses both cough severity and the impact of cough on health-related quality of life. METHODS: We conducted a systematic review to evaluate instruments that assess cough frequency or impact in adults, adolescents, and children with acute or chronic cough. RESULTS: Seventy-eight studies were included, of which eight were randomized controlled trials and 70 were observational studies. In all age groups, audio and video electronic recording devices had good reliability compared with other methods of assessing cough frequency but had variable correlation with other cough assessments, such as visual analog scale scores, quality-of-life questionnaires, cough diaries, and tussigenic challenges. Among adult and adolescent patients, the Leicester Cough Questionnaire (LCQ) and the Cough-Specific Quality-of-Life Questionnaire (CQLQ) were valid and reliable, showing high intraclass and test-retest correlations. Among children, the Parent Cough-Specific Quality of Life Questionnaire and Pediatric Cough Questionnaire were valid and reliable. CONCLUSIONS: Electronic recording devices can be valid assessments of cough frequency. The LCQ and CQLQ for adults and the Parent Cough-Specific Quality of Life questionnaire for children are valid instruments for assessing cough impact. There is limited but insufficient evidence to determine the reliability or concurrent validity of the different types of cough diaries or visual analog scale scores. There are also limited data to support the responsiveness of recording devices. There is good responsiveness data for the LCQ and CQLQ, but more evidence is needed.

Oral contraceptive pills as primary prevention for ovarian cancer: a systematic review and meta-analysis.

OBJECTIVE: To estimate the overall reduction in ovarian cancer risk associated with the use of oral contraceptive pills (OCPs) and whether reduction in risk is affected by specifics of OCP use, such as formulation or duration of use. DATA SOURCES: We searched PubMed, Embase, the Cochrane Database of Systematic Reviews, and ClinicalTrials.gov for studies published from January 1990 to June 2012, with primary analysis of studies published since January 2000. METHODS OF STUDY SELECTION: We reviewed 6,476 citations. We included English-language controlled studies with human participants reporting a quantitative association between exposure to OCPs (in which the explicit or implicit indication for OCP use was prevention of pregnancy or ovarian cancer) compared with no use of OCPs. Two investigators independently reviewed the title and abstract and full-text of articles for inclusion or exclusion decision; discordant decisions were resolved by team review and consensus. TABULATION, INTEGRATION, AND RESULTS: Fifty-five studies met inclusion criteria. A random-effects meta-analysis of 24 case-control and cohort studies showed significant reduction in ovarian cancer incidence in ever-users compared with never-users (odds ratio 0.73, 95% confidence interval 0.66-0.81). There was a significant duration-response relationship, with reduction in incidence of more than 50% among women using OCPs for 10 or more years. The lifetime reduction in ovarian cancer attributable to the use of OCPs is approximately 0.54% for a number-needed-to-treat of approximately 185 for a use period of 5 years. CONCLUSION: Significant duration-dependent reductions in ovarian cancer incidence in the general population are associated with OCP use.

Assessment of the quality of existing patient educational tools focused on sudden cardiac arrest: a systematic evaluation by the Sudden Cardiac Arrest Thought Leadership Alliance.

BACKGROUND: Conveying contemporary treatment options for those at risk of sudden cardiac arrest (SCA) is challenging. The purpose of the present research was to evaluate the quality and usability of available patient educational tools relevant to SCA and its treatment options, such as implantable cardioverter defibrillators (ICDs). We hypothesized that this review would identify gaps in areas of information for the enhancement of patient education and decision-making materials. METHODS: We used a formal instrument to assess specific domains of content, development, and effectiveness of 18 available SCA and ICD educational tools. The multidisciplinary review panel included two electrophysiologists, two general cardiologists, a cardiac psychologist, a health services researcher, and a patient advocate. RESULTS: Of the 18 education tools, four were rated as "good, may need revisions, but sufficient for use", 12 were rated as "marginal, needs revision prior to use", and two were rated as "poor, inadequate for use". None of the tools were rated as being of "very good" or "excellent" quality. CONCLUSION: There appear to be opportunities to improve the quality and completeness of existing educational tools for patients with SCA and ICD. While many tools have been developed, they fall below current standards for supporting informed medical decision-making.

Priorities for comparative effectiveness reviews in cardiovascular disease.

BACKGROUND: Comparative effectiveness reviews offer a systematic method to critically appraise existing research and to identify unaddressed clinical areas in cardiovascular disease where significant morbidity, mortality, and variation in the use of resources persist. To delineate and help select areas where comparative effectiveness reviews are needed, the Effective Health Care Program of the Agency for Healthcare Research and Quality involved stakeholders in prioritization of the research agenda. METHODS AND RESULTS: We involved a diverse panel of stakeholders representing a broad range of clinical, policy, and patient perspectives. To assist in prioritization of topics for evidence synthesis, we created a framework evaluating 12 cardiovascular disease subcategories that reflect American College of Cardiology/American Heart Association disease-based guidelines. We performed an environmental scan for each disease subcategory to populate this framework with existing knowledge, levels of evidence, and degrees of public interest. Through a formalized process, 4 disease subcategories were prioritized: chronic coronary artery disease, ventricular arrhythmias, heart failure, and cerebrovascular disease. Within these subcategories, 11 topics that address the comparative safety and effectiveness of existing treatments and evaluate emerging treatments were nominated by the stakeholder panel to proceed for feasibility assessment before developing comparative effectiveness reviews. CONCLUSIONS: Using a systematic process deriving consensus from multiple stakeholders across cardiovascular disease states, we generated a prioritized list of evidence synthesis topics to inform decision makers. The topics vetted through this process seek to determine the comparative safety and effectiveness of a range of treatments, both established and emerging, and are immediately relevant for prevalent disease states.

Enabling health care decisionmaking through clinical decision support and knowledge management.

OBJECTIVES: To catalogue study designs used to assess the clinical effectiveness of CDSSs and KMSs, to identify features that impact the success of CDSSs/KMSs, to document the impact of CDSSs/KMSs on outcomes, and to identify knowledge types that can be integrated into CDSSs/KMSs. DATA SOURCES: MEDLINE(®), CINAHL(®), PsycINFO(®), and Web of Science(®). REVIEW METHODS: We included studies published in English from January 1976 through December 2010. After screening titles and abstracts, full-text versions of articles were reviewed by two independent reviewers. Included articles were abstracted to evidence tables by two reviewers. Meta-analyses were performed for seven domains in which sufficient studies with common outcomes were included. RESULTS: We identified 15,176 articles, from which 323 articles describing 311 unique studies including 160 reports on 148 randomized control trials (RCTs) were selected for inclusion. RCTs comprised 47.5 percent of the comparative studies on CDSSs/KMSs. Both commercially and locally developed CDSSs effectively improved health care process measures related to performing preventive services (n = 25; OR 1.42, 95% confidence interval [CI] 1.27 to 1.58), ordering clinical studies (n = 20; OR 1.72, 95% CI 1.47 to 2.00), and prescribing therapies (n = 46; OR 1.57, 95% CI 1.35 to 1.82). Fourteen CDSS/KMS features were assessed for correlation with success of CDSSs/KMSs across all endpoints. Meta-analyses identified six new success features: Integration with charting or order entry system. Promotion of action rather than inaction. No need for additional clinician data entry. Justification of decision support via research evidence. Local user involvement. Provision of decision support results to patients as well as providers. Three previously identified success features were confirmed: Automatic provision of decision support as part of clinician workflow. Provision of decision support at time and location of decisionmaking. Provision of a recommendation, not just an assessment. Only 29 (19.6%) RCTs assessed the impact of CDSSs on clinical outcomes, 22 (14.9%) assessed costs, and 3 assessed KMSs on any outcomes. The primary source of knowledge used in CDSSs was derived from structured care protocols. CONCLUSIONS: Strong evidence shows that CDSSs/KMSs are effective in improving health care process measures across diverse settings using both commercially and locally developed systems. Evidence for the effectiveness of CDSSs on clinical outcomes and costs and KMSs on any outcomes is minimal. Nine features of CDSSs/KMSs that correlate with a successful impact of clinical decision support have been newly identified or confirmed.

Identifying research needs for improving health care.

Insights from systematic reviews can help new studies better meet the priorities and needs of patients and communities. However, systematic reviews unfortunately have not yet achieved this position to direct and guide new research studies. The Agency for Healthcare Research and Quality's Evidence-based Practice Center Program uses systematic reviews to identify gaps in current evidence and has developed a systematic process of prioritizing these gaps with stakeholder input into clearly defined "future research needs." Eight Evidence-based Practice Centers began to apply this effort in 2010 to various clinical and policy topics. Gaps that prevented systematic reviewers from answering central questions of the review may include insufficient studies on subpopulations, insufficient studies with appropriate comparators, lack of appropriate outcomes measured, and methods problems. Stakeholder panels, consisting of advocacy groups, patients, researchers, clinicians, funders, and policymakers, help refine the gaps through multiple conference calls and prioritization exercises. Each report highlights a focused set of 4 to 15 high-priority needs with an accompanying description of possible considerations for study design. Identification of high-priority research needs could potentially speed the development and implementation of high-priority, stakeholder-engaged research.