Topic identification, selection, and prioritization for health technology assessment in selected countries: a mixed study design.

BACKGROUND: There is limited evidence-informed guidance on TISP processes for countries where health technology assessment (HTA) is in a nascent phase. We aimed to explore the range of topic identification, selection and prioritization (TISP) processes and practices for HTA in selected countries and identify aspects relevant to emerging HTA systems. METHODS: This mixed design study included a systematic literature review, an electronic survey, and individual interviews. We conducted a systematic literature review with criteria that were developed a priori to identify countries deemed to have a recently formalized HTA system. Based on the literature review, a twenty-three item online survey was shared with the identified countries, we completed follow-up interviews with ten participants who have experience with HTA. We analyzed documents, survey responses and interview transcripts thematically to identify lessons related to TISP processes and practices. RESULTS: The literature review identified 29 nine candidate countries as having a "potential" recently formalized HTA system. Twenty-one survey responses were analyzed and supplemented with ten individual interviews. We found variation in countries' approaches to TISP - particularly between pharmaceutical and non-pharmaceutical interventions. Results indicate that TISP is heavily driven by policy makers, expert involvement, and to a lesser extent, relevant stakeholders. The use of horizon-scanning and early warning systems is uncommon. Interviewee participants provided further insight to the survey data, reporting that political awareness and an institutional framework were important to support TISP. TISP can be optimized by stronger national regulations and legislative structures, in addition to education and advocacy about HTA among politicians and decision-makers. In some settings regional networks have been useful, particularly in the development of TISP guidelines and methodologies. Additionally, the technical capacity to conduct TISP, and access to relevant local data were factors limiting TISP in national settings. Increased network collaboration and capacity building were reported as future needs. CONCLUSIONS: This study provides current insights into a topic where there is limited published peer reviewed literature. TISP is an important first step of HTA, and topics should be selected and prioritized based on local need and relevance. The limited capacity for TISP in settings where HTA is emerging may be supported by local and international collaboration to increase capacity and knowledge. To succeed, both TISP and HTA need to be embedded within national health care priority setting and decision-making. More in-depth understanding of where countries are situtated in formalizing the TISP process may help others to overcome factors that facilitate or hinder progress.

Evidence to decision frameworks enabled structured and explicit development of healthcare recommendations.

OBJECTIVE: The aim of this study is to identify and describe the processes suggested for the formulation of healthcare recommendations in healthcare guidelines available in guidance documents. METHODS: We searched international databases in May 2020 to retrieve guidance documents published by organizations dedicated to guideline development. Pairs of researchers independently selected and extracted data about the characteristics of the guidance document, including explicit or implicit recommendation-related criteria and processes considered, as well as the use of evidence to decision (EtD) frameworks. RESULTS: We included 68 guidance documents. Most organizations reported a system for grading the strength of recommendations (88%), half of them being the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach. Two out of three guidance documents (66%) proposed the use of a framework to guide the EtD process. The GRADE-EtD framework was the most often reported framework (19 organizations, 42%), whereas 20 organizations (44%) proposed their own multicriteria frameworks. Using any EtD framework was related with a more comprehensive set of recommendation-related criteria compared to no framework, especially for criteria like values, equity, and acceptability. CONCLUSION: Although limited, the use of EtD frameworks was associated with the inclusion of relevant recommendation criteria. Among the EtD structured frameworks, the GRADE-EtD framework offers the most comprehensive perspective for evidence-informed decision-making processes.

Developing a tool to assess the skills to perform a health technology assessment.

BACKGROUND: Health technology assessment (HTA) brings together evidence from various disciplines while using explicit methods to assess the value of health technologies. In resource-constrained settings, there is a growing demand to measure and develop specialist skills, including those for HTA, to aid the implementation of Universal Healthcare Coverage. The purpose of this study was twofold: a) to find validated tools for the assessment of the technical capacity to conduct a HTA, and if none were found, to develop a tool, and b) to describe experiences of its pilot. METHODS: First, a mapping review identified tools to assess the skills to conduct a HTA. A medical librarian conducted a comprehensive search in four databases (MEDLINE, Embase, Web of Science, ERIC). Then, incorporating results from the mapping and following an iterative process involving stakeholders and experts, we developed a HTA skills assessment tool. Finally, using an online platform to gather and analyse responses, in collaboration with our institutional partner, we piloted the tool in Ghana, and sought feedback on their experiences. RESULTS: The database search yielded 3871 records; fifteen those were selected based on a priori criteria. These records were published between 2003 and 2018, but none covered all technical skills to conduct a HTA. In the absence of an instrument meeting our needs, we developed a HTA skill assessment tool containing four sections (general information, core and soft skills, and future needs). The tool was designed to be administered to a broad range of individuals who would potentially contribute to the planning, delivery and evaluation of HTA. The tool was piloted with twenty-three individuals who completed the skills assessment and shared their initial impressions of the tool. CONCLUSIONS: To our knowledge, this is the first comprehensive tool enabling the assessment of technical skills to conduct a HTA. This tool allows teams to understand where their individual strengths and weakness lie. The tool is in the early validation phases and further testing is needed. TRIAL REGISTRATION: Not applicable.

An institutional ethnographic analysis of public and patient engagement activities at a national health technology assessment agency.

OBJECTIVE: The practice of public and patient engagement (PPE) in health technology assessment (HTA) has spread worldwide, yet gaps in knowledge remain. We carried out an institutional ethnography of the Canadian Agency for Drugs and Technologies in Health (CADTH) public and patient involvement in HTA. METHODS: The research took place over 15 months and included observational work in the institutional settings, text review, and interviews with individuals working for or involved with the agency. RESULTS: We found that despite demonstrated commitment to PPE, organizational history, governance structure, and practices were impediments to a unified approach to PPE. Unclear role descriptions for committee members and differences in philosophy and priority given to PPE across the organization presented challenges to effective participation. The high degree of value given to evidence-based principles at times conflicted with meaningful integration of patient input. A lack of clear goals and processes, roles, and differential treatment of evidence in PPE served to minimize the importance of patient experiences and to displace their validity. An acknowledgment of conflicts between multiple epistemic traditions at work within HTA activities may strengthen organizational approaches to PPE. CONCLUSION: HTA organizations can learn from this study by reflecting on the challenges described and the recommendations offered to address them. We suggest solidifying CADTH's commitment to PPE with clear agency-wide roles and direction, values, and outcomes, a comprehensive framework, and policy and procedures. An acknowledgment of diverse epistemic traditions, as well as leadership and expertise in PPE, will strengthen CADTH's PPE activities and sustain its leadership position in the HTA field.

Ethical Challenges Related to Patient Involvement in Health Technology Assessment.

Including information and values from patients in HTA has the potential to improve both the process and outcomes of health technology policy decisions. Accordingly, funding and structural incentives to include patients in HTA activities have increased over the past several years. Unfortunately, these incentives have not yet been accompanied by a corresponding increase in resources, time, or commitment to responsiveness. In this Perspectives piece, we reflect on our collective experiences participating in, conducting, and overseeing patient engagement activities within HTA to highlight the ethical challenges associated with this area of activity. While we remain committed to the idea that patient engagement activities strengthen the findings, relevance, and legitimacy of health technology policy, we are deeply concerned about the potential for these activities to do ethical harm. We use this analysis to call for action to introduce strong protections against ethical violations that may harm patients participating in HTA engagement activities.

Evaluating Patient, Family and Public Engagement in Health Services Improvement and System Redesign.

As efforts to actively involve patients, family members and the broader public in health service improvement and system redesign have grown, increasing attention has also been paid to evaluation of their engagement in the health system. We discuss key concepts and approaches related to evaluation, drawing particular attention to different and potentially competing goals, stakeholders and epistemological entry points. Evaluation itself can be supported by an increasing number of frameworks and tools, matched to the relevant purpose and approach. The patient engagement evaluation field faces several challenges, including the need for greater specification of both the form and the context of engagement, the need to balance the measurement imperative with the relational aspects of care and the need for supportive organizations with the capacity and commitment to undertake high-quality engagement and its evaluation.

Aquatic exercise training for fibromyalgia.

BACKGROUND: Exercise training is commonly recommended for individuals with fibromyalgia. This review examined the effects of supervised group aquatic training programs (led by an instructor). We defined aquatic training as exercising in a pool while standing at waist, chest, or shoulder depth. This review is part of the update of the 'Exercise for treating fibromyalgia syndrome' review first published in 2002, and previously updated in 2007. OBJECTIVES: The objective of this systematic review was to evaluate the benefits and harms of aquatic exercise training in adults with fibromyalgia. SEARCH METHODS: We searched The Cochrane Library 2013, Issue 2 (Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Health Technology Assessment Database, NHS Economic Evaluation Database), MEDLINE, EMBASE, CINAHL, PEDro, Dissertation Abstracts, WHO international Clinical Trials Registry Platform, and AMED, as well as other sources (i.e., reference lists from key journals, identified articles, meta-analyses, and reviews of all types of treatment for fibromyalgia) from inception to October 2013. Using Cochrane methods, we screened citations, abstracts, and full-text articles. Subsequently, we identified aquatic exercise training studies. SELECTION CRITERIA: Selection criteria were: a) full-text publication of a randomized controlled trial (RCT) in adults diagnosed with fibromyalgia based on published criteria, and b) between-group data for an aquatic intervention and a control or other intervention. We excluded studies if exercise in water was less than 50% of the full intervention. DATA COLLECTION AND ANALYSIS: We independently assessed risk of bias and extracted data (24 outcomes), of which we designated seven as major outcomes: multidimensional function, self reported physical function, pain, stiffness, muscle strength, submaximal cardiorespiratory function, withdrawal rates and adverse effects. We resolved discordance through discussion. We evaluated interventions using mean differences (MD) or standardized mean differences (SMD) and 95% confidence intervals (95% CI). Where two or more studies provided data for an outcome, we carried out meta-analysis. In addition, we set and used a 15% threshold for calculation of clinically relevant differences. MAIN RESULTS: We included 16 aquatic exercise training studies (N = 881; 866 women and 15 men). Nine studies compared aquatic exercise to control, five studies compared aquatic to land-based exercise, and two compared aquatic exercise to a different aquatic exercise program.We rated the risk of bias related to random sequence generation (selection bias), incomplete outcome data (attrition bias), selective reporting (reporting bias), blinding of outcome assessors (detection bias), and other bias as low. We rated blinding of participants and personnel (selection and performance bias) and allocation concealment (selection bias) as low risk and unclear. The assessment of the evidence showed limitations related to imprecision, high statistical heterogeneity, and wide confidence intervals. Aquatic versus controlWe found statistically significant improvements (P value < 0.05) in all of the major outcomes. Based on a 100-point scale, multidimensional function improved by six units (MD -5.97, 95% CI -9.06 to -2.88; number needed to treat (NNT) 5, 95% CI 3 to 9), self reported physical function by four units (MD -4.35, 95% CI -7.77 to -0.94; NNT 6, 95% CI 3 to 22), pain by seven units (MD -6.59, 95% CI -10.71 to -2.48; NNT 5, 95% CI 3 to 8), and stiffness by 18 units (MD -18.34, 95% CI -35.75 to -0.93; NNT 3, 95% CI 2 to 24) more in the aquatic than the control groups. The SMD for muscle strength as measured by knee extension and hand grip was 0.63 standard deviations higher compared to the control group (SMD 0.63, 95% CI 0.20 to 1.05; NNT 4, 95% CI 3 to 12) and cardiovascular submaximal function improved by 37 meters on six-minute walk test (95% CI 4.14 to 69.92). Only two major outcomes, stiffness and muscle strength, met the 15% threshold for clinical relevance (improved by 27% and 37% respectively). Withdrawals were similar in the aquatic and control groups and adverse effects were poorly reported, with no serious adverse effects reported. Aquatic versus land-basedThere were no statistically significant differences between interventions for multidimensional function, self reported physical function, pain or stiffness: 0.91 units (95% CI -4.01 to 5.83), -5.85 units (95% CI -12.33 to 0.63), -0.75 units (95% CI -10.72 to 9.23), and two units (95% CI -8.88 to 1.28) respectively (all based on a 100-point scale), or in submaximal cardiorespiratory function (three seconds on a 100-meter walk test, 95% CI -1.77 to 7.77). We found a statistically significant difference between interventions for strength, favoring land-based training (2.40 kilo pascals grip strength, 95% CI 4.52 to 0.28). None of the outcomes in the aquatic versus land comparison reached clinically relevant differences of 15%. Withdrawals were similar in the aquatic and land groups and adverse effects were poorly reported, with no serious adverse effects in either group. Aquatic versus aquatic (Ai Chi versus stretching in the water, exercise in pool water versus exercise in sea water)Among the major outcomes the only statistically significant difference between interventions was for stiffness, favoring Ai Chi (1.00 on a 100-point scale, 95% CI 0.31 to 1.69). AUTHORS' CONCLUSIONS: Low to moderate quality evidence relative to control suggests that aquatic training is beneficial for improving wellness, symptoms, and fitness in adults with fibromyalgia. Very low to low quality evidence suggests that there are benefits of aquatic and land-based exercise, except in muscle strength (very low quality evidence favoring land). No serious adverse effects were reported.