Which value aspects are relevant for the evaluation of medical devices? Exploring stakeholders' views through a Web-Delphi process.

BACKGROUND: Implementation and uptake of health technology assessment for evaluating medical devices require including aspects that different stakeholders consider relevant, beyond cost and effectiveness. However, the involvement of stakeholders in sharing their views still needs to be improved. OBJECTIVE: This article explores the relevance of distinct value aspects for evaluating different types of medical devices according to stakeholders' views. METHODS: Thirty-four value aspects collected through literature review and expert validation were the input for a 2-round Web-Delphi process. In the Web-Delphi, a panel of participants from five stakeholders' groups (healthcare professionals, buyers and policymakers, academics, industry, and patients and citizens) judged the relevance of each aspect, by assigning a relevance-level ('Critical', 'Fundamental', 'Complementary', or 'Irrelevant'), for two types of medical devices separately: 'Implantable' and 'In vitro tests based on biomarkers'. Opinions were analysed at the panel and group level, and similarities across devices were identified. RESULTS: One hundred thirty-four participants completed the process. No aspects were considered 'Irrelevant', neither for the panel nor for stakeholder groups, in both types of devices. The panel considered effectiveness and safety-related aspects 'Critical' (e.g., 'Adverse events for the patient'), and costs-related aspects 'Fundamental' (e.g., 'Cost of the medical device'). Several additional aspects not included in existing frameworks' literature, e.g., related to environmental impact and devices' usage by the healthcare professional, were deemed as relevant by the panel. A moderate to substantial agreement across and within groups was observed. CONCLUSION: Different stakeholders agree on the relevance of including multiple aspects in medical devices' evaluation. This study produces key information to inform the development of frameworks for valuing medical devices, and to guide evidence collection.

Advancing hospital-based health technology assessment: evaluating genomic panel contracting strategies for blood tumors through a multimethodology.

INTRODUCTION: The adoption of genomic technologies in the context of hospital-based health technology assessment presents multiple practical and organizational challenges. OBJECTIVE: This study aimed to assist the Instituto Português de Oncologia de Lisboa Francisco Gentil (IPO Lisboa) decision makers in analyzing which acute myeloid leukemia (AML) genomic panel contracting strategies had the highest value-for-money. METHODS: A tailored, three-step approach was developed, which included: mapping clinical pathways of AML patients, building a multicriteria value model using the MACBETH approach to evaluate each genomic testing contracting strategy, and estimating the cost of each strategy through Monte Carlo simulation modeling. The value-for-money of three contracting strategies - "Standard of care (S1)," "FoundationOne Heme test (S2)," and "New diagnostic test infrastructure (S3)" - was then analyzed through strategy landscape and value-for-money graphs. RESULTS: Implementing a larger gene panel (S2) and investing in a new diagnostic test infrastructure (S3) were shown to generate extra value, but also to entail extra costs in comparison with the standard of care, with the extra value being explained by making available additional genetic information that enables more personalized treatment and patient monitoring (S2 and S3), access to a broader range of clinical trials (S2), and more complete databases to potentiate research (S3). CONCLUSION: The proposed multimethodology provided IPO Lisboa decision makers with comprehensive and insightful information regarding each strategy's value-for-money, enabling an informed discussion on whether to move from the current Strategy S1 to other competing strategies.

Scenarios for population health inequalities in 2030 in Europe: the EURO-HEALTHY project experience.

BACKGROUND: Health inequalities have been consistently reported across and within European countries and continue to pose major challenges to policy-making. The development of scenarios regarding what could affect population health (PH) inequalities across Europe in the future is considered critical. Scenarios can help policy-makers prepare and better cope with fast evolving challenges. OBJECTIVE: This paper describes the three 2030 time-horizon scenarios developed under the EURO-HEALTHY project, depicting the key factors that may affect the evolution of PH inequalities across European regions. METHODS: A three-stage socio-technical approach was applied: i) identification of drivers (key factors expected to affect the evolution of PH inequalities across European regions until 2030) - this stage engaged in a Web-Delphi process a multidisciplinary panel of 51 experts and other stakeholders representing the different perspectives regarding PH inequalities; ii) generation of scenario structures - different drivers' configurations (i.e. their hypotheses for evolution) were organized into coherent scenario structures using the Extreme-World Method; and iii) validation of scenario structures and generation of scenario narratives. Stages ii) and iii) were conducted in two workshops with a strategic group of 13 experts with a wide view about PH inequalities. The scenario narratives were elaborated with the participants' insights from both the Web-Delphi process and the two workshops, together with the use of evidence (both current and future-oriented) on the different areas within the PH domain. RESULTS: Three scenarios were developed for the evolution of PH inequalities in Europe until 2030: 'Failing Europe' (worst-case but plausible picture of the future), 'Sustainable Prosperity' (best-case but plausible picture of the future), and an interim scenario 'Being Stuck' depicting a 'to the best of our knowledge' evolution. These scenarios show the extent to which a combination of Political, Economic, Social, Technological, Legal and Environmental drivers shape future health inequalities, providing information for European policy-makers to reflect upon whether and how to design robust policy solutions to tackle PH inequalities. CONCLUSIONS: The EURO-HEALTHY scenarios were designed to inform both policy design and appraisal. They broaden the scope, create awareness and generate insights regarding the evolution of PH inequalities across European regions.

ANIPHI: An innovative pedagogical platform based on the Delphi method to support animal welfare teaching.

As a teaching subject, animal welfare is challenging for educators and learners, as was recently shown in a recent survey on the evolution of animal welfare teaching in Europe. Among several suggestions to overcome the current resistance to implementing animal welfare education, we highlight two. The first is that animal welfare education should be based on learner-centred approaches; the second is that it should encompass both animal welfare science and ethics and law. To the best of our knowledge, there are no learner-centred pedagogical approaches that can simultaneously explore scientific and ethical concepts. Furthermore, when exploring ethical concepts within the educational context, there is the additional challenge of being able to depart from discussion and debate to a systematic organization of knowledge. Our work simultaneously addresses these two challenges, presenting the design and implementation of a novel web-based learner-centred pedagogical platform for farm animal welfare teaching. The platform, named ANIPHI, uses the Delphi method's iterative nature as a learning process to generate both reflection and (online) debate among learners. ANIPHI can be used by educators in an online environment, in a classroom environment, or in a combination of the two environments. ANIPHI was developed within the ERASMUS+ ANICARE project and is an open web-based platform for all educators interested in teaching farm animal welfare. Given ANIPHI's flexible and user-friendly nature, the platform simultaneously exposes learners to ethical and scientific concepts in different educational realities, according to the educator's objectives. Furthermore, videos depicting different husbandry practices across different types of animal production and countries are embedded in the platform. These videos are commented on by the farmer himself and by animal scientists, which enriches the learner's experience. Educators across the ANICARE consortium have already successfully tested the ANIPHI platform for different farm animal welfare topics. We conclude this article by presenting one example of using ANIPHI in a real-life educational context, where we discuss some aspects of the design and use of our pedagogical platform.

Evaluation of Inter-Observer Reliability of Animal Welfare Indicators: Which Is the Best Index to Use?

This study focuses on the problem of assessing inter-observer reliability (IOR) in the case of dichotomous categorical animal-based welfare indicators and the presence of two observers. Based on observations obtained from Animal Welfare Indicators (AWIN) project surveys conducted on nine dairy goat farms, and using udder asymmetry as an indicator, we compared the performance of the most popular agreement indexes available in the literature: Scott's π, Cohen's k, kPABAK, Holsti's H, Krippendorff's α, Hubert's Γ, Janson and Vegelius' J, Bangdiwala's B, Andrés and Marzo's ∆, and Gwet's γ(AC1). Confidence intervals were calculated using closed formulas of variance estimates for π, k, kPABAK, H, α, Γ, J, ∆, and γ(AC1), while the bootstrap and exact bootstrap methods were used for all the indexes. All the indexes and closed formulas of variance estimates were calculated using Microsoft Excel. The bootstrap method was performed with R software, while the exact bootstrap method was performed with SAS software. k, π, and α exhibited a paradoxical behavior, showing unacceptably low values even in the presence of very high concordance rates. B and γ(AC1) showed values very close to the concordance rate, independently of its value. Both bootstrap and exact bootstrap methods turned out to be simpler compared to the implementation of closed variance formulas and provided effective confidence intervals for all the considered indexes. The best approach for measuring IOR in these cases is the use of B or γ(AC1), with bootstrap or exact bootstrap methods for confidence interval calculation.

Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach.

Environmental health (EH) is influenced by complex interactions between health and the built and natural environments, there being little research on its specificities in urban settings. The use of suitable indicators to monitor and assess EH is fundamental in informing evidence-based interventions at the local level. A participatory approach to selecting indicators to inform the monitoring and assessment of EH in Lisbon is herein considered. Evidence derived from a systematic review of literature and data from Lisbon and Portuguese databases were analyzed by 12 Portuguese experts in individual semi-structured interviews. The interviews aimed at identifying relevant indicators and important emerging issues in the Lisbon urban setting. The outputs from the interviews were validated by a two-round Web-Delphi process in which panelists (22 experts) from different areas of expertise expressed their views regarding the relevance of the indicators for the analysis of EH in urban settings. Seventeen indicators were validated in the Web-Delphi process. High participation achieved along this process supports the view that this participatory approach was useful for validation. Results from the adopted participatory approach point out gaps in the collection of noise and mobility indicators data and raise emerging issues on housing indicators that require further research. The results also suggest the need for local action to improve indicators and tools in order to help the monitorization of EH in urban contexts. The adopted participatory approach can be replicated for other Portuguese and European urban settings.