Navigating the path towards successful implementation of the EU HTA Regulation: key takeaways from the 2023 Spring Convention of the European Access Academy.

BACKGROUND: The European Regulation on Health Technology Assessment (EU HTA R), effective since January 2022, aims to harmonize and improve the efficiency of common HTA across Member States (MS), with a phased implementation from January 2025. At "midterms" of the preparation phase for the implementation of the Regulation our aim was to identify and prioritize tangible action points to move forward. METHODS: During the 2023 Spring Convention of the European Access Academy (EAA), participants from different nationalities and stakeholder backgrounds discussed readiness and remaining challenges for the Regulation's implementation and identified and prioritized action points. For this purpose, participants were assigned to four working groups: (i) Health Policy Challenges, (ii) Stakeholder Readiness, (iii) Approach to Uncertainty and (iv) Challenges regarding Methodology. Top four action points for each working group were identified and subsequently ranked by all participants during the final plenary session. RESULTS: Overall "readiness" for the Regulation was perceived as neutral. Prioritized action points included the following: Health Policy, i.e. assess adjustability of MS laws and health policy processes; Stakeholders, i.e. capacity building; Uncertainty, i.e. implement HTA guidelines as living documents; Methodology, i.e. clarify the Population, Intervention, Comparator(s), Outcomes (PICO) identification process. CONCLUSIONS: At "midterms" of the preparation phase, the focus for the months to come is on executing the tangible action points identified at EAA's Spring Convention. All action points centre around three overarching themes: harmonization and standardization, capacity building and collaboration, uncertainty management and robust data. These themes will ultimately determine the success of the EU HTA R in the long run.

Using Real-World Data in the Health Technology Assessment of Pharmaceuticals: Strengths, Difficulties, and a Pragmatic Way Forward.

In the past decade, there have been increasing calls for greater use of real-world evidence (RWE) and data (RWD), with the explicit goal of enabling faster provision of effective medicines to patients in need. The push for decision makers to accept RWE is especially noticeable in the pursuit of regulatory approval, but RWE, particularly when used to estimate the relative effectiveness of interventions, is not always readily accepted by agencies responsible for reimbursement and pricing of new pharmaceuticals and, to a varying degree, is not accepted across jurisdictions. This lack of trust hampers the use of RWE despite a very large and growing literature base on the principles of how RWE should be used. In this article, we suggest an important part of the explanation of why this situation has arisen and make suggestions for its alleviation. Given that problems commonly arise that are particular to the question being asked and the data sources being used, general guidance on the principles of how to use RWD cannot cover all eventualities. Therefore, we are suggesting the creation of an archive, or repository, to record uses of RWD in support of decisions by funding bodies or their advisors. This article introduces a proposed, structured classification of decision types using RWE, around which evidence can be assembled in a curated source (RWD/RWE taxonomy) and thus facilitate judgments on when evidence is "good enough." This article is part of a series in a special issue of this journal that looks at the barriers to optimal use of RWE in health technology assessment and how to overcome them. We begin significantly to populate our "taxonomy" with examples in an accompanying article. We also propose recommendations for international standards of evaluating the acceptability of RWD governance practices.

The Real-World Evidence Workstream in EUreccA 2025: How the Task Was Addressed.

This is one of a series of articles that consider the barriers to optimal use of real-world evidence (RWE) in health technology assessment (HTA) as well as ways to overcome them. The work was carried out as part of EUreccA 2025 (European Initiative for New Reimbursement and Access Approaches 2025), in particular with the RWE workstream embodied within that collaboration. The starting premises of this workstream were as follows: (1) the acceptance of RWE by HTA agencies and payers in the assessment of drugs is suboptimal and variable between jurisdictions, and (2) if that were not the case, the path of new pharmaceuticals to patients could be quicker and less expensive. Elsewhere in this issue we set out the conclusions we had reached in the EUreccA RWE workstream. In this article, we set out the methodology used to conduct the totality of the EureccA 2025 RWE workstream effort, which led us to those conclusions. The main results, strengths, and limitations of the individual parts are discussed further in separate articles in this supplement. Through scoping work, we generated 4 key topics within which to identify and address the barriers to optimal RWE use in HTA. Through pragmatic literature searches, stakeholder engagement, and case studies, we suggest ways in which the problems identified may be addressed as a contribution to progress in this area.

Across-Country Variations of Real-World Data and Evidence for Drugs: A 5-European-Country Study.

OBJECTIVES: This study aimed to describe the role of real-world data (RWD) and real-world evidence (RWE) in health technology assessment (HTA) in 5 European countries and to identify the hurdles to the acceptance of RWE and suggest directions toward its more effective use. METHODS: Authors from France, Germany, Italy, and Sweden used a common template to extract evidence. For England, the Cancer Drugs Fund was described and analyzed as a particular model for the use of RWD to provide evidence for coverage decisions and managed entry agreements. RESULTS: In all countries except Germany, HTA bodies acknowledged the relevance of RWD/RWE to address postlaunch uncertainties. In Germany, evidence from randomized controlled trials remains the gold standard, and evidence based on RWD is generally rejected. Multiple sources of RWD exist, but the quality, the immediate relevance of existing sources, and their interoperability limit their adaptation to the specifics of a given drug. This leads to skepticism about the validity of the evidence. Timing is also a key issue: the production of evidence may not be synchronized with the HTA and pricing bodies' agendas. The Cancer Drugs Fund case emphasizes that a strong partnership among all stakeholders and a pragmatic use of existing data, alongside clinical evidence provided by companies, are key success factors. CONCLUSIONS: A continuous investment in national health information systems is a key issue for providing valid RWE. Processes and aids to guide the acceptability and usage of RWE derived from pairing between sources and questions are essential.

Data Governance for Real-World Data Management: A Proposal for a Checklist to Support Decision Making.

OBJECTIVES: Real-world data (RWD) and real-world evidence (RWE) can provide extensive information on healthcare for use in health technology assessment and decision making. Nevertheless, there is a lack of consensus surrounding the appropriate data governance (DG) practices for RWD/RWE. Data sharing is also a large concern, especially considering evolving data protection regulations. Our objective is to propose recommendations for international standards of evaluating the acceptability of RWD governance practices. METHODS: After reviewing the literature, we created a checklist targeting DG practices for RWD/RWE. We then carried out a 3-round Delphi panel, including European policy makers, health technology assessment experts, and hospital managers. The consensus for each statement was measured and the checklist adjusted accordingly. RESULTS: The literature review identified the main topics regarding RWD/RWE DG practices: data privacy and security, data management and linkage, data access management, and the generation and use of RWE. Members of the Delphi panel (21 experts/25 invited) were presented a total of 24 statements related to each of the topics. Experts demonstrated a progressive level of consensus and importance ratings in all topics and to most statements. We suggest a refined checklist in which the statements rated less important or with less consensus have been removed. CONCLUSIONS: This study suggests how the DG of RWD/RWE could be qualitatively evaluated. We propose checklists that could be used by all RWD/RWE users to help ensure the quality and integrity of RWD/RWE governance and complement data protection law.

Structure and Content of a Taxonomy to Support the Use of Real-World Evidence by Health Technology Assessment Practitioners and Healthcare Decision Makers.

This is one of a series of articles that consider the barriers to optimal use of real-world evidence (RWE) in health technology assessment and how to overcome them. The work was performed as part of EUreccA 2025, in particular with the RWE workstream embodied within that collaboration. Elsewhere in this issue we described the reasoning and process that led us to develop practical tools to support RWE use, including this taxonomy and explained the methods used to do so. The taxonomy classifies questions that are typically addressed using real-world data in health technology assessment and the data sources typically used to address these questions. In this article, we describe the taxonomy itself. For as many of the pairings as possible, we have provided links to advice and methods on how to address the associated question using those data. We have also provided links to examples of RWE use in practical decision making to answer the questions posed. Our work is not complete, but we believe it is sufficient to demonstrate the value of such a taxonomy and information source if it is completed and curated as a "wiki" by the community that would use it.

Mobile applications to prescribe physical exercise in frail older adults: review of the available tools in app stores.

INTRODUCTION: Different remote interventions, such as applications (apps), have been used to continue promoting healthy ageing and preventing disability during the COVID-19 pandemic. The growing trend of apps in health is exponential and may facilitate scaling up physical activity prescription. Numerous tools are available, but little is known regarding their appropriateness, validation and recommendation, especially for frail older adults. METHODS: In-house, we developed an application that makes both the Apple app Store and the Google Play Store searchable using topic-related keywords and facilitates the extraction of basic app-information of the search results. The study was aimed at apps available to an English-speaking market. The resulting apps were filtered using various inclusion and exclusion criteria. The resultant apps underwent a more in-depth characterisation and searches for scientific publications on each app website and PubMed. RESULTS: From an initial search result of >2,800 apps, 459 met the initial inclusion criteria. After a more in-depth review of their features, 39 apps remained for possible app in older frail patients. After testing them, 22 apps were excluded. Seventeen apps fit the inclusion and exclusion criteria and were deemed appropriate after peer review. Of these, only one app, Vivifrail, had any type of publication/published evidence. CONCLUSION: Apps can be valuable tool in prescribing exercise for frail older adults living in the community. However, few apps seem useful on a large scale, and there is limited evidence to support their effectiveness. It is important to invest in adapting Information and Communication Technologies to this population group.

Value drivers for pharmaceutical products in health technology assessment (HTA) in Saudi Arabia: Results from a capacity Building, Multi-Stakeholder workshop.

OBJECTIVES: Capacity building exercises are important to increase understanding of healthcare processes by key stakeholders, and to facilitate open discussions to build consensus. This study explored the views of a multi-stakeholder group of local Saudi experts on possible value elements that could be important for health technology assessment (HTA) processes and methods regarding pharmaceutical products in Saudi Arabia ('value drivers'). METHODS: A diversified group of local experts were invited to a two-day capacity building workshop from 18 to 19 December 2019 in Riyadh, Saudi Arabia. Information regarding the participants' demographic and educational/professional background, along with their self-assessed knowledge and experience of HTAs and the concept of value in the pharmaceutical market was collected. For each of 22 value drivers identified during a targeted literature search, participants were asked either to 'opt out' of its consideration for future HTA assessments, or rate it from 1 to 10 (low-high) on feasibility and acceptability. RESULTS: Efficacy and safety were the highest rated value drivers for acceptability and feasibility. Explicit cost-effectiveness thresholds had the lowest ratings for acceptability and feasibility. Participants highlighted data availability and accuracy as a potential challenge to HTA implementation in Saudi Arabia. CONCLUSIONS: Participants valued a pharmaceutical product's efficacy and safety alongside the consideration of disease characteristics for HTA processes. Participants also valued a binding HTA recommendation and the use of local real-world evidence, where available, to support HTA submissions.

Cost-Effectiveness of the EdAl (Educació en Alimentació) Program: A Primary School-Based Study to Prevent Childhood Obesity.

BACKGROUND: The cost-effectiveness of childhood obesity prevention interventions is critical for their sustained implementation. This study evaluated the cost-effectiveness of the Educació en Alimentació (EdAl) program, a school-based intervention for reducing obesity. METHODS: Total EdAl program implementation costs and per-child costs were estimated. Cost-effectiveness, defined using the incremental cost-effectiveness ratio (ICER), was estimated as the difference between the intervention and control group costs divided by the obesity-related outcome effects for boys (avoided cases of obesity, obesity prevalence, body mass index [BMI], and BMI z-score units) for each group. As a significant difference (4.39%) in the reduction of obesity prevalence between the intervention and control groups was observed for boys in the EdAl program, the data were calculated only for boys. RESULTS: The intervention cost was 24,246.53 € for 1,550 children (15.64 €/child/3 years) or 5.21 €/child/year. The ICERs/boy were 968.66 € to avoid one case of obesity, 3.6 € to reduce the obesity prevalence by 1%, 44.68 € to decrease BMI by one unit, and 65.16 € to reduce the BMI z-score by one unit. CONCLUSIONS: The cost of reducing the obesity prevalence in boys by 4.39% was 5.21 €/child/year, half the cost proposed by the Spanish Health Ministry, indicating that the EdAl program is cost-effective.

DEFINING INNOVATION WITH RESPECT TO NEW MEDICINES: A SYSTEMATIC REVIEW FROM A PAYER PERSPECTIVE.

OBJECTIVES: The aim of this study was to investigate how innovation is defined with respect to new medicines. METHODS: MEDLINE, Embase, and EconLit databases were searched for articles published between January 1, 2010 and May 25, 2016 that described a relevant definition of innovation. Identified definitions were analyzed by mapping the concepts described onto a set of ten dimensions of innovation. RESULTS: In total, thirty-six articles were included, and described a total of twenty-five different definitions of innovation. The most commonly occurring dimension was therapeutic benefit, with novelty and the availability of existing treatments the second and third most common dimensions. Overall, there was little agreement in the published literature on what characteristics of new medicines constitute rewardable innovation. CONCLUSIONS: Alignment across countries and among regulators, health technology assessment bodies and payers would help manufacturers define research policies that can drive innovation, but may be challenging, as judgements about what aspects of innovation should be rewarded vary among stakeholders, and depend on political and societal factors.

An Inclusive Civil Society Dialogue for Successful Implementation of the EU HTA Regulation: Call to Action to Ensure Appropriate Involvement of Stakeholders and Collaborators.

OBJECTIVES: Stakeholder involvement has long been considered a success factor for a joint European health technology assessment (HTA) process, and its relevance is now anchored in the EU HTA Regulation's (EU HTAR) legislative wording. Therefore, we aimed to explore the roles, challenges, and most important activities to increase the level of involvement per stakeholder group. METHODS: At the 2022 Fall Convention of the European Access Academy (EAA), working groups addressed the involvement of patients, clinicians, regulators, health technology developers (HTD), and national HTA bodies and payers within the EU HTA process. Each working group revisited the pre-convention survey results, determined key role characteristics for each stakeholder, and agreed on the most important activities to fulfill the role profile. Finally, the activities suggested per group were prioritized by plenary group. RESULTS: The prioritized actions for patients included training and capacity building, the establishment of a patient involvement committee, and the establishment of a patient unit at the EC secretariat. For clinicians, it included alignment on evidence assessment from a clinical vs. HTA point of view, capacity building, and standardization of processes. The most important actions for regulators are to develop joint regulatory-HTA guidance documents, align processes and interfaces under the regulation, and share discussions on post-licensing evidence generation. HTDs prioritized scientific advice capacity and the review of the scoping process, and further development of the scope of the assessment report fact checks. The top three actions for national HTA bodies and payers included clarification on the early HTD dialogue process, political support and commitment, and clarification on financial support. CONCLUSIONS: Addressing the activities identified as the most important for stakeholders/collaborators in the EU HTA process (e.g., in the implementation of the EU HTA Stakeholder Network and of the guidance documents developed by the EUnetHTA 21 consortium) will be key to starting an "inclusive civil society dialogue", as suggested by the European Commission's Pharmaceutical Strategy.

Shifting from glucose diagnosis to the new HbA1c diagnosis reduces the capability of the Finnish Diabetes Risk Score (FINDRISC) to screen for glucose abnormalities within a real-life primary healthcare preventive strategy.

BACKGROUND: To investigate differences in the performance of the Finnish Diabetes Risk Score (FINDRISC) as a screening tool for glucose abnormalities after shifting from glucose-based diagnostic criteria to the proposed new hemoglobin (Hb)A1c-based criteria. METHODS: A cross-sectional primary-care study was conducted as the first part of an active real-life lifestyle intervention to prevent type 2 diabetes within a high-risk Spanish Mediterranean population. Individuals without diabetes aged 45-75 years (n = 3,120) were screened using the FINDRISC. Where feasible, a subsequent 2-hour oral glucose tolerance test and HbA1c test were also carried out (n = 1,712). The performance of the risk score was calculated by applying the area under the curve (AUC) for the receiver operating characteristic, using three sets of criteria (2-hour glucose, fasting glucose, HbA1c) and three diagnostic categories (normal, pre-diabetes, diabetes). RESULTS: Defining diabetes by a single HbA1c measurement resulted in a significantly lower diabetes prevalence (3.6%) compared with diabetes defined by 2-hour plasma glucose (9.2%), but was not significantly lower than that obtained using fasting plasma glucose (3.1%). The FINDRISC at a cut-off of 14 had a reasonably high ability to predict diabetes using the diagnostic criteria of 2-hour or fasting glucose (AUC = 0.71) or all glucose abnormalities (AUC = 0.67 and 0.69, respectively). When HbA1c was used as the primary diagnostic criterion, the AUC for diabetes detection dropped to 0.67 (5.6% reduction in comparison with either 2-hour or fasting glucose) and fell to 0.55 for detection of all glucose abnormalities (17.9% and 20.3% reduction, respectively), with a relevant decrease in sensitivity of the risk score. CONCLUSIONS: A shift from glucose-based diagnosis to HbA1c-based diagnosis substantially reduces the ability of the FINDRISC to screen for glucose abnormalities when applied in this real-life primary-care preventive strategy.

Efficacy, Safety, and Evaluation Criteria of mHealth Interventions for Depression: Systematic Review.

BACKGROUND: Depression is a significant public health issue that can lead to considerable disability and reduced quality of life. With the rise of technology, mobile health (mHealth) interventions, particularly smartphone apps, are emerging as a promising approach for addressing depression. However, the lack of standardized evaluation tools and evidence-based principles for these interventions remains a concern. OBJECTIVE: In this systematic review and meta-analysis, we aimed to evaluate the efficacy and safety of mHealth interventions for depression and identify the criteria and evaluation tools used for their assessment. METHODS: A systematic review and meta-analysis of the literature was carried out following the recommendations of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. Studies that recruited adult patients exhibiting elevated depressive symptoms or those diagnosed with depressive disorders and aimed to assess the effectiveness or safety of mHealth interventions were eligible for consideration. The primary outcome of interest was the reduction of depressive symptoms, and only randomized controlled trials (RCTs) were included in the analysis. The risk of bias in the original RCTs was assessed using version 2 of the Cochrane risk-of-bias tool for randomized trials. RESULTS: A total of 29 RCTs were included in the analysis after a comprehensive search of electronic databases and manual searches. The efficacy of mHealth interventions in reducing depressive symptoms was assessed using a random effects meta-analysis. In total, 20 RCTs had an unclear risk of bias and 9 were assessed as having a high risk of bias. The most common element in mHealth interventions was psychoeducation, followed by goal setting and gamification strategies. The meta-analysis revealed a significant effect for mHealth interventions in reducing depressive symptoms compared with nonactive control (Hedges g=-0.62, 95% CI -0.87 to -0.37, I2=87%). Hybrid interventions that combined mHealth with face-to-face sessions were found to be the most effective. Three studies compared mHealth interventions with active controls and reported overall positive results. Safety analyses showed that most studies did not report any study-related adverse events. CONCLUSIONS: This review suggests that mHealth interventions can be effective in reducing depressive symptoms, with hybrid interventions achieving the best results. However, the high level of heterogeneity in the characteristics and components of mHealth interventions indicates the need for personalized approaches that consider individual differences, preferences, and needs. It is also important to prioritize evidence-based principles and standardized evaluation tools for mHealth interventions to ensure their efficacy and safety in the treatment of depression. Overall, the findings of this study support the use of mHealth interventions as a viable method for delivering mental health care. TRIAL REGISTRATION: PROSPERO CRD42022304684; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=304684.

Regulatory and HTA Considerations for Development of Real-World Data Derived External Controls.

Regulators and Health Technology Assessment (HTA) bodies are increasingly familiar with, and publishing guidance on, external controls derived from real-world data (RWD) to generate real-world evidence (RWE). We recently conducted a systematic literature review (SLR) evaluating publicly available information on the use of RWD-derived external controls to contextualize outcomes from uncontrolled trials submitted to the European Medicines Agency (EMA), the US Food and Drug Administration (FDA), and/or select HTA bodies. The review identified several key operational and methodological aspects for which more detailed guidance and alignment within and between regulatory agencies and HTA bodies is necessary. This paper builds on the SLR findings by delineating a set of key takeaways for the responsible generation of fit-for-purpose RWE. Practical methodological and operational guidelines for designing, conducting, and reporting RWD-derived external control studies are explored and discussed. These considerations include: (i) early engagement with regulators and HTA bodies during the study planning phase; (ii) consideration of the appropriateness and comparability of external controls across multiple dimensions, including eligibility criteria, temporality, population representation, and clinical evaluation; (iii) ensuring adequate sample sizes, including hypothesis testing considerations; (iv) implementation of a clear and transparent strategy for assessing and addressing data quality, including data missingness across trials and RWD; (v) selection of comparable and meaningful endpoints that are operationalized and analyzed using appropriate analytic methods; and (vi) conduct of sensitivity analyses to assess the robustness of findings in the context of uncertainty and sources of potential bias.

Effectively Leveraging RWD for External Controls: A Systematic Literature Review of Regulatory and HTA Decisions.

Real-world data (RWD)-derived external controls can be used to contextualize efficacy findings for investigational therapies evaluated in uncontrolled trials. As the number of submissions to regulatory and health technology assessment (HTA) bodies using external controls rises, and in light of recent regulatory and HTA guidance on the appropriate use of RWD, there is a need to address the operational and methodological challenges impeding the quality of real-world evidence (RWE) generation and the consistency in evaluation of RWE across agencies. This systematic review summarizes publicly available information on the use of external controls to contextualize outcomes from uncontrolled trials for all indications from January 1, 2015, through August 20, 2021, that were submitted to the European Medicines Agency, the US Food and Drug Administration, and/or select major HTA bodies (National Institute for Health and Care Excellence (NICE), Haute Autorité de Santé (HAS), Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), and Gemeinsamer Bundesausschuss (G-BA)). By systematically reviewing submissions to regulatory and HTA bodies in the context of recent guidance, this study provides quantitative and qualitative insights into how external control design and analytic choices may be viewed by different agencies in practice. The primary operational and methodological aspects identified for discussion include, but are not limited to, engagement of regulators and HTA bodies, approaches to handling missing data (a component of data quality), and selection of real-world endpoints. Continued collaboration and guidance to address these and other aspects will inform and assist stakeholders attempting to generate evidence using external controls.

The role of stakeholder involvement in the evolving EU HTA process: Insights generated through the European Access Academy's multi-stakeholder pre-convention questionnaire.

Involvement of all relevant stakeholders will be of utmost importance for the success of the developing EU HTA harmonization process. A multi-step procedure was applied to develop a survey across stakeholders/collaborators within the EU HTA framework to assess their current level of involvement, determine their suggested future role, identify challenges to contribution, and highlight efficient ways to fulfilling their role. The 'key' stakeholder groups identified and covered by this research included: patients', clinicians', regulatory, and Health Technology Developer representatives. The survey was circulated to a wide expert audience including all relevant stakeholder groups in order to determine self-perception by the 'key' stakeholders regarding involvement in the HTA process (self-rating), and in a second, slightly modified version of the questionnaire, to determine the perception of 'key' stakeholder involvement by HTA bodies, payers, and policymakers (external rating). Predefined analyses were conducted on the submitted responses. Fifty-four responses were received (patients 9; clinicians: 8; regulators: 4; HTDs 14; HTA bodies: 7; Payers: 5; policymakers 3; others 4). The mean self-perceived involvement score was consistently lower for each of the 'key' stakeholder groups than the respective external ratings. Based on the qualitative insights generated in the survey, a RACI Chart (Responsible/Accountable/Consulted/Informed) was developed for each of the stakeholder groups to determine their roles and involvement in the current EU HTA process. Our findings suggest extensive effort and a distinct research agenda are required to ensure adequate involvement of the key stakeholder groups in the evolving EU HTA process.

Use of the incremental cost-effectiveness ratio for decision-making policies-what is the problem? A perspective paper.

INTRODUCTION: Drug reimbursement decisions that spark public controversy are potential signals that processes used to reach such decisions do not adequately reflect society's goals. Such controversial decisions appear to be a characteristic of Quality-Adjusted Life Year (QALY)-based Incremental Cost Effectiveness Ratio (ICER)-dominated decision-making systems. QALY-based ICER-heavy systems have several known weaknesses that lead to individual and societal preferences being either ignored or considered in an unsystematic and inconsistent manner. AREAS COVERED: We reprise some of the key inadequacies of QALY-based ICER analyses and suggest that there are other means including multicriteria decision analysis (MCDA) and cost-benefit analysis based on willingness to pay (WTP) measures by which to partially mitigate these weaknesses. EXPERT OPINION: For long, the inadequacies of QALY-based ICER-heavy decision-making systems have been rationalized with the answer: 'while the method is a second best, it is the best we currently have.' In light of the equally well-developed and widely utilized alternatives available, this resistance to improve assessment processes should not be accepted by policy makers. Health technology assessment bodies should consider and, with appropriate modifications, adopt these alternatives as they have the potential to result in more comprehensive, systematic, and accountable decision-making.

The emerging role of real-world data in advanced breast cancer therapy: Recommendations for collaborative decision-making.

Among stakeholders and decision-makers in advanced breast cancer, the demand for insights from real-world data (RWD) is increasing. Although RWD can be used to support decisions throughout different stages of a breast cancer drug's life cycle, barriers exist to its use and acceptance. We propose a collaborative approach to generating and using RWD that is meaningful to multiple stakeholders, and encourage frameworks toward international guidelines to help standardize RWD methodologies to achieve more efficient use of RWD insights.

How can a joint European health technology assessment provide an 'additional benefit' over the current standard of national assessments? : Insights generated from a multi-stakeholder survey in hematology/oncology.

OBJECTIVES: We conducted a multi-stakeholder survey to determine key areas where a joint European health technology assessment (HTA) could provide 'additional benefit' compared to the status quo of many parallel independent national and subnational assessments. METHODS: Leveraging three iterative Delphi cycles, a semiquantitative questionnaire was developed covering evidence challenges and heterogeneity of value drivers within HTAs across Europe with a focus on hematology/oncology. The questionnaire consisted of five sections: i) background information; ii) value drivers in HTA assessments today; iii) evolving evidence challenges; iv) heterogeneity of value drivers across Europe; v) impact of Europe's Beating Cancer Plan (EBCP). The questionnaire was circulated across n = 189 stakeholder institutions comprising HTA and regulatory bodies, clinical oncology associations, patient representatives, and industry associations. RESULTS: N = 30 responses were received (HTA bodies: 9; regulators: 10; patients' and physicians' associations: 3 each; industry: 5). Overall, 17 countries and EU level institutions were represented in the responses. Consistency across countries and stakeholder groups was high. Most relevant value drivers in HTAs today (scale 1, low to 5, high) were clinical trial design (mean 4.45), right endpoints (mean 4.40), and size of comparative effect (mean 4.33). Small patient numbers (mean 4.28) and innovative study designs (mean 4.1) were considered the most relevant evolving evidence challenges. Heterogeneity between regulatory and HTA evidence requirements and heterogeneity of the various national treatment standards and national HTA evidence requirements was high. All clinical and patient participants stated to have been with EBCP initiatives. CONCLUSIONS: For a European HTA to provide an 'additional benefit' over the multitude of existing national assessments key methodological and process challenges need to be addressed. These include approaches to address uncertainty in clinical development; comparator choice; consistency in approaching patient-relevant endpoints; and a transparent and consistent management of both HTA and regulatory procedures as well as their interface, including all involved stakeholder groups.

Mobile Applications in Mood Disorders and Mental Health: Systematic Search in Apple App Store and Google Play Store and Review of the Literature.

OBJECTIVES: The main objective of this work was to explore and characterize the current landscape of mobile applications available to treat mood disorders such as depression, bipolar disorder, and dysthymia. METHODS: We developed a tool that makes both the Apple App Store and the Google Play Store searchable using keywords and that facilitates the extraction of basic app information of the search results. All app results were filtered using various inclusion and exclusion criteria. We characterized all resultant applications according to their technical details. Furthermore, we searched for scientific publications on each app's website and PubMed, to understand whether any of the apps were supported by any type of scientific evidence on their acceptability, validation, use, effectiveness, etc. Results: Thirty apps were identified that fit the inclusion and exclusion criteria. The literature search yielded 27 publications related to the apps. However, these did not exclusively concern mood disorders. 6 were randomized studies and the rest included a protocol, pilot-, feasibility, case-, or qualitative studies, among others. The majority of studies were conducted on relatively small scales and 9 of the 27 studies did not explicitly study the effects of mobile application use on mental wellbeing. CONCLUSION: While there exists a wealth of mobile applications aimed at the treatment of mental health disorders, including mood disorders, this study showed that only a handful of these are backed by robust scientific evidence. This result uncovers a need for further clinically oriented and systematic validation and testing of such apps.