Best practice considerations on the assessment of robotic assisted surgical systems: results from an international consensus expert panel.

BACKGROUND: Health technology assessments (HTAs) of robotic assisted surgery (RAS) face several challenges in assessing the value of robotic surgical platforms. As a result of using different assessment methods, previous HTAs have reached different conclusions when evaluating RAS. While the number of available systems and surgical procedures is rapidly growing, existing frameworks for assessing MedTech provide a starting point, but specific considerations are needed for HTAs of RAS to ensure consistent results. This work aimed to discuss different approaches and produce guidance on evaluating RAS. METHODS: A consensus conference research methodology was adopted. A panel of 14 experts was assembled with international experience and representing relevant stakeholders: clinicians, health economists, HTA practitioners, policy makers, and industry. A review of previous HTAs was performed and seven key themes were extracted from the literature for consideration. Over five meetings, the panel discussed the key themes and formulated consensus statements. RESULTS: A total of ninety-eight previous HTAs were identified from twenty-five total countries. The seven key themes were evidence inclusion and exclusion, patient- and clinician-reported outcomes, the learning curve, allocation of costs, appropriate time horizons, economic analysis methods, and robotic ecosystem/wider benefits. CONCLUSIONS: Robotic surgical platforms are tools, not therapies. Their value varies according to context and should be considered across therapeutic areas and stakeholders. The principles set out in this paper should help HTA bodies at all levels to evaluate RAS. This work may serve as a case study for rapidly developing areas in MedTech that require particular consideration for HTAs.

On the integration of early health technology assessment in the innovation process: reflections from five stakeholders.

Early health technology assessment (HTA), which includes all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty, has seen many applications in recent years. However, it is still unclear how such early value assessments can be integrated into the technology innovation process. This commentary contributes to the discussion on the purposes early HTA can serve. Similarities and differences in the perspectives of five stakeholders (i.e., the hospital, the patient, the assessor, the medical device industry, and the policy maker) on the purpose, value, and potential challenges of early HTA are described. All five stakeholders agreed that integrating early HTA in the innovation process has the possibility to shape and refine an innovation, and inform research and development decisions. The early assessment, using a variety of methodologies, can provide insights that are relevant for all stakeholders but several challenges, for example, feasibility and responsibility, need to be addressed before early HTA can become standard practice. For early evaluations to be successful, all relevant stakeholders including patients need to be involved. Also, nimble, flexible assessment methods are needed that fit the dynamics of medical technology. Best practices should be shared to optimize both the innovation process and the methods to perform an early value assessment.

EXPLORING VALUES OF HEALTH TECHNOLOGY ASSESSMENT AGENCIES USING REFLECTIVE MULTICRITERIA AND RARE DISEASE CASE.

OBJECTIVES: Tackling ethical dilemmas faced by reimbursement decision makers requires deeper understanding of values on which health technology assessment (HTA) agencies are founded and how trade-offs are made. This was explored in this study including the case of rare disease. METHODS: Representatives from eight HTA explored values on which institutions are founded using a narrative approach and reflective multicriteria (developed from EVIDEM, criteria derived from ethical imperatives of health care). Trade-offs between criteria and the impact of incorporating defined priorities (including for rare diseases) were explored through a quantitative values elicitation exercise. RESULTS: Participants reported a diversity of substantive and procedural values with a common emphasis on scientific excellence, stakeholder involvement, independence, and transparency. Examining the ethical imperatives behind EVIDEM criteria was found to be useful to further explore substantive values. Most criteria were deemed to reflect institutions' values, while 70 percent of the criteria were reported by at least half of participants to be considered formally by their institutions. The quantitative values elicitation highlighted the difficulty to balance imperatives of "alleviating or preventing patient suffering," "serving the whole population equitably," "upholding healthcare system sustainability," and "making decisions informed by evidence and context" but may help share the ethical reasoning behind decisions. Incorporating "Priorities" (including for rare diseases) helped reveal trade-offs from other criteria and their underlying ethical imperatives. CONCLUSIONS: Reflective multicriteria are useful to explore substantive values of HTAs, reflect how these values and their ethical underpinnings can be operationalized into criteria, and explore the ethical reasoning at the heart of the healthcare debate.

Standardized reporting for rapid relative effectiveness assessments of pharmaceuticals.

OBJECTIVES: Many European countries perform rapid assessments of the relative effectiveness (RE) of pharmaceuticals as part of the reimbursement decision making process. Increased sharing of information on RE across countries may save costs and reduce duplication of work. The objective of this article is to describe the development of a tool for rapid assessment of RE of new pharmaceuticals that enter the market, the HTA Core Model® for Rapid Relative Effectiveness Assessment (REA) of Pharmaceuticals. METHODS: Eighteen member organisations of the European Network of Health Technology Assessment (EUnetHTA) participated in the development of the model. Different versions of the model were developed and piloted in this collaboration and adjusted accordingly based on feedback on the content and feasibility of the model. RESULTS: The final model deviates from the traditional HTA Core Model® used for assessing other types of technologies. This is due to the limited scope (strong focus on RE), the timing of the assessment (just after market authorisation), and strict timelines (e.g. 90 days) required for performing the assessment. The number of domains and assessment elements was limited and it was decided that the primary information sources should preferably be a submission file provided by the marketing authorisation holder and the European Public Assessment Report. CONCLUSIONS: The HTA Core Model® for Rapid REA (version 3.0) was developed to produce standardised transparent RE information of pharmaceuticals. Further piloting can provide input for possible improvements, such as further refining the assessment elements and new methodological guidance on relevant areas.

Piloting international production of rapid relative effectiveness assessments of pharmaceuticals.

BACKGROUND: This article describes the lessons learned from an international pilot assessment using the first version of the HTA Core Model® and Guidelines for rapid Relative Effectiveness Assessment (REA) of pharmaceuticals based on input from three different perspectives: the assessors, the users (health technology assessment organisations) and the marketing authorisation holder. METHODS: A pilot assessment was performed of pazopanib for the treatment of advanced or metastatic renal cell carcinoma for which 54 individuals from 22 EUnetHTA member organisations from 16 European countries gave their contribution. The work was divided in eight domain teams. Subsequently, results of these domain teams were synthesised in one pilot report. Feedback on the outcomes of the pilot was gathered throughout the project and through structured surveys. RESULTS: The first version of the assessment was produced in six months and consisted of 55 question and answer pairs, 8 domain reports and a synthesis section that combined the results from the different domains. The organisation of the pilot required intense coordination. Main points of criticism on the assessment were the lengthiness of the document and overlap of information throughout the assessment. CONCLUSIONS: A reduction in the number of authoring organisations and individuals participating is necessary to avoid information overlap and increase efficiency in undertaking the assessment. Involving several organisations (e.g. five) in an in-depth review could still ensure the benefit of broad participation from various countries. The focus of a rapid REA should be on the first four domains of the Model.

Pre-market clinical evaluations of innovative high-risk medical devices in Europe.

OBJECTIVES: High-quality clinical evidence is most often lacking when novel high-risk devices enter the European market. At the same time, a randomized controlled trial (RCT) is often initiated as a requirement for obtaining market access in the US. Should coverage in Europe be postponed until RCT data are available? We studied the premarket clinical evaluation of innovative high-risk medical devices in Europe compared with the US, and with medicines, where appropriate. METHODS: The literature and regulatory documents were checked. Representatives from industry, Competent Authorities, Notified Bodies, Ethics Committees, and HTA agencies were consulted. We also discuss patient safety and the transparency of information. RESULTS: In contrast to the US, there is no requirement in Europe to demonstrate the clinical efficacy of high-risk devices in the premarket phase. Patients in Europe can thus have earlier access to a potentially lifesaving device, but at the risk of insufficiently documented efficacy and safety. Variations in the stringency of clinical reviews, both at the level of Notified Bodies and Competent Authorities, do not guarantee patient safety. We tried to document the design of premarket trials in Europe and number of patients exposed, but failed as this information is not made public. Furthermore, the Helsinki Declaration is not followed with respect to the registration and publication of premarket trials. CONCLUSIONS: For innovative high-risk devices, new EU legislation should require the premarket demonstration of clinical efficacy and safety, using an RCT if possible, and a transparent clinical review, preferably centralized.

Sentinel lymph node melanoma metastases: Assessment of tumor burden for clinical prediction of outcome in the first Multicenter Selective Lymphadenectomy Trial (MSLT-I).

PURPOSE: As clinical management decisions in patients with Stage III melanoma have become more complex, precise pathologic characterization of sentinel lymph node (SLN) metastases has become critical to guide management. The extent of SLN involvement correlates with risk of adverse outcomes, but reported methods of disease quantification vary. We examined SLN metastases from patients participating in an international clinical trial and compared several methods of tumor burden quantification. METHODS: SLNs from 146 node-positive patients in the first Multicenter Selective Lymphadenectomy Trial (MSLT-I) were centrally-reviewed and characterized by number of tumor-positive nodes, percent nodal area tumor replacement, maximum dimension of largest metastasis, tumor penetrative depth, number of tumor foci, metastasis microanatomic location, and extracapsular extension. These data were analyzed for correlation with non-SLN metastasis and melanoma-specific survival (MSS). RESULTS: The median number of tumor-involved SLNs was 1. The median maximum metastasis dimension was 1.11 mm. Median SLN area involvement was 1.5%. Tumor burden measures were highly correlated with each other. Factors associated with non-SLN metastasis by univariable analysis were primary tumor ulceration and extent of metastases. Tumor thickness, ulceration, non-SLN metastasis and multiple measures of SLN tumor burden were significantly related to MSS on univariable analysis. After multivariable adjustment, number of involved SLNs (p = 0.05) and percent nodal area tumor replacement (p = 0.02) were independent predictors of MSS. CONCLUSION: Central review of MSLT-I pathology indicates that primary tumor and SLN tumor characteristics predict non-SLN metastasis and MSS. Percent nodal involvement was more powerfully prognostic than the more commonly used maximum dimension of largest metastasis.

When the Evidence Basis Breeds Controversies: Exploring the Value Profile of Robotic Surgery Beyond the Early Introduction Phase.

This article investigates qualitatively the value profile of the da Vinci® surgical robot after almost two decades of extensive clinical use and research. We aimed to understand whether the swiftly growing body of published studies on robotic prostate surgery can now, that is, beyond an early stage, guide decisions on the acquisition, procurement, and public provision of this innovation. We explored both published studies and the perspectives of diverse stakeholders in the Netherlands. Both arenas represent conflicting, often polarised arguments on the (added) value of da Vinci surgery. What was unclear a decade ago due to lack of evidence is now unclear because of controversies about evidence. The article outlines controversial value issues and indicates the unlikelihood that awaiting more research - amid the mantra "further studies are needed" - will resolve the controversy. The study underscores multi-stakeholder deliberation to resolve controversies regarding the value of advanced medical innovations.

Molecular classification of melanomas and nevi using gene expression microarray signatures and formalin-fixed and paraffin-embedded tissue.

Melanoma may be difficult to identify histologically and relatively high rates of misdiagnosis leads to many malpractice claims. Currently separation of melanomas from nevi is based primarily on light microscopic interpretation of hematoxylin and eosin stained sections with limited assistance from immunohistology. To increase the accuracy of discrimination of benign and malignant melanocytic lesions we identified DNA microarray-derived gene expression profiles of different melanocytic lesions and evaluated the performance of these gene signatures as molecular diagnostic tools in the molecular classification and separation of melanomas and nevi. Melanocyte-derived cells were isolated by laser capture microdissection from 165 formalin-fixed and paraffin-embedded melanocytic nevi and melanoma tissue sections. RNA was isolated, amplified, labeled, and hybridized to a custom DNA microarray. In all 120 samples were used to identify differentially expressed genes and generate a gene expression classifier capable of distinguishing between melanomas and nevi. These classifiers were tested by the leave-one-out method and in a blinded study. RT-PCR verified the results. Unsupervised hierarchical clustering identified two distinct lesional groups that closely correlated with the histopathologically identified melanomas and nevi. Analysis of gene expression levels identified 36 significant differentially expressed genes. In comparison with nevi, melanomas expressed higher levels of genes promoting signal transduction, transcription, and cell growth. In contrast, expression of L1CAM (homolog) was reduced in melanomas relative to nevi. Genes differentially expressed in melanomas and nevi, on the basis of molecular signal, sub classified a group of unknown melanocytic lesions as melanomas or nevi and had high concordance rates with histopathology. Gene signatures established using DNA microarray gene expression profiling can distinguish melanomas from nevi, indicating the feasibility of using molecular classification as a supplement to standard histology. Our successful use of a standard formalin-fixed and paraffin-embedded tissue further supports the practicability of combining molecular diagnostic testing with histopathology in evaluation of difficult melanocytic lesions.

Nurturing Societal Values in and Through Health Innovations Comment on "What Health System Challenges Should Responsible Innovation in Health Address?"

Aligning innovation processes in healthcare with health system demands is a societal objective, not always achieved. In line with earlier contributions, Lehoux et al outline priorities for research, public communication, and policy action to achieve this objective. We endorse setting these priorities, while also highlighting a 'commitment gap' in collectively addressing system-level challenges. To acknowledge that stakeholders engaged in dialogue with one another are addressing the commitment gap is not a small step but a giant leap towards realising a socially responsible innovation agenda. Translating system-level demand signals into innovation opportunities is, therefore, the task-cum-art of all stakeholders, one that often prompts them to innovate how they deal with innovations.

Moving Towards Accountability for Reasonableness - A Systematic Exploration of the Features of Legitimate Healthcare Coverage Decision-Making Processes Using Rare Diseases and Regenerative Therapies as a Case Study.

BACKGROUND: The accountability for reasonableness (A4R) framework defines 4 conditions for legitimate healthcare coverage decision processes: Relevance, Publicity, Appeals, and Enforcement. The aim of this study was to reflect on how the diverse features of decision-making processes can be aligned with A4R conditions to guide decision-making towards legitimacy. Rare disease and regenerative therapies (RDRTs) pose special decision-making challenges and offer therefore a useful case study. METHODS: Features operationalizing each A4R condition as well as three different approaches to address these features (cost-per-QALY-focused and multicriteria-based) were defined and organized into a matrix. Seven experts explored these features during a panel run under the Chatham House Rule and provided general and RDRT-specific recommendations. Responses were analyzed to identify converging and diverging recommendations. RESULTS: Regarding Relevance, recommendations included supporting deliberation, stakeholder participation and grounding coverage decision criteria in normative and societal objectives. Thirteen of 17 proposed decision criteria were recommended by a majority of panelists. The usefulness of universal cost-effectiveness thresholds to inform allocative efficiency was challenged, particularly in the RDRT context. RDRTs raise specific issues that need to be considered; however, rarity should be viewed in relation to other aspects, such as disease severity and budget impact. Regarding Publicity, panelists recommended transparency about the values underlying a decision and value judgements used in selecting evidence. For Appeals, recommendations included a life-cycle approach with clear provisions for re-evaluations. For Enforcement, external quality reviews of decisions were recommended. CONCLUSION: Moving coverage decision-making processes towards enhanced legitimacy in general and in the RDRT context involves designing and refining approaches to support participation and deliberation, enhancing transparency, and allowing explicit consideration of multiple decision criteria that reflect normative and societal objectives.

Ethics in HTA: Examining the "Need for Expansion".

The article by Daniels and colleagues on expanding the scope of health technology assessment (HTA) to embrace ethical analysis has received endorsement and criticism from commentators in this journal. Referring to this debate, we examine in this article the extent and locus of ethical analysis in HTA processes. An expansion/no-expansion framing of HTA is, in our view, not very fruitful. We argue that meaningfulness and relevance to the needs of the population are what should determine the extent of ethics in HTA. Once 'relevance' is the guiding principle, engaging in ethical analysis becomes inevitable as values are all over the place in HTA, also in how assessors frame research questions. We also challenge dividing the locus of ethical analysis into assessment and appraisal as this would detach HTA from its purpose, ie, supporting legitimate decision-making. Ethical analysis should therefore be considered integral to the HTA process.

How can we assess the value of complex medical innovations in practice?

Rapid proliferation of medical innovations in the face of demographic changes and scarce resources is demanding a value-conscious entry of medical innovations into health care systems. An inquiry into value gains significance during the early diffusion phase of an innovation and becomes indispensable as the complexity of an innovation increases. In this editorial, we argue that a value assessment must pay attention to the social processes shaping the innovation's adoption and use, in particular, to the "promises" of the technology and actual "practices" with it. Promises and practices represent real-world value as they account for both outcomes and costs in practice. A systematic exploration of these loci of value, using insights from constructive technology assessment, enables us to make well-informed decisions on complex medical technologies.

Understanding the adoption dynamics of medical innovations: affordances of the da Vinci robot in the Netherlands.

This study explored the rather rapid adoption of a new surgical device - the da Vinci robot - in the Netherlands despite the high costs and its controversial clinical benefits. We used the concept 'affordances' as a conceptual-analytic tool to refer to the perceived promises, symbolic meanings, and utility values of an innovation constructed in the wider social context of use. This concept helps us empirically understand robot adoption. Data from 28 in-depth interviews with diverse purposively-sampled stakeholders, and from medical literature, policy documents, Health Technology Assessment reports, congress websites and patients' weblogs/forums between April 2009 and February 2014 were systematically analysed from the perspective of affordances. We distinguished five interrelated affordances of the robot that accounted for shaping and fulfilling its rapid adoption: 'characteristics-related' affordances such as smart nomenclature and novelty, symbolising high-tech clinical excellence; 'research-related' affordances offering medical-technical scientific excellence; 'entrepreneurship-related' affordances for performing better-than-the-competition; 'policy-related' affordances indicating the robot's liberalised provision and its reduced financial risks; and 'communication-related' affordances of the robot in shaping patients' choices and the public's expectations by resonating promising discourses while pushing uncertainties into the background. These affordances make the take-up and use of the da Vinci robot sound perfectly rational and inevitable. This Dutch case study demonstrates the fruitfulness of the affordances approach to empirically capturing the contextual dynamics of technology adoption in health care: exploring in-depth actors' interaction with the technology while considering the interpretative spaces created in situations of use. This approach can best elicit real-life value of innovations, values as defined through the eyes of (potential) users.