Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification.

OBJECTIVES: Early assessments of health technologies help to better align and integrate their development and assessment. Such assessments can take many forms and serve different purposes, hampering users in their selection of the most appropriate method for a specific goal. The aim of this scoping review was to structure the large set of methods according to their specific goal. METHODS: A scoping review was conducted using PubMed and reference lists of retrieved articles, to identify review studies with a methodological focus. From the included reviews, all individual methods were listed. Based on additional literature and examples, we extracted the specific goal of each method. All goals were clustered to derive a set of subclasses and methods were grouped into these subclasses. RESULTS: Of the 404 screened, 5 reviews were included, and 1 was added when searching reference lists. The reviews described 56 methods, of which 43 (77%) were included and classified as methods to (1) explore the nature and magnitude of the problem, (2) estimate the nature and magnitude of the expected (societal) value, (3) identify conditions for the potential value to materialize, and (4) help develop and design the type of research that is needed. CONCLUSIONS: The wide range of methods for exploring the societal value of health technologies at an early stage of development can be subdivided into a limited number of classes, distinguishing methods according to their specific objective. This facilitates selection of appropriate methods, depending on the specific needs and aims.

Clinical biomarker innovation: when is it worthwhile?

Background Choosing which biomarker tests to select for further research and development is not only a matter of diagnostic accuracy, but also of the clinical and monetary benefits downstream. Early health economic modeling provides tools to assess the potential effects of biomarker innovation and support decision-making. Methods We applied early health economic modeling to the case of diagnosing primary aldosteronism in patients with resistant hypertension. We simulated a cohort of patients using a Markov cohort state-transition model. Using the headroom method, we compared the currently used aldosterone-to-renin ratio to a hypothetical new test with perfect diagnostic properties to determine the headroom based on quality-adjusted life-years (QALYs) and costs, followed by threshold analyses to determine the minimal diagnostic accuracy for a cost-effective product. Results Our model indicated that a perfect diagnostic test would yield 0.027 QALYs and increase costs by €43 per patient. At a cost-effectiveness threshold of €20,000 per QALY, the maximum price for this perfect test to be cost-effective is €498 (95% confidence interval [CI]: €275-€808). The value of the perfect test was most strongly influenced by the sensitivity of the current biomarker test. Threshold analysis showed the novel test needs a sensitivity of at least 0.9 and a specificity of at least 0.7 to be cost-effective. Conclusions Our model-based approach evaluated the added value of a clinical biomarker innovation, prior to extensive investment in development, clinical studies and implementation. We conclude that early health economic modeling can be a valuable tool when prioritizing biomarker innovations in the laboratory.

Early Assessment of Proof-of-Problem to Guide Health Innovation.

BACKGROUND: Although the relevance of both push and pull factors is acknowledged in models of innovation, needs, broadly defined, are rarely considered, whereas supply-driven innovation in publicly funded health systems carries the risk that it may not match the underlying problems experienced by patients and consumers. OBJECTIVES: To explore a mixed-methods, multistakeholder approach that focuses on pertinent problems when assessing the potential value of an innovation as applied to a case of surgical innovation in meniscus surgery. METHODS: Through interviews of stakeholders (n = 11) we sought to identify current problems of meniscus surgery in the Netherlands. On the basis of the subsequent problem definitions, we used stakeholder and literature input to quantify the room for improvement and stakeholder engagement to uncover possible barriers and facilitators to the implementation of the proposed innovation. RESULTS: Despite being enthusiastic about the ingenuity of the proposed innovation and seeing some potential for cost saving, most stakeholders (n = 10) agreed that there are no major problems in current meniscus surgery meriting the innovation. They even discerned pragmatic barriers that would challenge the potential cost savings. CONCLUSIONS: By adopting a problem-oriented multistakeholder approach to early health technology assessment, we were able to estimate the potential value of an innovation in its social context, finding that, beyond the initial enthusiasm, the proposed innovation was unlikely to resolve the problems distinguished by the stakeholders. We concluded that our multistakeholder, mixed-methods approach to early health technology assessment is feasible and helps foster more demand-driven innovations.

Exploring the risk-reward balance in focal therapy for prostate cancer-a contribution to the debate.

Focal therapy (FT) for the treatment of localized prostate cancer offers an alternative strategy for men seeking active treatment. Although relatively new, existing studies suggest that the majority of men who undergo FT tend to maintain levels of genito-urinary function that are indistinguishable from their pre-treatment status. However, as part of the shared decision making process, men need to balance good tolerability against a greater risk of recurrence given that much of the prostate remains intact after FT. In order to explore this trade-off, we used decision modelling. Our findings show that the burden of functional complications associated with radical prostatectomy (RP) is considerable, as an average of 243 days of perfect health are lost per patient due to treatment-induced urinary incontinence and erectile dysfunction. Given this effectiveness gap in current care, we explored by how much mortality - as worst-case outcome of disease progression - could increase to still result in net health benefit. To do this we mapped the net health benefit/loss of FT, in comparison to RP, for different levels of function preservation and increases in mortality. We believe our modelling exercise might help inform future studies that seek to enhance our understanding of how men make treatment decisions.

Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes.

Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, health technology assessment agencies (HTAs) presently employ value assessment frameworks that are ill fitted to capture the range and diversity of stakeholder values and thereby risk compromising the legitimacy of their recommendations. We propose "evidence-informed deliberative processes" as an alternative framework with the aim to enhance this legitimacy. This framework integrates two increasingly popular and complementary frameworks for priority setting: multicriteria decision analysis and accountability for reasonableness. Evidence-informed deliberative processes are, on one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making through evidence-informed evaluation of the identified values. The framework has important implications for how HTA agencies should ideally organize their processes. First, HTA agencies should take the responsibility of organizing stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phases, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal. The framework should not be considered a blueprint for HTA agencies but rather an aspirational goal-agencies can take incremental steps toward achieving this goal.

P300 amplitude variations, prior probabilities, and likelihoods: A Bayesian ERP study.

The capability of the human brain for Bayesian inference was assessed by manipulating probabilistic contingencies in an urn-ball task. Event-related potentials (ERPs) were recorded in response to stimuli that differed in their relative frequency of occurrence (.18 to .82). A veraged ERPs with sufficient signal-to-noise ratio (relative frequency of occurrence > .5) were used for further analysis. Research hypotheses about relationships between probabilistic contingencies and ERP amplitude variations were formalized as (in-)equality constrained hypotheses. Conducting Bayesian model comparisons, we found that manipulations of prior probabilities and likelihoods were associated with separately modifiable and distinct ERP responses. P3a amplitudes were sensitive to the degree of prior certainty such that higher prior probabilities were related to larger frontally distributed P3a waves. P3b amplitudes were sensitive to the degree of likelihood certainty such that lower likelihoods were associated with larger parietally distributed P3b waves. These ERP data suggest that these antecedents of Bayesian inference (prior probabilities and likelihoods) are coded by the human brain.

Analyzing indirect effects in cluster randomized trials. The effect of estimation method, number of groups and group sizes on accuracy and power.

Cluster randomized trials assess the effect of an intervention that is carried out at the group or cluster level. Ajzen's theory of planned behavior is often used to model the effect of the intervention as an indirect effect mediated in turn by attitude, norms and behavioral intention. Structural equation modeling (SEM) is the technique of choice to estimate indirect effects and their significance. However, this is a large sample technique, and its application in a cluster randomized trial assumes a relatively large number of clusters. In practice, the number of clusters in these studies tends to be relatively small, e.g., much less than fifty. This study uses simulation methods to find the lowest number of clusters needed when multilevel SEM is used to estimate the indirect effect. Maximum likelihood estimation is compared to Bayesian analysis, with the central quality criteria being accuracy of the point estimate and the confidence interval. We also investigate the power of the test for the indirect effect. We conclude that Bayes estimation works well with much smaller cluster level sample sizes such as 20 cases than maximum likelihood estimation; although the bias is larger the coverage is much better. When only 5-10 clusters are available per treatment condition even with Bayesian estimation problems occur.

Facing off with Scylla and Charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance.

Measurement invariance (MI) is a pre-requisite for comparing latent variable scores across groups. The current paper introduces the concept of approximate MI building on the work of Muthén and Asparouhov and their application of Bayesian Structural Equation Modeling (BSEM) in the software Mplus. They showed that with BSEM exact zeros constraints can be replaced with approximate zeros to allow for minimal steps away from strict MI, still yielding a well-fitting model. This new opportunity enables researchers to make explicit trade-offs between the degree of MI on the one hand, and the degree of model fit on the other. Throughout the paper we discuss the topic of approximate MI, followed by an empirical illustration where the test for MI fails, but where allowing for approximate MI results in a well-fitting model. Using simulated data, we investigate in which situations approximate MI can be applied and when it leads to unbiased results. Both our empirical illustration and the simulation study show approximate MI outperforms full or partial MI In detecting/recovering the true latent mean difference when there are (many) small differences in the intercepts and factor loadings across groups. In the discussion we provide a step-by-step guide in which situation what type of MI is preferred. Our paper provides a first step in the new research area of (partial) approximate MI and shows that it can be a good alternative when strict MI leads to a badly fitting model and when partial MI cannot be applied.

Illustrating bayesian evaluation of informative hypotheses for regression models.

In the present article we illustrate a Bayesian method of evaluating informative hypotheses for regression models. Our main aim is to make this method accessible to psychological researchers without a mathematical or Bayesian background. The use of informative hypotheses is illustrated using two datasets from psychological research. In addition, we analyze generated datasets with manipulated differences in effect size to investigate how Bayesian hypothesis evaluation performs when the magnitude of an effect changes. After reading this article the reader is able to evaluate his or her own informative hypotheses.