Patient-Centered Clinical Trial Design for Heart Failure Devices via Bayesian Decision Analysis.

BACKGROUND: The statistical significance of clinical trial outcomes is generally interpreted quantitatively according to the same threshold of 2.5% (in one-sided tests) to control the false-positive rate or type I error, regardless of the burden of disease or patient preferences. The clinical significance of trial outcomes-including patient preferences-are also considered, but through qualitative means that may be challenging to reconcile with the statistical evidence. OBJECTIVE: We aimed to apply Bayesian decision analysis to heart failure device studies to choose an optimal significance threshold that maximizes the expected utility to patients across both the null and alternative hypotheses, thereby allowing clinical significance to be incorporated into statistical decisions either in the trial design stage or in the post-trial interpretation stage. In this context, utility is a measure of how much well-being the approval decision for the treatment provides to the patient. METHODS: We use the results from a discrete-choice experiment study focusing on heart failure patients' preferences, questioning respondents about their willingness to accept therapeutic risks in exchange for quantifiable benefits with alternative hypothetical medical device performance characteristics. These benefit-risk trade-off data allow us to estimate the loss in utility-from the patient perspective-of a false-positive or false-negative pivotal trial result. We compute the Bayesian decision analysis-optimal statistical significance threshold that maximizes the expected utility to heart failure patients for a hypothetical two-arm, fixed-sample, randomized controlled trial. An interactive Excel-based tool is provided that illustrates how the optimal statistical significance threshold changes as a function of patients' preferences for varying rates of false positives and false negatives, and as a function of assumed key parameters. RESULTS: In our baseline analysis, the Bayesian decision analysis-optimal significance threshold for a hypothetical two-arm randomized controlled trial with a fixed sample size of 600 patients per arm was 3.2%, with a statistical power of 83.2%. This result reflects the willingness of heart failure patients to bear additional risks of the investigational device in exchange for its probable benefits. However, for increased device-associated risks and for risk-averse subclasses of heart failure patients, Bayesian decision analysis-optimal significance thresholds may be smaller than 2.5%. CONCLUSIONS: A Bayesian decision analysis is a systematic, transparent, and repeatable process for combining clinical and statistical significance, explicitly incorporating burden of disease and patient preferences into the regulatory decision-making process.

Leveraging Patient Preference Information in Medical Device Clinical Trial Design.

Use of robust, quantitative tools to measure patient perspectives within product development and regulatory review processes offers the opportunity for medical device researchers, regulators, and other stakeholders to evaluate what matters most to patients and support the development of products that can best meet patient needs. The medical device innovation consortium (MDIC) undertook a series of projects, including multiple case studies and expert consultations, to identify approaches for utilizing patient preference information (PPI) to inform clinical trial design in the US regulatory context. Based on these activities, this paper offers a cogent review of considerations and opportunities for researchers seeking to leverage PPI within their clinical trial development programs and highlights future directions to enhance this field. This paper also discusses various approaches for maximizing stakeholder engagement in the process of incorporating PPI into the study design, including identifying novel endpoints and statistical considerations, crosswalking between attributes and endpoints, and applying findings to the population under study. These strategies can help researchers ensure that clinical trials are designed to generate evidence that is useful to decision makers and captures what matters most to patients.

Quantifying Benefit-Risk Preferences for Heart Failure Devices: A Stated-Preference Study.

BACKGROUND: Regulatory and clinical decisions involving health technologies require judgements about relative importance of their expected benefits and risks. We sought to quantify heart-failure patients' acceptance of therapeutic risks in exchange for improved effectiveness with implantable devices. METHODS: Individuals with heart failure recruited from a national web panel or academic medical center completed a web-based discrete-choice experiment survey in which they were randomized to one of 40 blocks of 8 experimentally controlled choice questions comprised of 2 device scenarios and a no-device scenario. Device scenarios offered an additional year of physical functioning equivalent to New York Heart Association class III or a year with improved (ie, class II) symptoms, or both, with 30-day mortality risks ranging from 0% to 15%, in-hospital complication risks ranging from 0% to 40%, and a remote adjustment device feature. Logit-based regression models fit participants' choices as a function of health outcomes, risks and remote adjustment. RESULTS: Latent-class analysis of 613 participants (mean age, 65; 49% female) revealed that two-thirds were best represented by a pro-device, more risk-tolerant class, accepting up to 9% (95% CI, 7%-11%) absolute risk of device-associated mortality for a one-year gain in improved functioning (New York Heart Association class II). Approximately 20% were best represented by a less risk-tolerant class, accepting a maximum device-associated mortality risk of 3% (95% CI, 1%-4%) for the same benefit. The remaining class had strong antidevice preferences, thus maximum-acceptable risk was not calculated. CONCLUSIONS: Quantitative evidence on benefit-risk tradeoffs for implantable heart-failure device profiles may facilitate incorporating patients' views during product development, regulatory decision-making, and clinical practice.