Silence is golden, but my measures still see-why cheaper-but-noisier outcome measures in large simple trials can be more cost-effective than gold standards.

OBJECTIVE: To assess the cost-effectiveness of using cheaper-but-noisier outcome measures, such as a short questionnaire, for large simple clinical trials. BACKGROUND: To detect associations reliably, trials must avoid bias and random error. To reduce random error, we can increase the size of the trial and increase the accuracy of the outcome measurement process. However, with fixed resources, there is a trade-off between the number of participants a trial can enrol and the amount of information that can be collected on each participant during data collection. METHODS: To consider the effect on measurement error of using outcome scales with varying numbers of categories, we define and calculate the variance from categorisation that would be expected from using a category midpoint; define the analytic conditions under which such a measure is cost-effective; use meta-regression to estimate the impact of participant burden, defined as questionnaire length, on response rates; and develop an interactive web-app to allow researchers to explore the cost-effectiveness of using such a measure under plausible assumptions. RESULTS: An outcome scale with only a few categories greatly reduced the variance of non-measurement. For example, a scale with five categories reduced the variance of non-measurement by 96% for a uniform distribution. We show that a simple measure will be more cost-effective than a gold-standard measure if the relative increase in variance due to using it is less than the relative increase in cost from the gold standard, assuming it does not introduce bias in the measurement. We found an inverse power law relationship between participant burden and response rates such that a doubling the burden on participants reduces the response rate by around one third. Finally, we created an interactive web-app ( https://benjiwoolf.shinyapps.io/cheapbutnoisymeasures/ ) to allow exploration of when using a cheap-but-noisy measure will be more cost-effective using realistic parameters. CONCLUSION: Cheaper-but-noisier questionnaires containing just a few questions can be a cost-effective way of maximising power. However, their use requires a judgement on the trade-off between the potential increase in risk of information bias and the reduction in the potential of selection bias due to the expected higher response rates.

The performance of a Bayesian value-based sequential clinical trial design in the presence of an equivocal cost-effectiveness signal: evidence from the HERO trial.

BACKGROUND: There is increasing interest in the capacity of adaptive designs to improve the efficiency of clinical trials. However, relatively little work has investigated how economic considerations - including the costs of the trial - might inform the design and conduct of adaptive clinical trials. METHODS: We apply a recently published Bayesian model of a value-based sequential clinical trial to data from the 'Hydroxychloroquine Effectiveness in Reducing symptoms of hand Osteoarthritis' (HERO) trial. Using parameters estimated from the trial data, including the cost of running the trial, and using multiple imputation to estimate the accumulating cost-effectiveness signal in the presence of missing data, we assess when the trial would have stopped had the value-based model been used. We used re-sampling methods to compare the design's operating characteristics with those of a conventional fixed length design. RESULTS: In contrast to the findings of the only other published retrospective application of this model, the equivocal nature of the cost-effectiveness signal from the HERO trial means that the design would have stopped the trial close to, or at, its maximum planned sample size, with limited additional value delivered via savings in research expenditure. CONCLUSION: Evidence from the two retrospective applications of this design suggests that, when the cost-effectiveness signal in a clinical trial is unambiguous, the Bayesian value-adaptive design can stop the trial before it reaches its maximum sample size, potentially saving research costs when compared with the alternative fixed sample size design. However, when the cost-effectiveness signal is equivocal, the design is expected to run to, or close to, the maximum sample size and deliver limited savings in research costs.

Real-world clinical and economic outcomes for patients with advanced non-small cell lung cancer enrolled in a clinical trial following comprehensive genomic profiling via liquid biopsy.

BACKGROUND: Oncology clinical trial enrollment is strongly recommended for patients with cancer who are not eligible for established and approved therapies. Many trials are specific to biomarker-targeted therapies, which are typically managed as specialty pharmacy services. Comprehensive genomic profiling (CGP) of advanced cancers has been shown to detect biomarkers, guide targeted treatment, improve outcomes, and result in the clinical trial enrollment of patients, which is modeled to offset pharmacy costs experienced by US payers, yet payer policy coverage remains inconsistent. A common concern limiting coverage of CGP by payers is the potential of identifying biomarkers beyond guideline-recommended treatments, which creates a perception that insurance companies are being positioned to "pay for research." However, these biomarkers can increase clinical trial eligibility, and specialty pharmacy management may have an interest in maximizing the clinical trial enrollment of members. OBJECTIVE: To investigate if clinical trial enrollment following liquid biopsy CGP for non-small cell lung cancer (NSCLC) is clinically and/or economically impactful from a payer claims perspective. METHODS: Clinical and economic outcomes were studied using a real-world clinical genomic database (including payer claims data) from patients with NSCLC who enrolled in clinical trials immediately following liquid biopsy CGP (using Guardant360) and matched NSCLC patient controls also tested with liquid biopsy CGP. RESULTS: Real-world overall survival was significantly (log-rank P < 0.0001) better for patients enrolled in clinical trials with similar costs of care, albeit with more outpatient encounters among those enrolled compared with matched controls. CONCLUSIONS: The results, together with previous analyses, suggest that, in addition to the clinical benefits associated with targeted therapies directed by CGP and other testing approaches, payers and specialty pharmacy managers may consider clinical trial direction and enrollment as a clinical and economic benefit of liquid biopsy CGP and adopt this into coverage decision frameworks and formularies.

Assessing the net financial benefits of employing digital endpoints in clinical trials.

In the last few decades, developers of new drugs, biologics, and devices have increasingly leveraged digital health technologies (DHTs) to assess clinical trial digital endpoints. To our knowledge, a comprehensive assessment of the financial net benefits of digital endpoints in clinical trials has not been conducted. We obtained data from the Digital Medicine Society (DiMe) Library of Digital Endpoints and the US clinical trials registry, ClinicalTrials.gov. The benefit metrics are changes in trial phase duration and enrollment associated with the use of digital endpoints. The cost metric was obtained from an industry survey of the costs of including digital endpoints in clinical trials. We developed an expected net present value (eNPV) model of the cash flows for new drug development and commercialization to assess financial value. The value measure is the increment in eNPV that occurs when digital endpoints are employed. We also calculated a return on investment (ROI) as the ratio of the estimated increment in eNPV to the mean digital endpoint implementation cost. For phase II trials, the increase in eNPV varied from $2.2 million to $3.3 million, with ROIs between 32% and 48% per indication. The net benefits were substantially higher for phase III trials, with the increase in eNPV varying from $27 million to $40 million, with ROIs that were four to six times the investment. The use of digital endpoints in clinical trials can provide substantial extra value to sponsors developing new drugs, with high ROIs.

Joint recommendations on cost calculation and estimation in paediatric clinical trials.

The conduct of clinical trials in paediatrics is essential to improve drug therapy in children. In Europe, paediatric clinical trials have been supported by the European Paediatric Regulation since 2007, but there is still a great need for high-quality clinical trials. The personnel and time required to conduct clinical trials in accordance with EU Regulations 536/2014 and 745/2017 is considerably higher compared to other studies, such as observational studies. It is important that this additional workload for the trial centre is fully compensated, also taking into account EU state aid rules. In paediatric trials, it is necessary to take into account the special requirements of paediatric and adolescent medicine when calculating the additional costs. Within the framework of the pan-European paediatric study network c4c/GermanNetPaeT, a working group dealt with specific aspects of cost calculation in order to support paediatric study centres in internal cost calculation as well as in the subsequent preparation of financing requirements for industrial sponsors or public funders. In several workshops the working group developed a cost calculation template with the content derived from the "Joint recommendations for a total services account as a factor in simplifying contracts" of the Deutsche Hochschulmedizin (DHM, German University Medicine), the Netzwerk der Koordinierungszentren für Klinische Studien (KKS Network, Network of Coordinating Centres for Clinical Trials) and the Verband Forschender Arzneimittelhersteller (vfa, German Association of Research-Based Pharmaceutical Companies). By estimating the specific time required for measures and investigations as part of a sample study, the background to the increased time required was discussed and a list with aspects to be considered for cost calculation was compiled together with the study centres. The paediatrics-specific aspects mentioned in detail are intended to increase understanding of the particular problem of higher costs for clinical trials involving children and adolescents and the need for correspondingly appropriate remuneration. This transparent and comprehensible presentation of the higher financial requirements for both the study centres and the financial supporters is intended to promote the high-quality conduct of clinical trials in paediatric study centres in the long term.

Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators.

Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.

Refining the impact of genetic evidence on clinical success.

The cost of drug discovery and development is driven primarily by failure1, with only about 10% of clinical programmes eventually receiving approval2-4. We previously estimated that human genetic evidence doubles the success rate from clinical development to approval5. In this study we leverage the growth in genetic evidence over the past decade to better understand the characteristics that distinguish clinical success and failure. We estimate the probability of success for drug mechanisms with genetic support is 2.6 times greater than those without. This relative success varies among therapy areas and development phases, and improves with increasing confidence in the causal gene, but is largely unaffected by genetic effect size, minor allele frequency or year of discovery. These results indicate we are far from reaching peak genetic insights to aid the discovery of targets for more effective drugs.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies.

BACKGROUND: The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status. METHODS: We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results. RESULTS: The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]). CONCLUSIONS: Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed. TRIAL REGISTRATION: https://osf.io/q5hcs.

A multidimensional assessment of payback: Evaluating the impact of Breast Cancer Trials.

AIM: This study evaluates the impact created by clinical trial investment and research undertaken by Breast Cancer Trials (BCT). METHODS: The approach involved using the Payback Framework, and included a review of BCT archival information, public health data, a questionnaire sent to BCT members, individual interviews with key informants, a focus group with members of the organization's Consumer Advisory Panel, and case studies of select BCT trials. The evaluation assessed the evidence against the Payback Framework criteria: informing policy and product development, knowledge production, the research system, health and health sector benefits, and broader economic benefits. RESULTS: Analysis using the Payback Framework revealed impact was created in each category and a range of positive outcomes were identified for various stakeholder groups. BCT is maximizing the impact of its research and contributing to a global pool of scientific knowledge by collaborating with over 100 institutions and 820 researchers, yet its benefits go beyond research contributions. Findings highlight strong financial returns from BCT's research program and that BCT has played an important role in enhancing the public perception of clinical trials by promoting participation in trials, educating and empowering participants, and communicating trial outcomes. CONCLUSION: BCT's clinical trials have had a significant impact on the prevention, detection, treatment, and management of breast cancer. Measuring and reporting impact can be resource intensive but support BCT in remaining accountable to all those invested in the organization and in breast cancer clinical trials, evidencing the multiple dimensions of payback resulting from the organization's research.

Consent document translation expense hinders inclusive clinical trial enrolment.

Patients from historically under-represented racial and ethnic groups are enrolled in cancer clinical trials at disproportionately low rates in the USA1-3. As these patients often have limited English proficiency4-7, we hypothesized that one barrier to their inclusion is the cost to investigators of translating consent documents. To test this hypothesis, we evaluated more than 12,000 consent events at a large cancer centre and assessed whether patients requiring translated consent documents would sign consent documents less frequently in studies lacking industry sponsorship (for which the principal investigator pays the translation costs) than for industry-sponsored studies (for which the translation costs are covered by the sponsor). Here we show that the proportion of consent events for patients with limited English proficiency in studies not sponsored by industry was approximately half of that seen in industry-sponsored studies. We also show that among those signing consent documents, the proportion of consent documents translated into the patient's primary language in studies without industry sponsorship was approximately half of that seen in industry-sponsored studies. The results suggest that the cost of consent document translation in trials not sponsored by industry could be a potentially modifiable barrier to the inclusion of patients with limited English proficiency.

Comparison of Availability of Trial Results in ClinicalTrials.gov and PubMed by Data Source and Funder Type.

This study examines the dissemination of trial results by data source (ie, ClinicalTrials.gov and PubMed) and funder type (ie, industry and nonindustry).

Dissemination of the Results of Pediatric Clinical Trials Funded by the US National Institutes of Health.

This study examines practices related to trial registration and results submission in ClinicalTrials.gov and publication of pediatric clinical trials funded by the National Institutes of Health.

Assessing the Financial Value of Decentralized Clinical Trials.

BACKGROUND: Deployment of remote and virtual clinical trial methods and technologies, referred to collectively as decentralized clinical trials (DCTs), represents a profound shift in clinical trial practice. To our knowledge, a comprehensive assessment of the financial net benefits of DCTs has not been conducted. METHODS: We developed an expected net present value (eNPV) model of the cash flows for new drug development and commercialization to assess the financial impact of DCTs. The measure of DCT value is the increment in eNPV that occurs, on average, when DCT methods are employed in comparison to when they are not. The model is populated with parameter values taken from published studies, Tufts CSDD benchmark data, and Medable Inc. data on DCT projects. We also calculated the return on investment (ROI) in DCTs as the ratio of the increment in eNPV to the DCT implementation cost. RESULTS: We found substantial value from employing DCT methods in phase II and phase III trials. If we assume that DCT methods are applied to both phase II and phase III trials the increase in value is $20 million per drug that enters phase II, with a seven-fold ROI. CONCLUSIONS: DCTs can provide substantial extra value to sponsors developing new drugs, with high returns to investment in these technologies. Future research on this topic should focus on expanding the data to larger datasets and on additional aspects of clinical trial operations not currently measured.

Funding Sources of Therapeutic and Vaccine Clinical Trials for COVID-19 vs Non-COVID-19 Indications, 2020-2021.

Importance: Effective COVID-19 vaccines and therapeutics reached the market within the first year of the pandemic. This rate of development and availability was an unprecedented achievement that required attention to numerous research and development, regulatory, and policy challenges. However, only limited evidence is currently available on the sources of funding for COVID-19 clinical trials. Objective: To compare the number and funding sources of clinical trials aimed at investigating therapeutics and vaccines for COVID-19 vs those for all non-COVID-19 indications. Design, Setting, and Participants: In this cross-sectional study, clinical trials in phase 1 to 3 that were registered to start between January 1, 2020, and August 31, 2021, were examined. All relevant data were collected from ClinicalTrials.gov. Main Outcomes and Measures: Number of clinical trials and their funding sources. Results: A total of 1977 clinical trials that addressed COVID-19 therapeutics and vaccines were registered worldwide with starting dates from January 1, 2020, to August 31, 2021. This cohort represented 13.9% of all trials (N = 14 274) during the same period. Most of the COVID-19 therapeutic and vaccine clinical trials were funded by public sources (1144 [57.9%]), followed by industry (540 [27.3%]) and public-private partnerships (293 [14.8%]). Most of these studies focused on the development of anti-COVID-19 therapeutics (1680 [85.0%]) rather than vaccines (297 [15.0%]). Conclusions and Relevance: The findings of this study suggest that publicly funded research and medical institutions played a leading role as funding sources for generating effective COVID-19 therapeutics and vaccines during the first 1.5 years of the pandemic and were most likely instrumental in their rapid development. It may be beneficial for the public sector to maintain the affordability and global access to these therapeutics and vaccines to ensure that they remain available for use worldwide.

A scoping review on patient heterogeneity in economic evaluations of precision medicine based on basket trials.

INTRODUCTION: Considerable challenges in the economic evaluation of precision medicines have been mentioned in previous studies. However, they have not addressed how an economic assessment would be conducted based on basket trials (novel studies for evaluation of precision medicine effects) in which the included populations have specific biomarkers and various cancers. Since basket trial populations have remarkable heterogeneity, this study aims to investigate the concept of heterogeneity and specific method(s) for considering it in economic evaluations through guidelines and studies that could be applicable in economic evaluation based on basket trials. AREA COVERED: We searched PubMed, Web of Science, Scopus, Google Scholar, and Google to find studies and pharmacoeconomics guidelines. The inclusion criteria included subjects of patient heterogeneity and suggested explicit method(s). Thirty-nine guidelines and 43 studies were included and evaluated. None of these materials mentioned disease types in a target population as a factor causing heterogeneity. Moreover, in economic evaluations, patient heterogeneity has been considered with four general approaches subgroup analysis, individual-based models, sensitivity analysis, and regression models. EXPERT OPINION: Type of disease is not considered a contributing factor in population heterogeneity, and the probable appropriate method for this issue could be individual-based models.

How much is the lack of retention evidence costing trial teams in Ireland and the UK?

BACKGROUND: Evidence to support the use of many retention strategies in clinical trials is lacking. Despite this, trial teams still need to have some form of retention strategy in their trials to try and avoid high attrition rates. This study aimed to estimate how much this lack of retention evidence might be costing trials in Ireland and the UK. METHODS: We selected the top ten most routinely used retention strategies by Clinical Trial Units in the UK and made assumptions as to how each of these strategies was most likely to be implemented and the costs involved in doing this. We applied our costing model to a hypothetical trial scenario in both Ireland and the UK as well as to three published trial protocols. We developed the costing model and calculated the costs in Microsoft Excel. RESULTS: Retention strategies were often poorly specified, meaning we had to make assumptions about implementation and in some cases about the strategy itself. Based on our assumptions, some retention strategies can be extremely expensive; some of the costliest strategies included "data collection scheduled with routine care" (€900-€32,503.25), "a timeline of participant visits for sites"-with integrated participant reminder (€304.74-€14,803.70), and "routine site visits by CTU staff" and "investigator meetings face to face", both costing (€777.67-€14,753.48). Others such as "telephone reminders for questionnaire response" (€34.58-€568.62), "a timeline of participant visits for sites"-site reminder alone (€79.18-€112.23), and "targeted recruitment of sites/GPs" (€30-€1620) were less costly compared to the other strategies. DISCUSSION: The resources invested in the use of some retention strategies may outweigh known or imagined benefits on retention. Where benefits are currently unknown, evaluation should be a priority. CONCLUSION: More evaluation of the effectiveness and cost of trial retention strategies is needed to avoid widespread use of strategies that are both expensive and ineffective.