Guidelines for Reporting Trial Protocols and Completed Trials Modified Due to the COVID-19 Pandemic and Other Extenuating Circumstances: The CONSERVE 2021 Statement.

Importance: Extenuating circumstances can trigger unplanned changes to randomized trials and introduce methodological, ethical, feasibility, and analytical challenges that can potentially compromise the validity of findings. Numerous randomized trials have required changes in response to the COVID-19 pandemic, but guidance for reporting such modifications is incomplete. Objective: As a joint extension for the CONSORT and SPIRIT reporting guidelines, CONSERVE (CONSORT and SPIRIT Extension for RCTs Revised in Extenuating Circumstances) aims to improve reporting of trial protocols and completed trials that undergo important modifications in response to extenuating circumstances. Evidence: A panel of 37 international trial investigators, patient representatives, methodologists and statisticians, ethicists, funders, regulators, and journal editors convened to develop the guideline. The panel developed CONSERVE following an accelerated, iterative process between June 2020 and February 2021 involving (1) a rapid literature review of multiple databases (OVID Medline, OVID EMBASE, and EBSCO CINAHL) and gray literature sources from 2003 to March 2021; (2) consensus-based panelist meetings using a modified Delphi process and surveys; and (3) a global survey of trial stakeholders. Findings: The rapid review yielded 41 673 citations, of which 38 titles were relevant, including emerging guidance from regulatory and funding agencies for managing the effects of the COVID-19 pandemic on trials. However, no generalizable guidance for all circumstances in which trials and trial protocols might face unanticipated modifications were identified. The CONSERVE panel used these findings to develop a consensus reporting guidelines following 4 rounds of meetings and surveys. Responses were received from 198 professionals from 34 countries, of whom 90% (n = 178) indicated that they understood the concept definitions and 85.4% (n = 169) indicated that they understood and could use the implementation tool. Feedback from survey respondents was used to finalize the guideline and confirm that the guideline's core concepts were applicable and had utility for the trial community. CONSERVE incorporates an implementation tool and checklists tailored to trial reports and trial protocols for which extenuating circumstances have resulted in important modifications to the intended study procedures. The checklists include 4 sections capturing extenuating circumstances, important modifications, responsible parties, and interim data analyses. Conclusions and Relevance: CONSERVE offers an extension to CONSORT and SPIRIT that could improve the transparency, quality, and completeness of reporting important modifications to trials in extenuating circumstances such as COVID-19.

Cost Drivers of a Hospital-Acquired Bacterial Pneumonia and Ventilator-Associated Bacterial Pneumonia Phase 3 Clinical Trial.

Background: Studies indicate that the prevalence of multidrug-resistant infections, including hospital-acquired bacterial pneumonia and ventilator-associated bacterial pneumonia (HABP/VABP), has been rising. There are many challenges associated with these disease conditions and the ability to develop new treatments. Additionally, HABP/VABP clinical trials are very costly to conduct given their complex protocol designs and the difficulty in recruiting and retaining patients. Methods: With input from clinicians, representatives from industry, and the US Food and Drug Administration, we conducted a study to (1) evaluate the drivers of HABP/VABP phase 3 direct and indirect clinical trial costs; (2) to identify opportunities to lower these costs; and (3) to compare (1) and (2) to endocrine and oncology clinical trials. Benchmark data were gathered from proprietary and commercial databases and used to create a model that calculates the fully loaded (direct and indirect) cost of typical phase 3 HABP/VABP endocrine and oncology clinical trials. Results: Results indicate that the cost per patient for a 200-site, 1000-patient phase 3 HABP/VABP study is $89600 per patient. The cost of screen failures and screen failure rates are the main cost drivers. Conclusions: Results indicate that biopharmaceutical companies and regulatory agencies should consider strategies to improve screening and recruitment to decrease HABP/VABP clinical trial costs.

Quantifying the magnitude and cost of collecting extraneous protocol data.

Although most research professionals believe that protocol designs contain a growing number of unnecessary and redundant procedures generating unused data, incurring high cost, and jeopardizing study success, there are no published studies systematically examining this issue. Between November 2011 and May 2012, Tufts Center for the Study of Drug Development conducted a study among a working group of 15 pharmaceutical companies in which a total of 25,103 individual protocol procedures were evaluated and classified using clinical study reports and analysis plans. The results show that the typical later-stage protocol had an average of 7 objectives and 13 end points of which 53.8% are supplementary. One (24.7%) of every 4 procedures performed per phase-III protocol and 17.7% of all phase-II procedures per protocol were classified as "Noncore" in that they supported supplemental secondary, tertiary, and exploratory end points. For phase-III protocols, 23.6% of all procedures supported regulatory compliance requirements and 15.9% supported those for phase-II protocols. The study also found that on average, $1.7 million (18.5% of the total) is spent in direct costs to administer Noncore procedures per phase-III protocol and $0.3 million (13.1% of the total) in direct costs are spent on Noncore procedures for each phase-II protocol. Based on the results of this study, the total direct cost to perform Noncore procedures for all active annual phase-II and phase-III protocols is conservatively estimated at $3.7 billion annually, not including the indirect costs associated with collecting and managing Noncore procedure data and the ethical costs of exposing study volunteers to unnecessary risks associated with conducting extraneous procedures.

Evaluating the completeness and accuracy of MedWatch data.

The Food and Drug Administration's MedWatch system--a voluntary surveillance program--received 600,000 adverse event reports on marketed drugs and devices in 2011. The Food and Drug Administration credits the MedWatch system with improving awareness, and expediting early detection, of drug and device risks and in illuminating the adoption of medical treatments. Reporting bias has been acknowledged as a limitation of the MedWatch system. No systematic assessment of the accuracy and completeness of adverse event reporting has been conducted, yet inaccurate adverse event reporting may lead drug safety professionals to draw incorrect conclusions, manufacturers may be wrongly forced to suspend and withdraw medications and interventions, health professionals may mistakenly alter their clinical practices, and patients may be denied safe and effective treatments. In 2011, the Tufts Center for the Study of Drug Development gathered and analyzed 10.2 million adverse event reports filed with the MedWatch system. Patient information was generally complete and accurate. Suspect product information, on the other hand, showed high levels of incomplete and inaccurate data. Start and end dates of suspect product use had 37% and 23% completion rates, respectively. Dosage level was completed only 31% of the time, and product lot numbers had only a 9% completion rate. More than 25% of the names of reported suspect products were inaccurate, and 31% of suspect product start dates were inaccurate. Higher levels of completion and accuracy were associated with reports filed closer to the date when the adverse event was observed. Implications of the results and suggested improvements are discussed.

A Case Study Assessment on the Rationale for, and Relevance of, Non-Core Protocol Data.

To better understand the nature of Non-Core procedures and derive new insight into protocol simplification and optimization, Tufts CSDD collaborated with the FDA and sponsor companies to assess alignment on the rationale for collecting, and relevance of, Non-Core protocol data. Twelve sponsor companies classified and rated 700 distinct procedures from 19 pivotal trials supporting new drug and biologics approvals. FDA reviewers classified and rated 80 distinct procedures for three of the 19 pivotal trials. The results of this assessment indicate areas of alignment and misalignment. Sponsors and FDA reviewers agreed on the classification for more than half of endpoints. However, FDA reviewers classified a much higher percentage of procedures as Non-Core (26% vs. 18%) with the largest proportion (50%) of these procedures perceived as Core by sponsor companies. Sponsors indicated that one-out-of-six Non-Core procedures were administered due to perceived regulatory requirement and expectation. The results of this study characterize the challenge in aligning the different-and potentially conflicting-imperatives of sponsors and regulators and speak to the importance of more effective FDA-sponsor communication to help simplify protocol designs.

Examining the Association Between DCT Solutions Use and Participant Diversity in Clinical Trials.

BACKGROUND: Whereas anecdotal reports suggest that the use of decentralized clinical trial (DCT) solutions can improve participant diversity in clinical trials there is no quantitative evidence to support such reports. METHODS: Tufts CSDD conducted this initial study based on data collected from prior research and publicly available data drawn from Clinicaltrials.gov to compare and contrast participant diversity in trials which included various DCT solutions - virtual visits or televisits, home visits, devices or wearables, and the use of local labs. RESULTS: The results of this analysis indicate that the use of local labs is associated with a lower percentage of white participants; the use of virtual visits or televisits is associated with a lower percentage of Black participants; and the use of devices or wearables was not associated with any significant change in participant demographics. The use of home visits could not be tested for significant differences.

New Benchmarks on Protocol Amendment Experience in Oncology Clinical Trials.

BACKGROUND: The drug development industry's focus on cancer-related treatments continues to rise, with narrow patient populations and complex procedures increasing the complexity of oncology protocols at an accelerated rate compared to non-oncology drugs. Tufts Center for the Study of Drug Development utilized data from a study investigating the impact of protocol amendments to compare how oncology clinical trials differ from non-oncology and identify opportunities to optimize performance in oncology clinical trials. METHODS: Sixteen drug development industry companies contributed data from 950 protocols and 2,188 amendments to a study conducted in 2022 investigating protocol amendments. Analysis compared differences in amendment impact and causes between 249 oncology and 701 non-oncology protocols. RESULTS: Compared to non-oncology, oncology protocols had a significantly higher prevalence (72.1% and 91.1%, respectively) and number (3.0 and 4.0, respectively) of protocol amendments. Oncology protocols with amendments had significantly lower participant completion rates compared to oncology protocols without amendments, while no significant differences were found among non-oncology. During the COVID-19 pandemic, the study found an increased number of substantial amendments, lower completion rates, and higher dropout rates among oncology protocols compared to before the pandemic. CONCLUSIONS: Efforts to prevent avoidable protocol amendments in the industry have not been effective in oncology, where increasingly complex designs are reflected in difficult to predict cycle times, barriers to recruitment and retention and an increase in protocol amendments.

New Estimates on the Cost of a Delay Day in Drug Development.

Two frequently cited figures by clinical research insiders and observers - the cost of missing a day to generate prescription drug sales and the cost of a day to conduct a clinical trial - are outdated and based on anecdotal evidence. In late 2023, the Tufts Center for the Study of Drug Development conducted empirical research to gather more accurate and granular estimates and to test whether average sales per day have changed over time. 645 drugs launched since 2000, and 409 clinical trial budgets were drawn from commercially available and proprietary data sets and analyzed. The results indicate that a single day equals approximately $500,000 in lost prescription drug or biologic sales, with daily prescription sales for infectious, hematologic, cardiovascular, and gastrointestinal diseases among the highest. The results also show that each year, the average sales per day of prescription drugs and biologics has decreased by approximately $80,000-$100,000. The estimated direct daily cost to conduct a clinical trial is approximately $40,000 per day for phase II and III clinical trials, with those in respiratory, rheumatology, and dermatology having the highest relative daily direct costs.

New Benchmarks on Demographic Disparities in Pivotal Trials Supporting FDA-Approved Drugs and Biologics.

BACKGROUND: A lack of diversity and representation in clinical trials is an established issue in drug development, and the COVID-19 pandemic increased awareness of the problem among the general public. This awareness has led to increased pressure on drug development sponsors, as well as additional attention and regulation from federal bodies, to improve the diversity of clinical trials. This study updates existing baselines regarding demographic disparities, as well as detecting early signs that the situation may be starting to improve. METHODS: Building on an existing dataset, this study collected and analyzed pivotal trial demographic data for drugs and biologics approved by the FDA between 2007 and 2021. Demographic data were collected from applications on the FDA website and clinicaltrials.gov, and compared to indication-specific demographic data when available, or US census estimates when they were not. Regression analyses were used to test for significant trends in reporting of demographic data and representation in pivotal trials, as well as the effect of representation on clinical trial duration and FDA review. RESULTS: Reporting of demographic data has improved significantly for all three demographic categories (sex, racial identity, and ethnic identity) over the observed time period (p < 0.0001). During this time period, overrepresentation of white participants has decreased significantly (p < 0.0001), and representation of Black participants has increased (p = 0.0003). Other racial and ethnic identities did not show significant trends. Representation of demographic subgroups was not significant predictors of trial duration except for the representation of Black participants, which was a negative correlation, indicating that as representation of Black participants increases, trial duration decreases (p = 0.0350).

Quantifying Site Burden to Optimize Protocol Performance.

BACKGROUND: The increase in protocol complexity and the resulting rise in the effort required by investigative sites to implement protocols have been well documented, but existing measures of site burden only offer an incomplete view of the burden experienced by site personnel. The introduction of Decentralized Clinical Trials-trials supported by remote and virtual technologies and services-is expected to impact the burden imposed on sites, but this impact has not yet been systematically measured. METHODS: The Tufts Center for the Study of Drug Development conducted an online survey among clinical research sites worldwide and gathered 355 responses assessing the burden associated with distinct activities and procedures related to the implementation of clinical trial protocols using traditional and decentralized approaches. RESULTS: A high percentage of investigative sites (50.5%) have had no experience with DCT solutions and only a small percentage (6.6%) have participated in fully decentralized clinical trials. Overall, half of respondents view DCT solutions as more burdensome than traditional clinical trials. In general, activities related to operational and managerial aspects of trial implementation were viewed as less burdensome when done remotely, while clinical procedures or elements that require study team-patient interactions were viewed as more burdensome when using DCT approaches versus in-person or traditional methods.

Benchmarking Site Activation and Patient Enrollment.

Clinical trial conduct poses numerous challenges, many pertaining to patient recruitment. The primary objectives of this study were to update benchmarks on site activation and patient enrollment gathered in previous Tufts CSDD studies and examine current usage of recruitment and retention tactics. The data collection focused on site activation and patient enrollment metrics used for studies. Analyses were conducted comparing results from 2012, 2019 and 2023. The results indicate that actual enrollments exceeded planned enrollments for a majority of studies and timelines were shorter than expected. In addition, differences were found for enrollment achievement by global region and site type. Further investigation into studies conducted during a later time frame and post-pandemic could be compared with current benchmarks to examine differences.

New Benchmarks on Protocol Amendment Practices, Trends and their Impact on Clinical Trial Performance.

The Tufts Center for the Study of Drug Development (Tufts CSDD) conducted a follow-up study in 2022 to assess trends in protocol amendment experiences and the impact amendments have had on clinical trial performance, particularly during the COVID-19 pandemic. Sixteen pharmaceutical companies and contract research organizations provided data on 950 protocols and 2188 amendments. The results show that, since 2015, the prevalence of protocols with at least one amendment in phases I-IV has increased substantially (from 57 to 76%) and the mean number of amendments per protocol has increased 60% to 3.3, up from 2.1. Phase I and III protocols saw the highest increases in the mean number of amendments implemented per protocol. A much higher percentage of amendments-77%-were deemed unavoidable with regulatory agency requests and changes to the study strategy as the top reasons cited for amending a protocol. The total average duration to implement an amendment has nearly tripled during the past decade. The time from identifying the need-to-amend to last oversight approval now takes an average of 260 days and the mean duration during which investigative sites operate with different versions of the clinical trial protocol spans 215 days. Protocols that implemented at least one amendment were more effective at increasing patient screening volume and reducing the actual number of patients enrolled relative to plan. Lastly, the prevalence of protocols with at least one amendment and mean number of amendments was significantly higher for protocols conducted during the pandemic.

Comprehensive Assessment of Risk-Based Quality Management Adoption in Clinical Trials.

BACKGROUND: Risk-based monitoring (RBM) and risk-based quality management (RBQM) offer a compelling approach to increase efficiency, speed and quality in clinical trials by prioritizing and mitigating risks related to essential safety and efficacy data. Since 2013, the FDA and EMA have encouraged the use of RBM/RBQM, however adoption has been slow with limited understanding of the barriers to adoption. METHODS: The Tufts Center for the Study of Drug Development conducted an online survey among pharmaceutical, biotechnology, and contract research organizations and gathered 206 responses on 32 distinct RBQM practices. RESULTS: On average, companies implemented RBQM in 57% of their clinical trials. Lower levels of adoption were observed among companies conducting fewer than 25 trials annually (48%) compared to those conducting more than 100 trials annually (63%). Primary barriers to adoption include lack of organizational knowledge and awareness, mixed perceptions of the value proposition of RBQM, and poor change management planning and execution. Insights into improving the level of adoption are discussed.

Oncology drug development and approval of systemic anticancer therapy by the U.S. Food and Drug Administration.

BACKGROUND: Regulatory approval of oncology drugs is the cornerstone of the development process and approval characteristics shape eventual utilization. Approval trends and characteristics provide valuable information for drug developers and regulators and ultimately affect clinicians and patients. METHODS: Indication characteristics were tabulated for drugs approved by the U.S. Food and Drug Administration (FDA) for systemic therapy of malignancies from 1949 through October 2011. Variables included time to approval, initial/supplemental indication, tumor type, stage of disease, specification of protein expression or genetic information, drug class, trial design, concomitant agent, trial size, and endpoint. RESULTS: A total of 121 unique anticancer agents, including 242 unique indications, were approved. The number of trials for each indication has decreased; however, trial size has increased and more randomized controlled trials have been performed. Trial designs have increasingly used time-to-event endpoints and rarely have used symptom-based primary endpoints. Approvals have been primarily single agent, with less emphasis on palliative treatments and increasing emphasis on advanced disease stages and requirements for prior therapy. Molecular specifications in labels have increased, but they are present in less than 30% of recent indications and are not associated with shorter approval times. CONCLUSION: Approval of oncology agents is occurring in increasingly more challenging settings, suggesting gaps between eventual practice and development in potentially suboptimal indications. Molecular specifications promise to enhance development, yet widespread use in label indications has not yet been achieved.

Protocol Design and Performance Benchmarks by Phase and by Oncology and Rare Disease Subgroups.

BACKGROUND: Benchmark data characterizing protocol design practices and performance informs clinical trial design decisions and serves as important baseline measures for assessing protocol design behaviors and their impact during and post-pandemic. METHODS: Tufts CSDD, in collaboration with a working group of 20 major and mid-sized pharmaceutical companies and CROs, gathered phase I-III data from protocols completed just prior to the start of the global pandemic. RESULTS: Data for 187 protocols were analyzed to derive benchmarks overall and for two primary subgroups: oncology vs. non-oncology protocols and rare disease vs. non-rare disease protocols. The results show a continuing upward trend across all protocol design variables. Phase II and III protocols average more endpoints, eligibility criteria, protocol pages; investigative sites; countries and datapoints collected. Oncology and rare disease protocols' enrolled-to-completion rates are much lower, involve a much higher average number of countries and investigative sites, require more planned patient visits and generate considerably more clinical research data. As such, oncology and rare disease clinical trial cycle times are longer-most notably at time periods occurring after study startup and prior to database lock-due to intense patient recruitment and retention challenges. CONCLUSIONS: The results of this study present valuable design insights and comparative baseline measures. The implications of these results and the expected impact of decentralized clinical trials on protocol design practices and performance is discussed.

Applying Systems Thinking to Inform Decentralized Clinical Trial Planning and Deployment.

Recently, there has been a growing interest in understanding how decentralized clinical trial (DCT) solutions can mitigate existing challenges in clinical development, particularly participant burden and access, and the collection, management, and quality of clinical data. This paper examines DCT deployments, emphasizing how they are integrated and how they may impact clinical trial oversight, management, and execution. We propose a conceptual framework that employs systems thinking to evaluate the impact on key stakeholders through a reiterative assessment of pain points. We conclude that decentralized solutions should be customized to meet patient needs and preferences and the unique requirements of each clinical trial. We discuss how DCT elements introduce new demands and pressures within the existing system and reflect on enablers that can overcome DCT implementation challenges. As stakeholders look for ways to make clinical research more relevant and accessible to a larger and more diverse patient population, further robust and granular research is needed to quantify the impact of DCTs empirically.

Insights from a Multi-company Workshop to Apply a Patient Participation Burden Algorithm to Protocol Data.

BACKGROUND: Utilizing a participation burden algorithm developed in a previous study, Tufts CSDD, in collaboration with ZS, led a workshop among 8 pharmaceutical companies to validate the methodology of benchmarking the participation burden of a set of retrospective protocols and comparing these data to a prospective protocol design. METHODS: Eight participating companies collected data for 66 retrospective protocols and participation burden scores were calculated for each. Data from one prospective protocol was provided and prospective burden scores were compared to mean retrospective protocol burden for each company. Participating companies provided feedback on data collection process and final reports. RESULTS: Comparisons between retrospective and prospective burden scores revealed higher comparative burden in lab and blood procedures. Companies were able to gather most requested data, but some variables hypothesized to affect burden were not available to sponsors. Time constraints were reported as a challenge throughout the data collection process. CONCLUSIONS: Feedback indicated the need for establishing a larger database to enable comparisons between protocols with the same therapeutic area and indication. Investigating the impact of standard of care burden by indication on overall participation burden and encouraging sponsors to collect more accurate data contributing to participation burden at the site level are also important takeaways from this exercise.

Benchmarking Patient Engagement Capabilities and Preparedness of Drug Development Sponsors.

Consistent implementation and measurement of patient engagement initiatives across the industry have remained aspirational and elusive despite strong interest in adopting patient-centric approaches. One factor contributing to this inertia stems from a lack of standardized implementation of patient engagement activities, which varies widely from company to company, making it difficult to track and measure. Further, empirical evidence mapping the impact of patient engagement capabilities on clinical research outcomes has remained sparse. To address this gap, the Drug Information Association (DIA) and Tufts Center for the Study of Drug Development (Tufts CSDD) at the Tufts University School of Medicine developed and administered an assessment tool that companies can use to not only evaluate their organization's patient engagement capabilities and implementation preparedness but can also measure the impact of such activities on trial outcomes. Results showed that while most organizations are providing logistical support to increase patient engagement in the form of travel stipends, accommodation, and financial incentives, most are not implementing more involved forms of patient engagement such as gathering patient input through patient input panels or patient steering committees. This paper discusses the process for designing and administering this assessment tool, the results of the assessment, and future implications.

Assessing the Financial Value of Decentralized Clinical Trials.

BACKGROUND: Deployment of remote and virtual clinical trial methods and technologies, referred to collectively as decentralized clinical trials (DCTs), represents a profound shift in clinical trial practice. To our knowledge, a comprehensive assessment of the financial net benefits of DCTs has not been conducted. METHODS: We developed an expected net present value (eNPV) model of the cash flows for new drug development and commercialization to assess the financial impact of DCTs. The measure of DCT value is the increment in eNPV that occurs, on average, when DCT methods are employed in comparison to when they are not. The model is populated with parameter values taken from published studies, Tufts CSDD benchmark data, and Medable Inc. data on DCT projects. We also calculated the return on investment (ROI) in DCTs as the ratio of the increment in eNPV to the DCT implementation cost. RESULTS: We found substantial value from employing DCT methods in phase II and phase III trials. If we assume that DCT methods are applied to both phase II and phase III trials the increase in value is $20 million per drug that enters phase II, with a seven-fold ROI. CONCLUSIONS: DCTs can provide substantial extra value to sponsors developing new drugs, with high returns to investment in these technologies. Future research on this topic should focus on expanding the data to larger datasets and on additional aspects of clinical trial operations not currently measured.