Quality Assessment of Tumor Size Data Collection for Pancreatic and Breast Cancer in SEER.

BACKGROUND AND OBJECTIVES: In 2017, the Surveillance, Epidemiology, and End Results (SEER) program piloted a reactive quality audit plan (r-QAP) to analyze Collaborative Stage (CS) tumor size in breast and pancreatic cancer. Preevaluation objectives were to establish procedures and analytic scope for SEER quality audits, cutoffs for data completeness/accuracy, and key decision checkpoints. METHODS: Tumor size data between 2004-2014 were selected from SEER registries for pancreatic and breast cancers, and initially assessed by site and registry for completeness. Further exploration was undertaken via cross tabulation in SEER with the American Joint Committee on Cancer (AJCC) 6th edition derived T data item to evaluate discrepancies between these closely related variables. RESULTS: For both cancer sites, completeness improved between 2004 and 2014, with the proportion of known tumor size values increasing from 60.6% to 79.2% in pancreatic cancer and from 94.0% to 95.9% in breast cancer. Tumor size plausibility categories were established wherein any tumor over 100 mm for pancreatic cancer or over 200 mm for breast cancer were considered highly unlikely. Only 2% of pancreas tumors and 0.1% of breast tumors were implausibly large per site-specific cutoffs. Less than 2% of all tumor size values were potentially discrepant in cross-tabulation with AJCC 6th edition derived T for each site. CONCLUSIONS: Most tumor size values appear to fall within acceptable ranges based on r-QAP activities, and implausibly large tumor size values are rare. Different natural histories and clinical presentation for pancreatic and breast cancer illustrate the need for site-specific cutoffs. Our results indicate that there are no major quality issues in the SEER research database for the CS tumor size data item in either pancreatic or breast cancer.

A Randomized Controlled Trial of an Additional Funding Intervention to Improve Clinical Trial Enrollment.

Background: A low proportion of adults with cancer are recruited to clinical trials. Cancer Council Victoria provides funding to clinical trial sites through its statewide Cancer Trials Management Scheme (CTMS). Historically, there appeared to be a relationship between budget-allocated funding and the number of patients recruited. A randomized controlled trial was conducted to test whether additional funding in 2013 would increase trial recruitment. Methods: A total of 18 trial centers ("sites") received usual CTMS funds, whereas 16 intervention sites received usual funds plus additional funds, proportional to recruitment in 2011; additional payments to sites in the intervention group ranged from $6,750 to $234,000 AUD (≈$6,750-$234,000 USD at the time). This represented an average 11.8% (interquartile range [IQR], 8.0%, 12.3%) increase in sites' budgets. Sites were required to use the funds with the aim of increasing recruitment. The study end point was the number of new participants recruited to trials in 2013. An online survey assessed strategies used to increase recruitment. Results: The median number of new trial recruits per site in 2013 was 21 (IQR, 5-39) in the control arm and 12.5 (IQR, 3.5-44.5) in the intervention arm. The ratio of new trial recruitment numbers at the intervention sites compared with control sites in 2013, adjusting for respective 2012 numbers and institution type, was 0.99 (95% CI, 0.69, 1.43; P=.96). The survey revealed most intervention sites used funding to increase staffing. Conclusions: Additional funding at a site level did not lead to a contemporaneous increase in trial recruitment.

Raising Public Awareness of Clinical Trials: Development of Messages for a National Health Communication Campaign.

Clinical trials are essential for developing new and effective treatments and improving patient quality of life; however, many trials cannot answer their primary research questions because they fall short of their recruitment goals. This article reports the results of formative research conducted in two populations, the public and primary care physicians, to identify messages that may raise awareness and increase interest in clinical trials and be used in a national communication campaign. Results suggested that participants were primarily motivated to participate in clinical trials out of a self-interest to help themselves first. Messages illustrated that current treatments were tested via clinical trials, helped normalize trials as routine practices, and reduced concerns over trying something new first. Participants wanted messages that portray trials as state-of-the-art choices that offer some hope, show people like themselves, and are described in a clear, concise manner with actionable steps for them to take. The study revealed some differences in message salience, with healthy audiences exhibiting lower levels of interest. Our results suggest that targeted messages are needed, and that communication with primary health-care providers is an important and necessary component in raising patient awareness of the importance of clinical trials.

A Novel Approach to Measuring Efficiency of Scientific Research Projects: Data Envelopment Analysis.

PURPOSE: Measuring the efficiency of resource allocation for the conduct of scientific projects in medical research is difficult due to, among other factors, the heterogeneity of resources supplied (e.g., dollars or FTEs) and outcomes expected (e.g., grants, publications). While this is an issue in medical science, it has been approached successfully in other fields by using data envelopment analysis (DEA). DEA has a number of advantages over other techniques as it simultaneously uses multiple heterogeneous inputs and outputs to determine which projects are performing most efficiently, referred to as being at the efficiency frontier, when compared to others in the data set. METHOD: This research uses DEA for the evaluation of supported translational science projects by the Oregon Clinical and Translational Research Institute (OCTRI), a NCATS Clinical & Translational Science Award (CTSA) recipient. RESULTS: These results suggest that the primary determinate of overall project efficiency at OCTRI is the amount of funding, with smaller amounts of funding providing more efficiency than larger funding amounts. CONCLUSION: These results, and the use of DEA, highlight both the success of using this technique in helping determine medical research efficiency and those factors to consider when distributing funds for new projects at CTSAs.

Building a roadmap for developing combination therapies for Alzheimer's disease.

Combination therapy has proven to be an effective strategy for treating many of the world's most intractable diseases. A growing number of investigators in academia, industry, regulatory agencies, foundations and advocacy organizations are interested in pursuing a combination approach to treating Alzheimer's disease. A meeting co-hosted by the Accelerate Cure/Treatments for Alzheimer's Disease Coalition, the Critical Path Institute and the Alzheimer's Association addressed challenges in designing clinical trials to test multiple treatments in combination and outlined a roadmap for making such trials a reality.

Charting a path toward combination therapy for Alzheimer's disease.

It is acknowledged that progress in combined therapeutic approaches for Alzheimer's disease (AD) will require an unprecedented level of collaboration. At a meeting co-hosted by the Accelerate Cure/Treatments for Alzheimer's Disease Coalition and the Critical Path Institute, investigators from industry, academia and regulatory agencies agreed on the need for combinatorial approaches to treating AD. The need for advancing multiple targets includes recognition for novel adaptive trial designs that incorporate existing and new biomarkers to evaluate drug effects independently and in combination. A combination trial now being planned may test drugs targeting different pathogenic pathways or multiple targets along a common pathway. Collaborations and consortia-based strategies are pivotal for success and a regulatory framework is recommended for success.

The National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium: summary and recommendations.

INTRODUCTION: Many challenges to clinical trial accrual exist, resulting in studies with inadequate enrollment and potentially delaying answers to important scientific and clinical questions. METHODS: The National Cancer Institute (NCI) and the American Society of Clinical Oncology (ASCO) cosponsored the Cancer Trial Accrual Symposium: Science and Solutions on April 29-30, 2010 to examine the state of accrual science related to patient/community, physician/provider, and site/organizational influences, and identify new interventions to facilitate clinical trial enrollment. The symposium featured breakout sessions, plenary sessions, and a poster session including 100 abstracts. Among the 358 attendees were clinical investigators, researchers of accrual strategies, research administrators, nurses, research coordinators, patient advocates, and educators. A bibliography of the accrual literature in these three major areas was provided to participants in advance of the meeting. After the symposium, the literature in these areas was revisited to determine if the symposium recommendations remained relevant within the context of the current literature. RESULTS: Few rigorously conducted studies have tested interventions to address challenges to clinical trials accrual. Attendees developed recommendations for improving accrual and identified priority areas for future accrual research at the patient/community, physician/provider, and site/organizational levels. Current literature continues to support the symposium recommendations. CONCLUSIONS: A combination of approaches addressing both the multifactorial nature of accrual challenges and the characteristics of the target population may be needed to improve accrual to cancer clinical trials. Recommendations for best practices and for future research developed from the symposium are provided.

Estimating return on investment in translational research: methods and protocols.

Assessing the value of clinical and translational research funding on accelerating the translation of scientific knowledge is a fundamental issue faced by the National Institutes of Health (NIH) and its Clinical and Translational Awards (CTSAs). To address this issue, the authors propose a model for measuring the return on investment (ROI) of one key CTSA program, the clinical research unit (CRU). By estimating the economic and social inputs and outputs of this program, this model produces multiple levels of ROI: investigator, program, and institutional estimates. A methodology, or evaluation protocol, is proposed to assess the value of this CTSA function, with specific objectives, methods, descriptions of the data to be collected, and how data are to be filtered, analyzed, and evaluated. This article provides an approach CTSAs could use to assess the economic and social returns on NIH and institutional investments in these critical activities.

A "three-plus-one" evaluation model for clinical research management.

Clinical research management (CRM) is a critical resource for the management of clinical trials and it requires proper evaluation. This article advances a model of evaluation that has three local levels, plus one global level, for evaluating the value of CRM. The primary level for evaluation is that of the study or processes level. The managerial or aggregate level concerns management of the portfolio of trials under the control of the CRM office. The third, often overlooked level of evaluation, is the strategic level, whose goal is encapsulated in the phrase, "doing the right trials, while doing trials right." The global ("plus one") evaluation level concerns the need to evaluate the ever-increasing number of multi-institutional and multinational studies. As there are host of evaluation metrics, this article provides representative examples of metrics at each level and provides methods that can aid in the selecting appropriate metrics for an organization.

Characteristics of oncology clinical trials: insights from a systematic analysis of ClinicalTrials.gov.

IMPORTANCE: Clinical trials are essential to cancer care, and data about the current state of research in oncology are needed to develop benchmarks and set the stage for improvement. OBJECTIVE: To perform a comprehensive analysis of the national oncology clinical research portfolio. DESIGN: All interventional clinical studies registered on ClinicalTrials.gov between October 2007 and September 2010 were identified using Medical Subject Heading terms and submitted conditions. They were reviewed to validate classification, subcategorized by cancer type, and stratified by design characteristics to facilitate comparison across cancer types and with other specialties. RESULTS: Of 40 970 interventional studies registered between October 2007 and September 2010, a total of 8942 (21.8%) focused on oncology. Compared with other specialties, oncology trials were more likely to be single arm (62.3% vs 23.8%; P < .001), open label (87.8% vs 47.3%; P < .001), and nonrandomized (63.9% vs 22.7%; P < .001). There was moderate but significant correlation between number of trials conducted by cancer type and associated incidence and mortality (Spearman rank correlation coefficient, 0.56 [P = .04] and 0.77 [P = .001], respectively). More than one-third of all oncology trials were conducted solely outside North America. CONCLUSIONS AND RELEVANCE: There are significant variations between clinical trials in oncology and other diseases, as well as among trials within oncology. The differences must be better understood to improve both the impact of cancer research on clinical practice and the use of constrained resources.

A virtual national laboratory for reengineering clinical translational science.

Clinical research is burdened by inefficiencies and complexities, with a poor record of trial completion, none of which is desirable. The Clinical and Translational Science Award (CTSA) Consortium, including more than 60 clinical research institutions, supports a unified national effort to become, in effect, a virtual national laboratory designed to identify, implement, evaluate, and extend process improvements across all parts of clinical research, from conception to completion. If adequately supported by academic health centers, industry, and funding agencies, the Consortium could become a test bed for improvements that can dramatically reduce wasteful complexity, thus increasing the likelihood of clinical trial completion.

The importance of doing trials right while doing the right trials.

Effort is being expended in investigating efficiency measures (i.e., doing trials right) through achievement of accrual and endpoint goals for clinical trials. It is time to assess the impact of such trials on meeting the critical needs of cancer patients by establishing effectiveness measures (i.e., doing the right trials).

The prevalence and economic impact of low-enrolling clinical studies at an academic medical center.

PURPOSE: The authors assessed the prevalence and associated economic impact of low-enrolling clinical studies at a single academic medical center. METHOD: The authors examined all clinical studies receiving institutional review board (IRB) review between FY2006 and FY2009 at Oregon Health & Science University (OHSU) for recruitment performance and analyzed them by type of IRB review (full-board, exempt, expedited), funding mechanism, and academic unit. A low-enrolling study included those with zero or one participant at the time of study termination. The authors calculated the costs associated with IRB review, financial setup, contract negotiation, and department study start-up activities and the total economic impact on OHSU of low-enrolling studies for FY2009. RESULTS: A total of 837 clinical studies were terminated during the study period, 260 (31.1%) of which were low-enrolling. A greater proportion of low-enrolling studies were government funded than industry funded (P=.006). The authors found significant differences among the various academic units with respect to percentages of low-enrolling studies (from 10% to 67%). The uncompensated economic impact of low-enrolling studies was conservatively estimated to be nearly $1 million for FY2009. CONCLUSIONS: A substantial proportion of clinical studies incurred high institutional and departmental expense but resulted in little scientific benefit. Although a certain percentage of low-enrolling studies can be expected in any research organization, the overall number of such studies must be managed to reduce the aggregate costs of conducting research and to maximize research opportunities. Effective, proactive interventions are needed to address the prevalence and impact of low enrollment.

Predicting accrual achievement: monitoring accrual milestones of NCI-CTEP-sponsored clinical trials.

PURPOSE: The need to increase the number oncology clinical trials with sufficient enrollments is a well-known issue, particularly for trials targeting therapeutic applications. It is critical to identify early predictors of eventual study accrual achievement. EXPERIMENTAL DESIGN: All nonpediatric phase I, I/II, II, and III therapeutic studies supported by the National Cancer Institute Cancer Therapy Evaluation Program (NCI-CTEP) between 2000 and 2007 (n = 764) were analyzed for accrual performance. Accrual achievement is defined as those enrolling 100% or more of the stated minimum accrual goal at the time of trial closure. Two accrual milestones were analyzed per trial: time to first patient enrollment and expected time to accrual goal. Multivariate logistic regression analysis was used to calculate the OR with respect to the likelihood of clinical trial accrual achievement. RESULTS: A total of 81.5% (n = 623) of the trials did not achieve the projected accrual goals within the anticipated accruing period. Furthermore, 37.2% (n = 284) of trials failed to achieve the minimum projected accrual at study closure regardless of time the trial was open. Trials that accrue the first enrollment beyond 2 months (n = 379, 49.6%) are significantly less likely to achieve the accrual performance than those trials that enroll patients under 2 months (OR: 0.637, 95% CI: 0.464-0.875, P = 0.005). Of the studies that are open beyond the anticipated enrollment period (n = 603), those do not achieve at least 60.0% of the projected minimum accrual (n = 391, 64.8%) have a significantly less likelihood of achieving final accruals by study closure (OR: 0.190, 95% CI: 0.055-0.652, P = 0.008). CONCLUSIONS: The time to first patient enrollment and expected time to accrual goal are shown to be valid measures to evaluate the likelihood of achieving the minimum projected accrual.

AccrualNet: Addressing Low Accrual Via a Knowledge-Based, Community of Practice Platform.

PURPOSE: Present the design and initial evaluation of a unique, Web-enabled platform for the development of a community of practice around issues of oncology clinical trial accrual. METHODS: The National Cancer Institute (NCI) conducted research with oncology professionals to identify unmet clinical trial accrual needs in the field. In response, a comprehensive platform for accrual resources, AccrualNet, was created by using an agile development process, storyboarding, and user testing. Literature and resource searches identified relevant content to populate the site. Descriptive statistics were tracked for resource and site usage. Use cases were defined to support implementation. RESULTS: ACCRUALNET HAS FIVE LEVELS: (1) clinical trial macrostages (prestudy, active study, and poststudy); (2) substages (developing a protocol, selecting a trial, preparing to open, enrolling patients, managing the trial, retaining participants, and lessons learned); (3) strategies for each substage; (4) multiple activities for each strategy; and (5) multiple resources for each activity. Since its launch, AccrualNet has had more than 45,000 page views, with the Tools & Resources, Conversations, and Training sections being the most viewed. Total resources have increased 69%, to 496 items. Analysis of articles in the site reveals that 22% are from two journals and 46% of the journals supplied a single article. To date, there are 29 conversations with 43 posts. Four use cases are discussed. CONCLUSION: AccrualNet represents a unique, centralized comprehensive-solution platform to systematically capture accrual knowledge for all stages of a clinical trial. It is designed to foster a community of practice by encouraging users to share additional strategies, resources, and ideas.

Phase III clinical trial development: a process of chutes and ladders.

PURPOSE: The Institute of Medicine report on cooperative groups and the National Cancer Institute (NCI) report from the Operational Efficiency Working Group both recommend changes to the processes for opening a clinical trial. This article provides evidence for the need for such changes by completing the first comprehensive review of all the time and steps required to open a phase III oncology clinical trial and discusses the effect of time to protocol activation on subject accrual. METHODS: The Dilts and Sandler method was used at four cancer centers, two cooperative groups, and the NCI Cancer Therapy Evaluation Program. Accrual data were also collected. RESULTS: Opening a phase III cooperative group therapeutic trial requires 769 steps, 36 approvals, and a median of approximately 2.5 years from formal concept review to study opening. Time to activation at one group ranged from 435 to 1,604 days, and time to open at one cancer center ranged from 21 to 836 days. At centers, group trials are significantly more likely to have zero accruals (38.8%) than nongroup trials (20.6%; P < 0.0001). Of the closed NCI Cancer Therapy Evaluation Program-approved phase III clinical trials from 2000 to 2007, 39.1% resulted in <21 accruals. CONCLUSIONS: The length, variability, and low accrual results demonstrate the need for the NCI clinical trials system to be reengineered. Improvements will be of only limited effectiveness if done in isolation; there is a need to return to the collaborative spirit with all parties creating an efficient and effective system. Recommendations put forth by the Institute of Medicine and Operational Efficiency Working Group reports, if implemented, will aid this renewal.

A sense of urgency: Evaluating the link between clinical trial development time and the accrual performance of cancer therapy evaluation program (NCI-CTEP) sponsored studies.

PURPOSE: Postactivation barriers to oncology clinical trial accruals are well documented; however, potential barriers prior to trial opening are not. We investigate one such barrier: trial development time. EXPERIMENTAL DESIGN: National Cancer Institute Cancer Therapy Evaluation Program (CTEP)-sponsored trials for all therapeutic, nonpediatric phase I, I/II, II, and III studies activated between 2000 and 2004 were investigated for an 8-year period (n = 419). Successful trials were those achieving 100% of minimum accrual goal. Time to open a study was the calendar time from initial CTEP submission to trial activation. Multivariate logistic regression analysis was used to calculate unadjusted and adjusted odds ratios (OR), controlling for study phase and size of expected accruals. RESULTS: Among the CTEP-approved oncology trials, 37.9% (n = 221) failed to attain the minimum accrual goals, with 70.8% (n = 14) of phase III trials resulting in poor accrual. A total of 16,474 patients (42.5% of accruals) accrued to those studies were unable to achieve the projected minimum accrual goal. Trials requiring less than 12 months of development were significantly more likely to achieve accrual goals (OR, 2.15; 95% confidence interval, 1.29-3.57, P = 0.003) than trials with the median development times of 12 to 18 months. Trials requiring a development time of greater than 24 months were significantly less likely to achieve accrual goals (OR, 0.40; 95% confidence interval, 0.20-0.78; P = 0.011) than trials with the median development time. CONCLUSIONS: A large percentage of oncology clinical trials do not achieve minimum projected accruals. Trial development time appears to be one important predictor of the likelihood of successfully achieving the minimum accrual goals.