ABSTRACT
OBJECTIVE: To quantify the strength of statistical evidence of randomised controlled trials (RCTs) for novel cancer drugs approved by the Food and Drug Administration (FDA) in the last two decades. STUDY DESIGN AND SETTING: We used data on overall survival (OS), progression-free survival (PFS), and tumour response (TR) for novel cancer drugs approved for the first time by the FDA between January 2000 and December 2020. We assessed strength of statistical evidence by calculating Bayes Factors (BFs) for all available endpoints, and we pooled evidence using Bayesian fixed-effect meta-analysis for indications approved based on two RCTs. Strength of statistical evidence was compared between endpoints, approval pathways, lines of treatment, and types of cancer. RESULTS: We analysed the available data from 82 RCTs corresponding to 68 indications supported by a single RCT and seven indications supported by two RCTs. Median strength of statistical evidence was ambiguous for OS (BF = 1.9; IQR 0.5-14.5), and strong for PFS (BF = 24,767.8; IQR 109.0-7.3*106) and TR (BF = 113.9; IQR 3.0-547,100). Overall, 44 indications (58.7%) were approved without clear statistical evidence for OS improvements and seven indications (9.3%) were approved without statistical evidence for improvements on any endpoint. Strength of statistical evidence was lower for accelerated approval compared to non-accelerated approval across all three endpoints. No meaningful differences were observed for line of treatment and cancer type. LIMITATIONS: This analysis is limited to statistical evidence. We did not consider non-statistical factors (e.g., risk of bias, quality of the evidence). CONCLUSION: BFs offer novel insights into the strength of statistical evidence underlying cancer drug approvals. Most novel cancer drugs lack strong statistical evidence that they improve OS, and a few lack statistical evidence for efficacy altogether. These cases require a transparent and clear explanation. When evidence is ambiguous, additional post-marketing trials could reduce uncertainty.
ABSTRACT
OBJECTIVE: To evaluate the personal protective effects of wearing versus not wearing surgical face masks in public spaces on self-reported respiratory symptoms over a 14 day period. DESIGN: Pragmatic randomised superiority trial. SETTING: Norway. PARTICIPANTS: 4647 adults aged ≥18 years: 2371 were assigned to the intervention arm and 2276 to the control arm. INTERVENTIONS: Participants in the intervention arm were assigned to wear a surgical face mask in public spaces (eg, shopping centres, streets, public transport) over a 14 day period (mask wearing at home or work was not mentioned). Participants in the control arm were assigned to not wear a surgical face mask in public places. MAIN OUTCOME MEASURES: The primary outcome was self-reported respiratory symptoms consistent with a respiratory infection. Secondary outcomes included self-reported and registered covid-19 infection. RESULTS: Between 10 February 2023 and 27 April 2023, 4647 participants were randomised of whom 4575 (2788 women (60.9%); mean age 51.0 (standard deviation 15.0) years) were included in the intention-to-treat analysis: 2313 (50.6%) in the intervention arm and 2262 (49.4%) in the control arm. 163 events (8.9%) of self-reported symptoms consistent with respiratory infection were reported in the intervention arm and 239 (12.2%) in the control arm. The marginal odds ratio was 0.71 (95% confidence interval (CI) 0.58 to 0.87; P=0.001) favouring the face mask intervention. The absolute risk difference was -3.2% (95% CI -5.2% to -1.3%; P<0.001). No statistically significant effect was found on self- reported (marginal odds ratio 1.07, 95% CI 0.58 to 1.98; P=0.82) or registered covid-19 infection (effect estimate and 95% CI not estimable owing to lack of events in the intervention arm). CONCLUSION: Wearing a surgical face mask in public spaces over 14 days reduces the risk of self-reported symptoms consistent with a respiratory infection, compared with not wearing a surgical face mask. TRIAL REGISTRATION: ClinicalTrials.gov NCT05690516.
Subject(s)
COVID-19 , Masks , SARS-CoV-2 , Self Report , Humans , Female , Male , COVID-19/prevention & control , COVID-19/epidemiology , Middle Aged , Norway/epidemiology , Adult , Respiratory Tract Infections/prevention & control , Aged , Pandemics/prevention & control , Coronavirus Infections/prevention & control , Coronavirus Infections/epidemiology , Coronavirus Infections/transmission , Pneumonia, Viral/prevention & control , Pneumonia, Viral/epidemiology , Pneumonia, Viral/transmission , BetacoronavirusABSTRACT
OBJECTIVES: Trials within Cohorts (TwiCs) is a pragmatic design approach that may overcome frequent challenges of traditional randomized trials such as slow recruitment, burdensome consent procedures, or limited external validity. This scoping review aims to identify all randomized controlled trials using the TwiCs design and to summarize their design characteristics, ways to obtain informed consent, output, reported challenges and mitigation strategies. STUDY DESIGN AND SETTING: Systematic search of Medline, Embase, Cochrane, trial registries and citation tracking up to December 2022. TwiCs were defined as randomized trials embedded in a cohort with postrandomization consent for the intervention group and no specific postrandomization consent for the usual care control group. Information from identified TwiCs was extracted in duplicate from protocols, publications, and registry entries. We analyzed the information descriptively and qualitatively to highlight methodological challenges and solutions related to nonuptake of interventions and informed consent procedure. RESULTS: We identified a total of 46 TwiCs conducted between 2005 and 2022 in 14 different countries by a handful of research groups. The most common medical fields were oncology (11/46; 24%), infectious diseases (8/46; 17%), and mental health (7/46; 15%). A typical TwiCs was investigator-initiated (46/46; 100%), publicly funded (36/46; 78%), and recruited outpatients (27/46; 59%). Excluding eight pilot trials, only 16/38 (42%) TwiCs adjusted their calculated sample size for nonuptake of the intervention, anticipating a median nonuptake of 25% (interquartile range 10%-32%) in the experimental arm. Seventeen TwiCs (45%) planned analyses to adjust effect estimates for nonuptake. Regarding informed consent, we observed three patterns: 1) three separate consents for cohort participation, randomization, and intervention (17/46; 37%); 2) combined consent for cohort participation and randomization and a separate intervention consent (10/46; 22%); and 3) consent only for cohort participation and intervention (randomization consent not mentioned; 19/46; 41%). CONCLUSION: Existing TwiCs are globally scattered across a few research groups covering a wide range of medical fields and interventions. Despite the potential advantages, the number of TwiCs remains small. The variability in consent procedures and the possibility of substantial nonuptake of the intervention warrants further research to guide the planning, implementation, and analysis of TwiCs.
Subject(s)
COVID-19 , Masks , Humans , COVID-19/prevention & control , COVID-19/transmission , SARS-CoV-2ABSTRACT
Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members toward one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but nonfinancial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analyzing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated toward COVID-19 elimination (Zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases toward advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.
ABSTRACT
RATIONALE: Novel therapeutic approaches are needed in stroke recovery. Whether pharmacological therapies are beneficial for enhancing stroke recovery is unclear. Dopamine is a neurotransmitter involved in motor learning, reward, and brain plasticity. Its prodrug levodopa is a promising agent for stroke recovery. AIM AND HYPOTHESIS: To investigate the hypothesis that levodopa, in addition to standardized rehabilitation therapy based on active task training, results in an enhancement of functional recovery in acute ischemic or hemorrhagic stroke patients compared to placebo. DESIGN: ESTREL (Enhancement of Stroke REhabilitation with Levodopa) is a randomized (ratio 1:1), multicenter, placebo-controlled, double-blind, parallel-group superiority trial. PARTICIPANTS: 610 participants (according to sample size calculation) with a clinically meaningful hemiparesis will be enrolled ⩽7 days after stroke onset. Key eligibility criteria include (i) in-hospital-rehabilitation required, (ii) capability to participate in rehabilitation, (iii) previous independence in daily living. INTERVENTION: Levodopa 100 mg/carbidopa 25 mg three times daily, administered for 5 weeks in addition to standardized rehabilitation. The study intervention will be initiated within 7 days after stroke onset. COMPARISON: Matching placebo plus standardized rehabilitation. OUTCOMES: The primary outcome is the between-group difference of the Fugl-Meyer-Motor Assessment (FMMA) total score measured 3 months after randomization. Secondary outcomes include patient-reported health and wellbeing (PROMIS 10 and 29), patient-reported assessment of improvement, Rivermead Mobility Index, modified Rankin Scale, National Institutes of Health Stroke Scale (NIHSS), and as measures of harm: mortality, recurrent stroke, and serious adverse events. CONCLUSION: The ESTREL trial will provide evidence of whether the use of Levodopa in addition to standardized rehabilitation in stroke patients leads to better functional recovery compared to rehabilitation alone.
ABSTRACT
OBJECTIVES: To assess to what extent the overall quality of evidence indicates changes to observe intervention effect estimates when new data become available. METHODS: We conducted a meta-epidemiological study. We obtained evidence from meta-analyses of randomized trials of Cochrane reviews addressing the same health-care question that was updated with inclusion of additional data between January 2016 and May 2021. We extracted the reported effect estimates with 95% confidence intervals (CIs) from meta-analyses and corresponding GRADE (Grading of Recommendations Assessment, Development, and Evaluation) assessments of any intervention comparison for the primary outcome in the first and the last updated review version. We considered the reported overall quality (certainty) of evidence (CoE) and specific evidence limitations (no, serious or very serious for risk of bias, imprecision, inconsistency, and/or indirectness). We assessed the change in pooled effect estimates between the original and updated evidence using the ratio of odds ratio (ROR), absolute ratio of odds ratio (aROR), ratio of standard errors (RoSE), direction of effects, and level of statistical significance. RESULTS: High CoE without limitations characterized 19.3% (n = 29) out of 150 included original Cochrane reviews. The update with additional data did not systematically change the effect estimates (mean ROR 1.00; 95% CI 0.99-1.02), which deviated 1.06-fold from the older estimates (median aROR; interquartile range [IQR]: 1.01-1.15), gained precision (median RoSE 0.87; IQR 0.76-1.00), and maintained the same direction with the same level of statistical significance in 93% (27 of 29) of cases. Lower CoE with limitations characterized 121 original reviews and graded as moderate CoE in 30.0% (45 of 150), low CoE in 32.0% (48 of 150), and very low CoE in 18.7% (28 of 150) reviews. Their update had larger absolute deviations (median aROR 1.12 to 1.33) and larger gains in precision (median RoSE 0.78-0.86) without clear and consistent differences between these categories of CoE. Changes in effect direction or statistical significance were also more common in the lower quality evidence, again with a similar extent across categories (without change in 75.6%, 64.6%, and 75.0% for moderate, low, very low CoE). As limitations increased, effect estimates deviated more (aROR 1.05 with zero, 1.11 with one, 1.25 with two, 1.24 with three limitations) and changes in direction or significance became more frequent (93.2% stable with no limitations, 74.5% with one, 68.2% with two, and 61.5% with three limitations). CONCLUSION: High-quality evidence without methodological deficiencies is trustworthy and stable, providing reliable intervention effect estimates when updated with new data. Evidence of moderate and lower quality may be equally prone to being unstable and cannot indicate if available effect estimates are true, exaggerated, or underestimated.
Subject(s)
Randomized Controlled Trials as Topic , Humans , Randomized Controlled Trials as Topic/standards , Evidence-Based Medicine/standards , Evidence-Based Medicine/methods , Meta-Analysis as Topic , Systematic Reviews as Topic/methodsABSTRACT
BACKGROUND: Technological devices such as smartphones, wearables and virtual assistants enable health data collection, serving as digital alternatives to conventional biomarkers. We aimed to provide a systematic overview of emerging literature on 'digital biomarkers,' covering definitions, features and citations in biomedical research. METHODS: We analysed all articles in PubMed that used 'digital biomarker(s)' in title or abstract, considering any study involving humans and any review, editorial, perspective or opinion-based articles up to 8 March 2023. We systematically extracted characteristics of publications and research studies, and any definitions and features of 'digital biomarkers' mentioned. We described the most influential literature on digital biomarkers and their definitions using thematic categorisations of definitions considering the Food and Drug Administration Biomarkers, EndpointS and other Tools framework (ie, data type, data collection method, purpose of biomarker), analysing structural similarity of definitions by performing text and citation analyses. RESULTS: We identified 415 articles using 'digital biomarker' between 2014 and 2023 (median 2021). The majority (283 articles; 68%) were primary research. Notably, 287 articles (69%) did not provide a definition of digital biomarkers. Among the 128 articles with definitions, there were 127 different ones. Of these, 78 considered data collection, 56 data type, 50 purpose and 23 included all three components. Those 128 articles with a definition had a median of 6 citations, with the top 10 each presenting distinct definitions. CONCLUSIONS: The definitions of digital biomarkers vary significantly, indicating a lack of consensus in this emerging field. Our overview highlights key defining characteristics, which could guide the development of a more harmonised accepted definition.
Subject(s)
Biomarkers , Humans , Biomarkers/analysis , Biomedical ResearchABSTRACT
BACKGROUND: Increasingly, patients, clinicians, and regulators call for more evidence on the impact of innovative medicines on quality of life (QoL). We assessed the effects of disease-modifying therapies (DMTs) on QoL in people with multiple sclerosis (PwMS). METHODS: Randomized trials assessing approved DMTs in PwMS with results for at least one outcome referred to as "quality of life" were searched in PubMed and ClinicalTrials.gov. RESULTS: We identified 38 trials published between 1999 and 2023 with a median of 531 participants (interquartile range (IQR) 202 to 941; total 23,225). The evaluated DMTs were mostly interferon-beta (n = 10; 26%), fingolimod (n = 7; 18%), natalizumab (n = 5; 13%), and glatiramer acetate (n = 4; 11%). The 38 trials used 18 different QoL instruments, with up to 11 QoL subscale measures per trial (median 2; IQR 1-3). QoL was never the single primary outcome. We identified quantitative QoL results in 24 trials (63%), and narrative statements in 15 trials (39%). In 16 trials (42%), at least one of the multiple QoL results was statistically significant. The effect sizes of the significant quantitative QoL results were large (median Cohen's d 1.02; IQR 0.3-1.7; median Hedges' g 1.01; IQR 0.3-1.69) and ranged between d 0.14 and 2.91. CONCLUSIONS: Certain DMTs have the potential to positively impact QoL of PwMS, and the assessment and reporting of QoL is suboptimal with a multitude of diverse instruments being used. There is an urgent need that design and reporting of clinical trials reflect the critical importance of QoL for PwMS.
Subject(s)
Multiple Sclerosis , Quality of Life , Humans , Multiple Sclerosis/drug therapy , Multiple Sclerosis/psychology , Randomized Controlled Trials as Topic , Outcome Assessment, Health Care , Immunologic Factors/therapeutic useABSTRACT
Background: Smoking cessation is challenging, despite making use of established smoking cessation therapies. Preclinical studies and one clinical pilot study suggest the antidiabetic drug glucagon-like peptide-1 (GLP-1) analogue to modulate addictive behaviours and nicotine craving. Previously, we reported the short-term results of a randomised, double-blind, placebo-controlled trial. Herein we report long-term abstinence rates and weight developments after 24 and 52 weeks. Methods: This single-centre, randomised, double-blind, placebo-controlled, parallel group trial was done at the University Hospital Basel in Switzerland. We randomly assigned (1:1) individuals with at least a moderate nicotine dependence willing to quit smoking to either a 12-week treatment with dulaglutide 1.5 mg or placebo subcutaneously once weekly in addition to standard of care smoking cessation therapy (varenicline 2 mg/day and behavioural counselling). After 12 weeks, dulaglutide or placebo injections were discontinued and the participants were followed up at week 24 and 52. The primary outcome of self-reported and biochemically confirmed point prevalence abstinence rate, and secondary outcome of secondary outcome of weight change were assessed at weeks 24 and 52. All participants who received one dose of the study drug were included in the intention to treat set and participants who received at least 10/12 doses of the study drug formed the per protocol set. The trial was registered at ClinicalTrials.gov, NCT03204396. Findings: Of the 255 participants who were randomly assigned between June 22, 2017 and December 3, 2020, 63% (80/127) (dulaglutide group) and 65% (83/128) (placebo group) were abstinent after 12 weeks. These abstinence rates declined to 43% (54/127) and 41% (52/128), respectively, after 24 weeks and to 32% (41/127) and 32% (41/128), respectively, after 52 weeks. Post-cessation weight gain was prevented in the dulaglutide group (-1.0 kg, standard deviation [SD] 2.7) as opposed to the placebo group (+1.9 kg, SD 2.4) after 12 weeks. However, at week 24, increases in weight from baseline were observed in both groups (median, interquartile range [IQR]: dulaglutide: +1.5 kg, [-0.4, 4.1], placebo: +3.0 kg, [0.6, 4.6], baseline-adjusted difference in weight change -1.0 kg (97.5% CI [-2.16, 0.16])), and at week 52 the groups showed similar weight gain (median, IQR: dulaglutide: +2.8 kg [-0.4, 4.7], placebo: +3.1 kg [-0.4, 6.0], baseline-adjusted difference in weight change: -0.35 kg (95% CI [-1.72, 1.01])). In the follow-up period (week 12 to week 52) 51 (51%) and 48 (48%) treatment-unrelated adverse events were recorded in the dulaglutide and the placebo group, respectively. No treatment-related serious adverse events or deaths occurred. Interpretation: Dulaglutide does not improve long-term smoking abstinence, but has potential to counteract weight gain after quitting. However, 3 months of treatment did not have a sustained beneficial effect on weight at 1 year. As post-cessation weight gain is highest in the first year after quitting smoking, future studies should consider a longer treatment duration with a GLP-1 analogue in abstinent individuals. Funding: Swiss National Science Foundation, the Gottfried and Julia Bangerter-Rhyner Foundation, the Goldschmidt-Jacobson Foundation, the Hemmi-Foundation, the University of Basel, the Swiss Academy of Medical Sciences.
ABSTRACT
BACKGROUND: Pragmatic trials are increasingly recognized for providing real-world evidence on treatment choices. OBJECTIVE: The objective of this study is to investigate the use and characteristics of pragmatic trials in multiple sclerosis (MS). METHODS: Systematic literature search and analysis of pragmatic trials on any intervention published up to 2022. The assessment of pragmatism with PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary-2) is performed. RESULTS: We identified 48 pragmatic trials published 1967-2022 that included a median of 82 participants (interquartile range (IQR) = 42-160) to assess typically supportive care interventions (n = 41; 85%). Only seven trials assessed drugs (15%). Only three trials (6%) included >500 participants. Trials were mostly from the United Kingdom (n = 18; 38%), Italy (n = 6; 13%), the United States and Denmark (each n = 5; 10%). Primary outcomes were diverse, for example, quality-of-life, physical functioning, or disease activity. Only 1 trial (2%) used routinely collected data for outcome ascertainment. No trial was very pragmatic in all design aspects, but 14 trials (29%) were widely pragmatic (i.e. PRECIS-2 score ⩾ 4/5 in all domains). CONCLUSION: Only few and mostly small pragmatic trials exist in MS which rarely assess drugs. Despite the widely available routine data infrastructures, very few trials utilize them. There is an urgent need to leverage the potential of this pioneering study design to provide useful randomized real-world evidence.
Subject(s)
Multiple Sclerosis , Pragmatic Clinical Trials as Topic , Humans , Multiple Sclerosis/drug therapy , Multiple Sclerosis/therapy , Randomized Controlled Trials as TopicABSTRACT
OBJECTIVES: Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods. STUDY DESIGN AND SETTING: We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions. RESULTS: Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring. CONCLUSION: Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.
Subject(s)
Research Design , Humans , Cross-Sectional StudiesABSTRACT
BACKGROUND: Clinical trial registries allow assessment of deviations of published trials from their protocol, which may indicate a considerable risk of bias. However, since entries in many registries can be updated at any time, deviations may go unnoticed. We aimed to assess the frequency of changes to primary outcomes in different historical versions of registry entries, and how often they would go unnoticed if only deviations between published trial reports and the most recent registry entry are assessed. METHODS AND FINDINGS: We analyzed the complete history of changes of registry entries in all 1746 randomized controlled trials completed at German university medical centers between 2009 and 2017, with published results up to 2022, that were registered in ClinicalTrials.gov or the German WHO primary registry (German Clinical Trials Register; DRKS). Data were retrieved on 24 January 2022. We assessed deviations between registry entries and publications in a random subsample of 292 trials. We determined changes of primary outcomes (1) between different versions of registry entries at key trial milestones, (2) between the latest registry entry version and the results publication, and (3) changes that occurred after trial start with no change between latest registry entry version and publication (so that assessing the full history of changes is required for detection of changes). We categorized changes as major if primary outcomes were added, dropped, changed to secondary outcomes, or secondary outcomes were turned into primary outcomes. We also assessed (4) the proportion of publications transparently reporting changes and (5) characteristics associated with changes. Of all 1746 trials, 23% (n = 393) had a primary outcome change between trial start and latest registry entry version, with 8% (n = 142) being major changes, that is, primary outcomes were added, dropped, changed to secondary outcomes, or secondary outcomes were turned into primary outcomes. Primary outcomes in publications were different from the latest registry entry version in 41% of trials (120 of the 292 sampled trials; 95% confidence interval (CI) [35%, 47%]), with major changes in 18% (54 of 292; 95% CI [14%, 23%]). Overall, 55% of trials (161 of 292; 95% CI [49%, 61%]) had primary outcome changes at any timepoint over the course of a trial, with 23% of trials (67 of 292; 95% CI [18%, 28%]) having major changes. Changes only within registry records, with no apparent discrepancy between latest registry entry version and publication, were observed in 14% of trials (41 of 292; 95% CI [10%, 19%]), with 4% (13 of 292; 95% CI [2%, 7%]) being major changes. One percent of trials with a change reported this in their publication (2 of 161 trials; 95% CI [0%, 4%]). An exploratory logistic regression analysis indicated that trials were less likely to have a discrepant registry entry if they were registered more recently (odds ratio (OR) 0.74; 95% CI [0.69, 0.80]; p<0.001), were not registered on ClinicalTrials.gov (OR 0.41; 95% CI [0.23, 0.70]; p = 0.002), or were not industry-sponsored (OR 0.29; 95% CI [0.21, 0.41]; p<0.001). Key limitations include some degree of subjectivity in the categorization of outcome changes and inclusion of a single geographic region. CONCLUSIONS: In this study, we observed that changes to primary outcomes occur in 55% of trials, with 23% trials having major changes. They are rarely transparently reported in the results publication and often not visible in the latest registry entry version. More transparency is needed, supported by deeper analysis of registry entries to make these changes more easily recognizable. Protocol registration: Open Science Framework (https://osf.io/t3qva; amendment in https://osf.io/qtd2b).
Subject(s)
Universities , Humans , Bias , Registries , Odds RatioABSTRACT
BACKGROUND: Pragmatic trials provide decision-oriented, real-world evidence that is highly applicable and generalizable. The interest in real-world evidence is fueled by the assumption that effects in the "real-world" are different to effects obtained under artificial, controlled, research conditions as often used for traditional explanatory trials. However, it is unknown which features of pragmatism, generalizability, and applicability would be responsible for such differences. There is a need to provide empirical evidence and promote meta-research to answer these fundamental questions on the pragmatism of randomized trials and real-world evidence. Here, we describe the rationale and design of the PragMeta database which pursues this goal ( www.PragMeta.org ). METHODS: PragMeta is a non-commercial, open data platform and infrastructure to facilitate research on pragmatic trials. It collects and shares data from published randomized trials that either have a specific design feature or other characteristic related to pragmatism or they form clusters of trials addressing the same research question but having different aspects of pragmatism. This lays the foundation to determine the relationship of various features of pragmatism, generalizability, and applicability with intervention effects or other trial characteristics. The database contains trial data actively collected for PragMeta but also allows to import and link existing datasets of trials collected for other purposes, forming a large-scale meta-database. PragMeta captures data on (1) trial and design characteristics (e.g., sample size, population, intervention/comparison, outcome, longitudinal structure, blinding), (2) effects estimates, and (3) various determinants of pragmatism (e.g., the use of routinely collected data) and ratings from established tools used to determine pragmatism (e.g., the PRagmatic-Explanatory Continuum Indicator Summary 2; PRECIS-2). PragMeta is continuously provided online, inviting the meta-research community to collaborate, contribute, and/or use the database. As of April 2023, PragMeta contains data from > 700 trials, mostly with assessments on pragmatism. CONCLUSIONS: PragMeta will inform a better understanding of pragmatism and the generation and interpretation of real-world evidence.
Subject(s)
Routinely Collected Health Data , Humans , Randomized Controlled Trials as Topic , Databases, Factual , Sample SizeABSTRACT
OBJECTIVES: Previous findings indicate limited reporting of systematic reviews with meta-analyses of time-to-event (TTE) outcomes. We assessed corresponding available information in trial publications included in such meta-analyses. STUDY DESIGN AND SETTING: We extracted data from all randomized trials in pairwise, hazard ratio (HR)-based meta-analyses of primary outcomes and overall survival of 50 systematic reviews systematically identified from the Cochrane Database and Core Clinical Journals. Data on methods and characteristics relevant for TTE analysis of reviews, trials, and outcomes were extracted. RESULTS: Meta-analyses included 235 trials with 315 trial analyses. Most prominently assessed was overall survival (91%). Definitions (61%), censoring reasons (41%), and follow-up specifications (56%) for trial outcomes were often missing. Available TTE data per trial were most frequently survival curves (83%), log-rank P values (76%), and HRs (72%). When trial TTE data recalculation was reported, reviews mostly specified HRs or P values (each 5%). Reviews primarily included intention-to-treat analyses (64%) and analyses not adjusted for covariates (25%). Except for missing outcome data, TTE-relevant trial characteristics, for example, informative censoring, treatment switching, and proportional hazards, were sporadically addressed in trial publications. Reporting limitations in trial publications translate to the review level. CONCLUSION: TTE (meta)-analyses, in trial and review publications, need clear reporting standards.
Subject(s)
Systematic Reviews as Topic , Humans , Data CollectionABSTRACT
OBJECTIVES: This Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept article offers systematic reviewers, guideline authors, and other users of evidence assistance in addressing randomized trial situations in which interventions or comparators differ from those in the target people, interventions, comparators, and outcomes. To clarify what GRADE considers under indirectness of interventions and comparators, we focus on a particular example: when comparator arm participants receive some or all aspects of the intervention management strategy (treatment switching). STUDY DESIGN AND SETTING: An interdisciplinary panel of the GRADE working group members developed this concept article through an iterative review of examples in multiple teleconferences, small group sessions, and e-mail correspondence. After presentation at a GRADE working group meeting in November 2022, attendees approved the final concept paper, which we support with examples from systematic reviews and individual trials. RESULTS: In the presence of safeguards against risk of bias, trials provide unbiased estimates of the effect of an intervention on the people as enrolled, the interventions as implemented, the comparators as implemented, and the outcomes as measured. Within the GRADE framework, differences in the people, interventions, comparators, and outcomes elements between the review or guideline recommendation targets and the trials as implemented constitute issues of indirectness. The intervention or comparator group management strategy as implemented, when it differs from the target comparator, constitutes one potential source of indirectness: Indirectness of interventions and comparators-comparator group receipt of the intervention constitutes a specific subcategory of said indirectness. The proportion of comparator arm participants that received the intervention and the apparent magnitude of effect bear on whether one should rate down, and if one does, to what extent. CONCLUSION: Treatment switching and other differences between review or guideline recommendation target interventions and comparators vs. interventions and comparators as implemented in otherwise relevant trials are best considered issues of indirectness.
Subject(s)
Bias , Evidence-Based Medicine , Randomized Controlled Trials as Topic , HumansABSTRACT
BACKGROUND: Cigarette smoking is the leading preventable cause of premature death. Despite dedicated programmes, quit rates remain low due to barriers such as nicotine withdrawal syndrome or post-cessation weight gain. Glucagon-like peptide-1 (GLP-1) analogues reduce energy intake and body weight and seem to modulate addictive behaviour. These GLP-1 properties are of major interest in the context of smoking cessation. The aim of this study is to evaluate the GLP-1 analogue dulaglutide as a new therapy for smoking cessation. METHODS: This is a placebo-controlled, double-blind, parallel group, superiority, single-centre randomized study including 255 patients. The intervention consists of a 12-week dulaglutide treatment phase with 1.5 mg once weekly or placebo subcutaneously, in addition to standard of care (behavioural counselling and pharmacotherapy with varenicline). A 40-week non-treatment phase follows. The primary outcome is the point prevalence abstinence rate at week 12. Smoking status is self-reported and biochemically confirmed by end-expiratory exhaled carbon monoxide measurement. Further endpoints include post-cessational weight gain, nicotine craving analysis, glucose homeostasis and long-term nicotine abstinence. Two separate substudies assess behavioural, functional and structural changes by functional magnetic resonance imaging and measures of energy metabolism (i.e. resting energy expenditure, body composition). DISCUSSION: Combining behavioural counselling and medical therapy, e.g. with varenicline, improves abstinence rates and is considered the standard of care. We expect a further increase in quit rates by adding a second component of medical therapy and assume a dual effect of dulaglutide treatment (blunting nicotine withdrawal symptoms and reducing post-cessational weight gain). This project is of high relevance as it explores novel treatment options aimed at preventing the disastrous consequences of nicotine consumption and obesity. TRIAL REGISTRATION: ClinicalTrials.gov NCT03204396 . Registered on June 26, 2017.
Subject(s)
Smoking Cessation , Substance Withdrawal Syndrome , Humans , Varenicline/therapeutic use , Nicotine , Smoking Cessation/methods , Glucagon-Like Peptide 1 , Substance Withdrawal Syndrome/drug therapy , Double-Blind Method , Weight Gain , Randomized Controlled Trials as TopicABSTRACT
Background: Quitting smoking is difficult due to barriers such as craving for cigarettes and post-cessation weight gain. Recent experimental data suggest a role of glucagon-like peptide-1 (GLP-1) in the pathophysiology of addiction in addition to appetite regulation and weight control. We hypothesized that a pharmacological intervention with the GLP-1 analogue dulaglutide during smoking cessation may improve abstinence rates and reduce post-cessation weight gain. Methods: This is a single-centre, randomized, double-blind, placebo-controlled, parallel group, superiority study conducted in the University Hospital Basel in Switzerland. We included adult smokers with at least moderate cigarette dependence who wanted to quit. Participants were randomly assigned to a 12-week treatment with dulaglutide 1.5 mg once weekly or placebo subcutaneously in addition to standard of care including behavioural counselling and oral varenicline pharmacotherapy of 2 mg/day. The primary outcome was self-reported and biochemically confirmed point prevalence abstinence rate at week 12. Secondary outcomes included post-cessation weight, glucose metabolism, and craving for smoking. All participants who received one dose of study drug were included in the primary and safety analyses. The trial was registered on ClinicalTrials.gov (NCT03204396). Findings: Between June 22, 2017, and December 3, 2020, 255 participants were enrolled and randomly assigned to each group (127 in the dulaglutide group and 128 in the placebo group). After 12 weeks, 63% (80/127) participants on dulaglutide and 65% (83/128) on placebo treatment were abstinent (difference in proportions -1.9% [95% Confidence interval (CI) -10.7, 14.4], p-value (p) = 0.859). Dulaglutide decreased post-cessation weight (-1 kg [standard deviation (SD) 2.7]), while weight increased on placebo (+1.9 kg [SD 2.4]). The baseline-adjusted difference in weight change between groups was -2.9 kg (95% CI -3.59, -2.3, p < 0.001). Haemoglobin A1c (HbA1c) level declined on dulaglutide treatment (baseline-adjusted median difference in HbA1c between groups -0.25% [interquartile range (IQR) -0.36, -0.14], p < 0.001). Craving for smoking declined during treatment without any difference between the groups. Treatment-emergent gastrointestinal symptoms were very common in both groups: 90% (114/127) of participants on dulaglutide and 81% (81/128) on placebo). Interpretation: Dulaglutide had no effect on abstinence rates but prevented post-cessation weight gain and decreased HbA1c levels. GLP-1 analogues may play a role in future cessation therapy targeting metabolic parameters such as weight and glucose metabolism. Funding: Swiss National Science Foundation, the Gottfried Julia Bangerter-Rhyner Foundation, the Goldschmidt-Jacobson Foundation, the Hemmi-Foundation, the University of Basel, the Swiss Academy of Medical Sciences.
ABSTRACT
Concerns have been raised that regulatory programs to accelerate approval of cancer drugs in cancer may increase uncertainty about benefits and harms for survival and quality of life (QoL). We analyzed all pivotal clinical trials and all non-pivotal randomized controlled trials (RCTs) for all cancer drugs approved for the first time by the FDA between 2000 and 2020. We report regulatory and trial characteristics. Effects on overall survival (OS), progression-free survival and tumor response were summarized in meta-analyses. Effects on QoL were qualitatively summarized. Between 2000 and 2020, the FDA approved 145 novel cancer drugs for 156 indications based on 190 clinical trials. Half of indications (49%) were approved without RCT evidence; 82% had a single clinical trial only. OS was primary endpoint in 14% of trials and QoL data were available from 25%. The median OS benefit was 2.55 months (IQR, 1.33-4.28) with a mean hazard ratio for OS of 0.75 (95%CI, 0.72-0.79, I2 = 42). Improvement for QoL was reported for 7 (4%) of 156 indications. Over time, priority review was used increasingly and the mean number of trials per indication decreased from 1.45 to 1.12. More trials reported results on QoL (19% in 2000-2005; 41% in 2016-2020). For 21 years, novel cancer drugs have typically been approved based on one single, often uncontrolled, clinical trial, measuring surrogate endpoints. This leaves cancer patients without solid evidence that novel drugs improve their survival or QoL and there is no indication towards improvement.