Selective outcome reporting in trials of behavioural health interventions in health psychology and behavioural medicine journals: a review.

Selective outcome reporting can result in overestimation of treatment effects, research waste, and reduced openness and transparency. This review aimed to examine selective outcome reporting in trials of behavioural health interventions and determine potential outcome reporting bias. A review of nine health psychology and behavioural medicine journals was conducted to identify randomised controlled trials of behavioural health interventions published since 2019. Discrepancies in outcome reporting were observed in 90% of the 29 trials with corresponding registrations/protocols. Discrepancies included 72% of trials omitting prespecified outcomes; 55% of trials introduced new outcomes. Thirty-eight percent of trials omitted prespecified and introduced new outcomes. Three trials (10%) downgraded primary outcomes in registrations/protocols to secondary outcomes in final reports; downgraded outcomes were not statistically significant in two trials. Five trials (17%) upgraded secondary outcomes to primary outcomes; upgraded outcomes were statistically significant in all trials. In final reports, three trials (7%) omitted outcomes from the methods section; three trials (7%) introduced new outcomes in results that were not in the methods. These findings indicate that selective outcome reporting is a problem in behavioural health intervention trials. Journal- and trialist-level approaches are needed to minimise selective outcome reporting in health psychology and behavioural medicine.

Outcomes of interventions in neonatal sepsis: A systematic review of qualitative research.

BACKGROUND: While a systematic review exists detailing neonatal sepsis outcomes from clinical trials, there remains an absence of a qualitative systematic review capturing the perspectives of key stakeholders. OBJECTIVES: Our aim is to identify outcomes from qualitative research on any intervention to prevent or improve the outcomes of neonatal sepsis that are important to parents, other family members, healthcare providers, policymakers, and researchers as a part of the development of a core outcome set (COS) for neonatal sepsis. SEARCH STRATEGY: A literature search was carried out using MEDLINE, EMBASE, CINAHL, and PsycInfo databases. SELECTION CRITERIA: Publications describing qualitative data relating to neonatal sepsis outcomes were included. DATA COLLECTION AND ANALYSIS: Drawing on the concepts of thematic synthesis, texts related to outcomes were coded and grouped. These outcomes were then mapped to the domain headings of an existing model. MAIN RESULTS: Out of 6777 records screened, six studies were included. Overall, 19 outcomes were extracted from the included studies. The most frequently reported outcomes were those in the domains related to parents, healthcare workers and individual organ systemas such as gastrointestinal system. The remaining outcomes were classified under the headings of general outcomes, miscellaneous outcomes, survival, and infection. CONCLUSIONS: The outcomes identified in this review are different from those reported in neonatal sepsis clinical trials, thus highlighting the importance of incorporating qualitative studies into COS development to encapsulate all relevant stakeholders' perspectives.

Exploring the effect on primary endpoints in trials testing targeted therapy interventions for rheumatoid arthritis: a meta-epidemiological study on the appropriate use of a core outcome set.

OBJECTIVES: To explore which core domain is best associated with the American College of Rheumatology (ACR) 20% response in trials assessing the effect of targeted interventions in rheumatoid arthritis (RA). METHODS: A meta-epidemiological study was performed on randomised trials investigating biologics and targeted agents compared with placebo or conventional disease-modifying antirheumatic drugs in patients with RA. The main outcome measures were ORs for the ACR 20% response and at least one of the eight core domains according to the existing RA core outcome set (COS) analysed based on standardised mean differences. RESULTS: 115 trials involving 55 422 patients with RA were eligible. The OR for achieving ACR 20% response was 3.19 (95% CI 2.96 to 3.44) for the experimental interventions relative to the comparators. The median number of COS domains reported was 6; 18 trials reported only 1 domain, 17 all 8. Univariable meta-regression analyses indicated that each of the eight core domains was significantly associated with ACR 20% response, yet improvements in physical disability explain a successful ACR 20% response the most. Including only trials reporting on all eight core domains, univariable meta-regression analyses proved improvement in fatigue to explain a successful ACR 20% response the most. CONCLUSIONS: Within this dataset, it is evident that the conclusions concerning our primary objective were significantly influenced by both the amount and characteristics of missing data. Our data suggest that fatigue could be more important for the primary endpoint than previously assumed, but this is based on limited data.

A survey of experts to identify methods to detect problematic studies: Stage 1 of the INSPECT-SR Project.

Background: Randomised controlled trials (RCTs) inform healthcare decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INSPECT-SR project is developing a tool to identify problematic RCTs in systematic reviews of healthcare-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion. Methods: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorised these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list. Results: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool. Conclusions: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

Study found increasing use of core outcome sets in Cochrane systematic reviews and identified facilitators and barriers.

OBJECTIVES: In 2019, only 7% of Cochrane systematic reviews (SRs) cited a core outcome set (COS) in relation to choosing outcomes, even though a relevant COS existed but was not mentioned (or cited) for a further 29% of SRs. Our objectives for the current work were to (1) examine the extent to which authors are currently considering COS to inform outcome choice in Cochrane protocols and completed SRs, and (2) understand author facilitators and barriers to using COS. STUDY DESIGN AND SETTING: We examined all completed Cochrane SRs published in the last 3 months of 2022 and all Cochrane protocols published in 2022 for the extent to which they: (a) cited a COS, (b) searched for COS, (c) used outcomes from existing COS, and (d) reported outcome inconsistency among included studies and/or noted the need for COS. One investigator extracted information; a second extractor verified all information, discussing discrepancies to achieve consensus. We then conducted an online survey of authors of the included SRs to assess awareness of COS and identify facilitators and barriers to using COS to inform outcome choice. RESULTS: Objective 1: We included 294 SRs of interventions (84 completed SRs and 210 published SR protocols), of which 13% cited specific COS and 5% did not cite but mentioned searching for COS. A median of 83% of core outcomes from cited COS (interquartile range [IQR] 57%-100%) were included in the corresponding SR. We identified a relevant COS for 39% of SRs that did not cite a COS. A median of 50% of core outcomes from noncited COS (IQR 35%-72%) were included in the corresponding SR. Objective 2: Authors of 236 (80%) of the 294 eligible SRs completed our survey. Seventy-seven percent of authors noted being aware of COS before the survey. Fifty-five percent of authors who did not cite COS but were aware of them reported searching for a COS. The most reported facilitators of using COS were author awareness of the existence of COS (59%), author positive perceptions of COS (52%), and recommendation in the Cochrane Handbook regarding COS use (48%). The most reported barriers related to matching of the scope of the COS and the SR: the COS target population was too narrow/broad relative to the SR population (29%) or the COS target intervention was too narrow/broad relative to the SR intervention (21%). Most authors (87%) mentioned that they would consider incorporating missing core outcomes in the SR/update. CONCLUSION: Since 2019, there is increasing consideration and awareness of COS when choosing outcomes for Cochrane SRs of interventions, but uptake remains low and can be improved further. Use of COS in SRs is important to improve outcome standardization, reduce research waste, and improve evidence syntheses of the relevant effects of interventions across health research.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

INTRODUCTION: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare-related interventions. METHODS AND ANALYSIS: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: (1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, (2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, (3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in, (4) a consensus meeting to select checks to be included in a draft tool and to determine its format and (5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies and will help patients by protecting them from the influence of false data on their healthcare. ETHICS AND DISSEMINATION: The University of Manchester ethics decision tool was used, and this returned the result that ethical approval was not required for this project (30 September 2022), which incorporates secondary research and surveys of professionals about subjects relating to their expertise. Informed consent will be obtained from all survey participants. All results will be published as open-access articles. The final tool will be made freely available.

Representation of published core outcome sets in practice guidelines.

OBJECTIVES: A core outcome set (COS) is an agreed standardized set of outcomes that should be measured and reported, as a minimum, in specific areas of health or health care. A COS is developed through a consensus process to ensure health care outcomes to be measured are relevant to decision-makers, including patients and health-care professionals. Use of COS in guideline development is likely to increase the relevance of the guideline to those decision-makers. Previous work has looked at the uptake of COS in trials, systematic reviews, health technology assessments and regulatory guidance but to date there has been no evaluation of the use of COS in practice guideline development. The objective of this study was to investigate the representation of core outcomes in a set of international practice guidelines. STUDY DESIGN AND SETTING: We searched for clinical guidelines relevant to ten high-quality COS (with focus on the United Kingdom, Germany, China, India, Canada, Denmark, United States and World Health Organisation). We matched scope between COS and guideline in terms of condition, population and outcome. We calculated the proportion of guidelines mentioning or referencing COS and the proportion of COS domains specifically, or generally, matching to outcomes specified in each guideline populations, interventions, comparators and outcome (PICO) statement. RESULTS: We found 38 guidelines that contained 170 PICO statements matching the scope of the ten COS and of sufficient quality to allow data extraction. None of the guidelines reviewed explicitly mentioned or referenced the relevant COS. The median (range) of the proportion of core outcomes covered either specifically or generally by the guideline PICO was 30% (0%-100%). CONCLUSION: There is no evidence that COS are being used routinely to inform the guideline development process, and concordance between outcomes in published guidelines and those in COS is limited. Further work is warranted to explore barriers and facilitators in the use of COS when developing clinical guidelines.

Towards a core outcome set for dysarthria after stroke: What should we measure?

OBJECTIVE: To identify and agree on what outcome domains should be measured in research and clinical practice when working with stroke survivors who have dysarthria. DESIGN: Delphi process, two rounds of an online survey followed by two online consensus meetings. SETTING: UK and Australia. PARTICIPANTS: Stroke survivors with experience of dysarthria, speech and language therapists/pathologists working in stroke and communication researchers. METHODS: Initial list of outcome domains generated from existing literature and with our patient and public involvement group to develop the survey. Participants completed two rounds of this survey to rate importance. Outcomes were identified as 'in', 'unclear' or 'out' from the second survey. All participants were invited to two consensus meetings to discuss these results followed by voting to identify critically important outcome domains for a future Core Outcome Set. All outcomes were voted on in the consensus meetings and included if 70% of meeting participants voted 'yes' for critically important. RESULTS: In total, 148 surveys were fully completed, and 28 participants attended the consensus meetings. A core outcome set for dysarthria after stroke should include four outcome domains: (a) intelligibility of speech, (b) ability to participate in conversations, (c) living well with dysarthria, (d) skills and knowledge of communication partners (where relevant). CONCLUSIONS: We describe the consensus of 'what' speech outcomes after stroke are valued by all stakeholders including those with lived experience. We share these findings to encourage the measurement of these domains in clinical practice and research and for future research to identify 'how' best to measure these outcomes.

Protocol of a scoping review of outcome domains in dermatology.

INTRODUCTION: Core outcome sets (COSs) are agreed outcomes (domains (subdomains) and instruments) that should be measured as a minimum in clinical trials or practice in certain diseases or clinical fields. Worldwide, the number of COSs is increasing and there might be conceptual overlaps of domains (subdomains) and instruments within disciplines. The aim of this scoping review is to map and to classify all outcomes identified with COS projects relating to skin diseases. METHODS AND ANALYSIS: We will conduct a scoping review of outcomes of skin disease-related COS initiatives to identify all concepts and their definitions. We will search PubMed, Embase and Cochrane library. The search dates will be 1 January 2010 (the point at which Core Outcome Measures in Effectiveness Trials (COMET) was established) to 1 January 2024. We will also review the COMET database and C3 website to identify parts of COSs (domains and/or instruments) that are being developed and published. This review will be supplemented by querying relevant stakeholders from COS organisations, dermatology organisations and patient organisations for additional COSs that were developed. The resulting long lists of outcomes will then be mapped into conceptually similar concepts. ETHICS AND DISSEMINATION: This study was supported by departmental research funds from the Department of Dermatology at Northwestern University. An ethics committee review was waived since this protocol was done by staff researchers with no involvement of patient care. Conflicts of interests, if any, will be addressed by replacing participants with relevant conflicts or reassigning them. The results will be disseminated through publication in peer-reviewed journals, social media posts and promotion by COS organisations.

Core outcome sets for trials of interventions to prevent and to treat multimorbidity in low- and middle-income countries: the COSMOS study.

Introduction: The burden of multimorbidity is recognised increasingly in low- and middle-income countries (LMICs), creating a strong emphasis on the need for effective evidence-based interventions. A core outcome set (COS) appropriate for the study of multimorbidity in LMIC contexts does not presently exist. This is required to standardise reporting and contribute to a consistent and cohesive evidence-base to inform policy and practice. We describe the development of two COS for intervention trials aimed at the prevention and treatment of multimorbidity in LMICs. Methods: To generate a comprehensive list of relevant prevention and treatment outcomes, we conducted a systematic review and qualitative interviews with people with multimorbidity and their caregivers living in LMICs. We then used a modified two-round Delphi process to identify outcomes most important to four stakeholder groups with representation from 33 countries (people with multimorbidity/caregivers, multimorbidity researchers, healthcare professionals, and policy makers). Consensus meetings were used to reach agreement on the two final COS. Registration: https://www.comet-initiative.org/Studies/Details/1580. Results: The systematic review and qualitative interviews identified 24 outcomes for prevention and 49 for treatment of multimorbidity. An additional 12 prevention, and six treatment outcomes were added from Delphi round one. Delphi round two surveys were completed by 95 of 132 round one participants (72.0%) for prevention and 95 of 133 (71.4%) participants for treatment outcomes. Consensus meetings agreed four outcomes for the prevention COS: (1) Adverse events, (2) Development of new comorbidity, (3) Health risk behaviour, and (4) Quality of life; and four for the treatment COS: (1) Adherence to treatment, (2) Adverse events, (3) Out-of-pocket expenditure, and (4) Quality of life. Conclusion: Following established guidelines, we developed two COS for trials of interventions for multimorbidity prevention and treatment, specific to LMIC contexts. We recommend their inclusion in future trials to meaningfully advance the field of multimorbidity research in LMICs.

Using behavioral science to increase core outcome set use in trials.

OBJECTIVES: Core outcome sets (COS) are agreed sets of outcomes for use in clinical trials, which can increase standardization and reduce heterogeneity of outcomes in research. Using a COS, or not, is a behavior that can potentially be increased using behavioral strategies. The aim of this study was to identify behavioral intervention components to potentially increase use of COS in trials. METHODS: This project was informed by the Behavior Change Wheel framework. Two reviewers extracted barriers and facilitators to COS use from four recently published studies examining COS use in trials. Barriers and facilitators were coded to the Capability, Opportunity, Motivation-Behavior (COM-B) model, which forms part of the Behavior Change Wheel. COM-B findings were mapped to intervention functions by two reviewers, and then mapped to behavior change techniques (BCTs). Full-team Affordability, Practicability, Effectiveness/Cost-effectiveness, Acceptability, Side effects/Safety, Equity ratings were used to reach consensus on intervention functions and BCTs. BCTs were operationalized using examples of tangible potential applications and were categorized based on similarity. RESULTS: Barriers and facilitators were identified for all capability, opportunity and motivation aspects of the COM-B model. Five intervention functions (education, training, enablement, persuasion, and modeling) and 15 BCTs were identified. Thirty-six BCT examples were developed, including providing information on benefits of COS for health research, and information choosing COS. BCT examples are categorized by approaches related to "workshops," "guidance," "audio/visual resources," and "other resources." CONCLUSION: Study findings represent diverse ways to potentially increase COS use in trials. Future work is needed to examine effects of these behavioral intervention components on COS use. If effective, increased use of COS can improve outcome reporting and minimize outcome heterogeneity and research waste.

Systematic review search strategies are poorly reported and not reproducible: a cross-sectional metaresearch study.

OBJECTIVES: To determine the reproducibility of biomedical systematic review search strategies. STUDY DESIGN AND SETTING: A cross-sectional reproducibility study was conducted on a random sample of 100 systematic reviews indexed in MEDLINE in November 2021. The primary outcome measure is the percentage of systematic reviews for which all database searches can be reproduced, operationalized as fulfilling six key Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension (PRISMA-S) reporting guideline items and having all database searches reproduced within 10% of the number of original results. Key reporting guideline items included database name, multi-database searching, full search strategies, limits and restrictions, date(s) of searches, and total records. RESULTS: The 100 systematic review articles contained 453 database searches. Only 22 (4.9%) database searches reported all six PRISMA-S items. Forty-seven (10.4%) database searches could be reproduced within 10% of the number of results from the original search; six searches differed by more than 1,000% between the originally reported number of results and the reproduction. Only one systematic review article provided the necessary search details to be fully reproducible. CONCLUSION: Systematic review search reporting is poor. To correct this will require a multifaceted response from authors, peer reviewers, journal editors, and database providers.

COHESION: a core outcome set for the treatment of neonatal encephalopathy.

BACKGROUND: Heterogeneity in outcomes reported in trials of interventions for the treatment of neonatal encephalopathy (NE) makes evaluating the effectiveness of treatments difficult. Developing a core outcome set for NE treatment would enable researchers to measure and report the same outcomes in future trials. This would minimise waste, ensure relevant outcomes are measured and enable evidence synthesis. Therefore, we aimed to develop a core outcome set for treating NE. METHODS: Outcomes identified from a systematic review of the literature and interviews with parents were prioritised by stakeholders (n = 99 parents/caregivers, n = 101 healthcare providers, and n = 22 researchers/ academics) in online Delphi surveys. Agreement on the outcomes was achieved at online consensus meetings attended by n = 10 parents, n = 18 healthcare providers, and n = 13 researchers/ academics. RESULTS: Seven outcomes were included in the final core outcome set: survival; brain injury on imaging; neurological status at discharge; cerebral palsy; general cognitive ability; quality of life of the child, and adverse events related to treatment. CONCLUSION: We developed a core outcome set for the treatment of NE. This will allow future trials to measure and report the same outcomes and ensure results can be compared. Future work should identify how best to measure the COS. IMPACT: We have identified seven outcomes that should be measured and reported in all studies for the treatment of neonatal encephalopathy. Previously, a core outcome set for neonatal encephalopathy treatments did not exist. This will help to reduce heterogeneity in outcomes reported in clinical trials and other studies, and help researchers identify the best treatments for neonatal encephalopathy.

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS).

Neonatal sepsis is a serious public health problem; however, there is substantial heterogeneity in the outcomes measured and reported in research evaluating the effectiveness of the treatments. Therefore, we aim to develop a Core Outcome Set (COS) for studies evaluating the effectiveness of treatments for neonatal sepsis. Since a systematic review of key outcomes from randomised trials of therapeutic interventions in neonatal sepsis was published recently, we will complement this with a qualitative systematic review of the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers. We will interpret the outcomes of both studies using a previously established framework. Stakeholders across three different groups i.e., (1) researchers, (2) healthcare providers, and (3) patients' parents/family members and parent representatives will rate the importance of the outcomes in an online Real-Time Delphi Survey. Afterwards, consensus meetings will be held to agree on the final COS through online discussions with key stakeholders. This COS is expected to minimize outcome heterogeneity in measurements and publications, improve comparability and synthesis, and decrease research waste.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INSPECT-SR project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare related interventions. Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: 1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, 2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, 3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in 4) a consensus meeting to select checks to be included in a draft tool and to determine its format, 5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies, and will help patients by protecting them from the influence of false data on their healthcare.

Gynecological trials frequently exclude people based on their symptoms rather than their condition: a systematic review of Cochrane reviews and their component trials.

OBJECTIVES: To identify strategies used in recent randomized controlled trials (RCTs) and their associated Cochrane Reviews where patients with the same gynecological condition present with different symptoms but would plausibly benefit from a common intervention. STUDY DESIGN AND SETTING: We searched the Cochrane library (February 2022) for reviews in polycystic ovarian syndrome (PCOS) and endometriosis. Reviews were included if the intervention was intended to treat all condition-specific symptoms. For each trial we recorded the strategy used and the number of potentially eligible participants excluded as a direct result of the chosen strategy. For each review we recorded the numbers of RCTs and participants excluded on the basis of symptoms experienced. RESULTS: There were 89 distinct PCOS trials in 13 reviews, and 13 Endometriosis trials in 11 reviews. Most trials restricted their eligibility to participants with specific symptoms (55% PCOS, 46% endometriosis). The second most common strategy was to measure and analyze clinical outcomes that were not relevant to all participants (38% PCOS, 31% endometriosis). Reviews excluded 27% of trials in participants evaluating the same intervention in participants experiencing the same condition based on the outcomes measured in the trials. CONCLUSION: Most gynecological trials exclude patients who could benefit from treatment or measure outcomes not relevant to all participants. We introduce a taxonomy to describe trial design strategies for conditions with heterogeneous symptoms.

Multi-Round versus Real-Time Delphi survey approach for achieving consensus in the COHESION core outcome set: a randomised trial.

BACKGROUND: Delphi surveys are commonly used to prioritise critical outcomes in core outcome set (COS) development. This trial aims to compare a three-round (Multi-Round) Delphi (MRD) with a Real-Time Delphi (RTD) in the prioritisation of outcomes for inclusion in a COS for neonatal encephalopathy treatments and explore whether 'feedback', 'iteration', and 'initial condition' effects may occur in the two survey methods. METHODS: We recruited 269 participants (parents/caregivers, healthcare providers and researchers/academics) of which 222 were randomised to either the MRD or the RTD. We investigated the outcomes prioritised in each survey and the 'feedback', 'iteration', and 'initial condition' effects to identify differences between the two survey methods. RESULTS: In the RTD, n = 92 participants (83%) fully completed the survey. In the MRD, n = 60 participants (54%) completed all three rounds. Of the 92 outcomes presented, 26 (28%) were prioritised differently between the RTD and MRD. Significantly fewer participants amended their scores when shown stakeholder responses in the RTD compared to the MRD ('feedback effect'). The 'iteration effect' analysis found most experts appeared satisfied with their initial ratings in the RTD and did not amend their scores following stakeholder response feedback. Where they did amend their scores, ratings were amended substantially, suggesting greater convergence. Variance in scores reduced with subsequent rounds of the MRD ('iteration effect'). Whilst most participants did not change their initial scores in the RTD, of those that did, later recruits tended to align their final score more closely to the group mean final score than earlier recruits (an 'initial condition' effect). CONCLUSION: The feedback effect differed between the two Delphi methods but the magnitude of this difference was small and likely due to the large number of observations rather than because of a meaningfully large difference. It did not appear to be advantageous to require participants to engage in three rounds of a survey due to the low change in scores. Larger drop-out through successive rounds in the MRD, together with a lesser convergence of scores and longer time to completion, indicate considerable benefits of the RTD approach. TRIAL REGISTRATION: NCT04471103. Registered on 14 July 2020.