Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2).

BACKGROUND: Addressing recruitment and retention challenges in trials is a key priority for methods research, but navigating the literature is difficult and time-consuming. In 2016, ORRCA (www.orrca.org.uk) launched a free, searchable database of recruitment research that has been widely accessed and used to support the update of systematic reviews and the selection of recruitment strategies for clinical trials. ORRCA2 aims to create a similar database to map the growing volume and importance of retention research. METHODS: Searches of Medline (Ovid), CINAHL, PsycINFO, Scopus, Web of Science Core Collection and the Cochrane Library, restricted to English language and publications up to the end of 2017. Hand searches of key systematic reviews were undertaken and randomised evaluations of recruitment interventions within the ORRCA database on 1 October 2020 were also reviewed for any secondary retention outcomes. Records were screened by title and abstract before obtaining the full text of potentially relevant articles. Studies reporting or evaluating strategies, methods and study designs to improve retention within healthcare research were eligible. Case reports describing retention challenges or successes and studies evaluating participant reported reasons for withdrawal or losses were also included. Studies assessing adherence to treatments, attendance at appointments outside of research and statistical analysis methods for missing data were excluded. Eligible articles were categorised into one of the following evidence types: randomised evaluations, non-randomised evaluations, application of retention strategies without evaluation and observations of factors affecting retention. Articles were also mapped against a retention domain framework. Additional data were extracted on research outcomes, methods and host study context. RESULTS: Of the 72,904 abstracts screened, 4,364 full texts were obtained, and 1,167 articles were eligible. Of these, 165 (14%) were randomised evaluations, 99 (8%) non-randomised evaluations, 319 (27%) strategies without evaluation and 584 (50%) observations of factors affecting retention. Eighty-four percent (n = 979) of studies assessed the numbers of participants retained, 27% (n = 317) assessed demographic differences between retained and lost participants, while only 4% (n = 44) assessed the cost of retention strategies. The most frequently reported domains within the 165 studies categorised as 'randomised evaluations of retention strategies' were participant monetary incentives (32%), participant reminders and prompts (30%), questionnaire design (30%) and data collection location and method (26%). CONCLUSION: ORRCA2 builds on the success of ORRCA extending the database to organise the growing volume of retention research. Less than 15% of articles were randomised evaluations of retention strategies. Mapping of the literature highlights several areas for future research such as the role of research sites, clinical staff and study design in enhancing retention. Future studies should also include cost-benefit analysis of retention strategies.

Factors that impact on recruitment to randomised trials in health care: a qualitative evidence synthesis.

BACKGROUND: Randomised trials (also referred to as 'randomised controlled trials' or 'trials') are the optimal way to minimise bias in evaluating the effects of competing treatments, therapies and innovations in health care. It is important to achieve the required sample size for a trial, otherwise trialists may not be able to draw conclusive results leading to research waste and raising ethical questions about trial participation. The reasons why potential participants may accept or decline participation are multifaceted. Yet, the evidence of effectiveness of interventions to improve recruitment to trials is not substantial and fails to recognise these individual decision-making processes. It is important to synthesise the experiences and perceptions of those invited to participate in randomised trials to better inform recruitment strategies. OBJECTIVES: To explore potential trial participants' views and experiences of the recruitment process for participation. The specific objectives are to describe potential participants' perceptions and experiences of accepting or declining to participate in trials, to explore barriers and facilitators to trial participation, and to explore to what extent barriers and facilitators identified are addressed by strategies to improve recruitment evaluated in previous reviews of the effects of interventions including a Cochrane Methodology Review. SEARCH METHODS: We searched the Cochrane Library, Medline, Embase, CINAHL, Epistemonikos, LILACS, PsycINFO, ORRCA, and grey literature sources. We ran the most recent set of searches for which the results were incorporated into the review in July 2017. SELECTION CRITERIA: We included qualitative and mixed-methods studies (with an identifiable qualitative component) that explored potential trial participants' experiences and perceptions of being invited to participate in a trial. We excluded studies that focused only on recruiters' perspectives, and trials solely involving children under 18 years, or adults who were assessed as having impaired mental capacity. DATA COLLECTION AND ANALYSIS: Five review authors independently assessed the titles, abstracts and full texts identified by the search. We used the CART (completeness, accuracy, relevance, timeliness) criteria to exclude studies that had limited focus on the phenomenon of interest. We used QSR NVivo to extract and manage the data. We assessed methodological limitations using the Critical Skills Appraisal Programme (CASP) tool. We used thematic synthesis to analyse and synthesise the evidence. This provided analytical themes and a conceptual model. We used the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in each finding. Our findings were integrated with two previous intervention effectiveness reviews by juxtaposing the quantitative and qualitative findings in a matrix. MAIN RESULTS: We included 29 studies (published in 30 papers) in our synthesis. Twenty-two key findings were produced under three broad themes (with six subthemes) to capture the experience of being invited to participate in a trial and making the decision whether to participate. Most of these findings had moderate to high confidence. We identified factors from the trial itself that influenced participation. These included how trial information was communicated, and elements of the trial such as the time commitment that might be considered burdensome. The second theme related to personal factors such as how other people can influence the individual's decision; and how a personal understanding of potential harms and benefits could impact on the decision. Finally, the potential benefits of participation were found to be key to the decision to participate, namely personal benefits such as access to new treatments, but also the chance to make a difference and help others. The conceptual model we developed presents the decision-making process as a gauge and the factors that influence whether the person will, or will not, take part. AUTHORS' CONCLUSIONS: This qualitative evidence synthesis has provided comprehensive insight into the complexity of factors that influence a person's decision whether to participate in a trial. We developed key questions that trialists can ask when developing their recruitment strategy. In addition, our conceptual model emphasises the need for participant-centred approaches to recruitment. We demonstrated moderate to high level confidence in our findings, which in some way can be attributed to the large volume of highly relevant studies in this field. We recommend that these insights be used to direct or influence or underpin future recruitment strategies that are developed in a participant-driven way that ultimately improves trial conduct and reduces research waste.

Development of an online resource for recruitment research in clinical trials to organise and map current literature.

BACKGROUND: Recruiting the target number of participants within the pre-specified time frame agreed with funders remains a common challenge in the completion of a successful clinical trial and addressing this is an important methodological priority. While there is growing research around recruitment, navigating this literature to support an evidence-based approach remains difficult. The Online resource for Recruitment Research in Clinical triAls project aims to create an online searchable database of recruitment research to improve access to existing evidence and to identify gaps for future research. METHODS: MEDLINE (Ovid), Scopus, Cochrane Database of Systematic Reviews and Cochrane Methodology Register, Science Citation Index Expanded and Social Sciences Citation Index within the ISI Web of Science and Education Resources Information Center were searched in January 2015. Search strategy results were screened by title and abstract, and full text obtained for potentially eligible articles. Studies reporting or evaluating strategies, interventions or methods used to recruit patients were included along with case reports and studies exploring reasons for patient participation or non-participation. Eligible articles were categorised as systematic reviews, nested randomised controlled trials and other designs evaluating the effects of recruitment strategies (Level 1); studies that report the use of recruitment strategies without an evaluation of impact (Level 2); or articles reporting factors affecting recruitment without presenting a particular recruitment strategy (Level 3). Articles were also assigned to 1, or more, of 42 predefined recruitment domains grouped under 6 categories. RESULTS: More than 60,000 records were retrieved by the search, resulting in 56,030 unique titles and abstracts for screening, with a further 23 found through hand searches. A total of 4570 full text articles were checked; 2804 were eligible. Six percent of the included articles evaluated the effectiveness of a recruitment strategy (Level 1), with most of these assessing aspects of participant information, either its method of delivery (33%) or its content and format (28%). DISCUSSION: Recruitment to clinical trials remains a common challenge and an important area for future research. The online resource for Recruitment Research in Clinical triAls project provides a searchable, online database of research relevant to recruitment. The project has identified the need for researchers to evaluate their recruitment strategies to improve the evidence base and broaden the narrow focus of existing research to help meet the complex challenges faced by those recruiting to clinical trials.

Strategies to improve recruitment to randomised trials.

BACKGROUND: Recruiting participants to trials can be extremely difficult. Identifying strategies that improve trial recruitment would benefit both trialists and health research. OBJECTIVES: To quantify the effects of strategies for improving recruitment of participants to randomised trials. A secondary objective is to assess the evidence for the effect of the research setting (e.g. primary care versus secondary care) on recruitment. SEARCH METHODS: We searched the Cochrane Methodology Review Group Specialised Register (CMR) in the Cochrane Library (July 2012, searched 11 February 2015); MEDLINE and MEDLINE In Process (OVID) (1946 to 10 February 2015); Embase (OVID) (1996 to 2015 Week 06); Science Citation Index & Social Science Citation Index (ISI) (2009 to 11 February 2015) and ERIC (EBSCO) (2009 to 11 February 2015). SELECTION CRITERIA: Randomised and quasi-randomised trials of methods to increase recruitment to randomised trials. This includes non-healthcare studies and studies recruiting to hypothetical trials. We excluded studies aiming to increase response rates to questionnaires or trial retention and those evaluating incentives and disincentives for clinicians to recruit participants. DATA COLLECTION AND ANALYSIS: We extracted data on: the method evaluated; country in which the study was carried out; nature of the population; nature of the study setting; nature of the study to be recruited into; randomisation or quasi-randomisation method; and numbers and proportions in each intervention group. We used a risk difference to estimate the absolute improvement and the 95% confidence interval (CI) to describe the effect in individual trials. We assessed heterogeneity between trial results. We used GRADE to judge the certainty we had in the evidence coming from each comparison. MAIN RESULTS: We identified 68 eligible trials (24 new to this update) with more than 74,000 participants. There were 63 studies involving interventions aimed directly at trial participants, while five evaluated interventions aimed at people recruiting participants. All studies were in health care.We found 72 comparisons, but just three are supported by high-certainty evidence according to GRADE.1. Open trials rather than blinded, placebo trials. The absolute improvement was 10% (95% CI 7% to 13%).2. Telephone reminders to people who do not respond to a postal invitation. The absolute improvement was 6% (95% CI 3% to 9%). This result applies to trials that have low underlying recruitment. We are less certain for trials that start out with moderately good recruitment (i.e. over 10%).3. Using a particular, bespoke, user-testing approach to develop participant information leaflets. This method involved spending a lot of time working with the target population for recruitment to decide on the content, format and appearance of the participant information leaflet. This made little or no difference to recruitment: absolute improvement was 1% (95% CI -1% to 3%).We had moderate-certainty evidence for eight other comparisons; our confidence was reduced for most of these because the results came from a single study. Three of the methods were changes to trial management, three were changes to how potential participants received information, one was aimed at recruiters, and the last was a test of financial incentives. All of these comparisons would benefit from other researchers replicating the evaluation. There were no evaluations in paediatric trials.We had much less confidence in the other 61 comparisons because the studies had design flaws, were single studies, had very uncertain results or were hypothetical (mock) trials rather than real ones. AUTHORS' CONCLUSIONS: The literature on interventions to improve recruitment to trials has plenty of variety but little depth. Only 3 of 72 comparisons are supported by high-certainty evidence according to GRADE: having an open trial and using telephone reminders to non-responders to postal interventions both increase recruitment; a specialised way of developing participant information leaflets had little or no effect. The methodology research community should improve the evidence base by replicating evaluations of existing strategies, rather than developing and testing new ones.

Soluble pre-fibrillar tau and ß-amyloid species emerge in early human Alzheimer's disease and track disease progression and cognitive decline.

Post-mortem investigations of human Alzheimer's disease (AD) have largely failed to provide unequivocal evidence in support of the original amyloid cascade hypothesis, which postulated deposition of ß-amyloid (Aß) aggregates to be the cause of a demented state as well as inductive to tau neurofibrillary tangles (NFTs). Conflicting evidence suggests, however, that Aß plaques and NFTs, albeit to a lesser extent, are present in a substantial subset of non-demented individuals. Hence, a range of soluble tau and Aß species has more recently been implicated as the disease-relevant toxic entities. Despite the incorporation of soluble proteins into a revised amyloid cascade hypothesis, a detailed characterization of these species in the context of human AD onset, progression and cognitive decline has been lacking. Here, lateral temporal lobe samples (Brodmann area 21) of 46 human cases were profiled via tau and Aß Western blot and native state dot blot protocols. Elevations in phospho-tau (antibodies: CP13, AT8 and PHF-1), pathological tau conformations (MC-1) and oligomeric tau (TOC1) agreed with medical diagnosis (non-AD cf. AD) and Braak stage classification (low, intermediate and high), alongside elevations in soluble Aß species (MOAB-2 and pyro-glu Aß) and a decline in levels of the amyloid precursor protein. Strong correlations were observed between individual Braak stages and multiple cognitive measures with all tau markers as well as total soluble Aß. In contrast to previous reports, SDS-stable Aß oligomers (*56) were not found to be reliable for all classifications and appeared likely to be a technical artefact. Critically, the robust predictive value of total soluble Aß was dependent on native state quantification. Elevations in tau and Aß within soluble fractions (Braak stage 2-3 cf. 0) were evident earlier than previously established in fibril-focused disease progression scales. Together, these data provide strong evidence that soluble forms of tau and Aß co-localise early in AD and are closely linked to disease progression and cognitive decline.

Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability.

OBJECTIVES: People from ethnic minority groups are underserved by randomized trials, and poor representation of these groups reduces generalizability of results. There is no guidance on which ethnicity categories are appropriate for use in trials and thus inconsistency exists. The purpose of this study is to establish, in a large sample of trials, if participant ethnicity is recorded, how it is obtained (categories used), and if its reporting varies from its recording. STUDY DESIGN AND SETTING: We reviewed trial documentation for 407 randomized controlled trials published in the UK National Institute of Health Research library from 2016 to 2021. We extracted data on the recording (if it was recorded and the categories used) and reporting (if the categories remained the same as those obtained, or not) of ethnicity for each trial along with demographics. In the analysis we categorized the manner of recording and reporting of ethnicity in the trials according to UK Census ethnicity categories. RESULTS: Ethnicity was recorded in 67.3% (n = 274) of trials. The location in the trial report where ethnicity was recorded was available for 42% (n = 116) of trials. The details on how ethnicity was collected (predefined categories or self-defined) was available for 54/274 (20%) of trials and details on the specifics of the categories recorded was available for 44 (16%) trials. Of the 44, 6 of those did not go on to report on ethnicity in the trial report. Of the remaining 38, only 13 reported ethnicity exactly as it had been recorded. Taken as a whole from the 407 trial reports examined 9.3% (38/407) of trials demonstrated exactly how they both recorded, and reported, ethnicity. Authors made reference to whom results were relevant in terms of ethnicity in 80/407 (19.7%). CONCLUSION: Ethnicity is underrecorded and underreported in clinical trials. This is a threat to the generalizability of the findings and needs to be improved.

Coming through when it matters most.

All teams would like to think they do their best work when the stakes are highest-when the company's future or their own rests on the outcome of their projects. But too often something else happens. In extensive studies of teams at professional service firms, Harvard Business School's Gardner has seen the same pattern emerge over and over: Teams become increasingly concerned with the risks of failure rather than the requirements of excellence. As a result, they revert to safe, standard approaches instead of delivering original solutions tailored to clients' needs. Gardner has a name for this phenomenon: the performance pressure paradox. Here's how it develops: As pressure mounts, team members start driving toward consensus in ways that shut out vital information. Without even realizing it, they give more weight to shared knowledge and dismiss specialized expertise, such as insights into the client's technologies, culture, and aspirations. The more generically inclined the team becomes, the more concerned the client grows, which turns up the pressure and pushes the team even further down the generic road. But forewarned is forearmed. By measuring each person's contribution deliberately, ruthlessly insisting that no one's contribution be marginalized, and framing new information within familiar contexts, teams can escape the performance pressure paradox and keep doing their best work when it matters most.

A good use of time? Providing evidence for how effort is invested in primary and secondary outcome data collection in trials.

BACKGROUND: Data collection is a substantial part of trial workload for participants and staff alike. How these hours of work are spent is important because stakeholders are more interested in some outcomes than others. The ORINOCO study compared the time spent collecting primary outcome data to the time spent collecting secondary outcome data in a cohort of trials. METHODS: We searched PubMed for phase III trials indexed between 2015 and 2019. From these, we randomly selected 120 trials evaluating a therapeutic intervention plus an additional random selection of 20 trials evaluating a public health intervention. We also added eligible trials from a cohort of 189 trials in rheumatology that had used the same core outcome set. We then obtained the time taken to collect primary and secondary outcomes in each trial. We used a hierarchy of methods that included data in trial reports, contacting the trial team and approaching individuals with experience of using the identified outcome measures. We calculated the primary to secondary data collection time ratio and notional data collection cost for each included trial. RESULTS: We included 161 trials (120 phase III; 21 core outcome set; 20 public health), which together collected 230 primary and 688 secondary outcomes. Full primary and secondary timing data were obtained for 134 trials (100 phase III; 17 core outcome set; 17 public health). The median time spent on primaries was 56.1 h (range: 0.0-10,746.7, IQR: 226.89) and the median time spent on secondaries was 190.7 hours (range: 0.0-1,356,832.9, IQR: 617.6). The median primary to secondary data collection time ratio was 1.0:3.0 (i.e. for every minute spent on primary outcomes, 3.0 were spent on secondaries). The ratio varied by trial type: phase III trials were 1.0:3.1, core outcome set 1.0:3.4 and public health trials 1.0:2.2. The median notional overall data collection cost was £8015.73 (range: £52.90-£31,899,140.70, IQR: £20,096.64). CONCLUSIONS: Depending on trial type, between two and three times as much time is spent collecting secondary outcome data than collecting primary outcome data. Trial teams should explicitly consider how long it will take to collect the data for an outcome and decide whether that time is worth it given importance of the outcome to the trial.

The People's Trial: supporting the public's understanding of randomised trials.

BACKGROUND: Randomised trials are considered the gold standard in providing robust evidence on the effectiveness of interventions. However, there are relatively few initiatives to help increase public understanding of what randomised trials are and why they are important. This limits the overall acceptance of and public participation in clinical trials. The People's Trial aims to help the public learn about randomised trials, to understand why they matter, and to be better equipped to think critically about health claims by actively involving them in all aspects of trial design. This was done by involving the public in the design, conduct, and dissemination of a randomised trial. METHODS: Using a reflexive approach, we describe the processes of development, conduct, and dissemination of The People's Trial. RESULTS: Over 3000 members of the public, from 72 countries, participated in The People's Trial. Through a series of online surveys, the public designed a trial called The Reading Trial. They chose the question the trial would try to answer and decided the components of the trial question. In December 2019, 991 participants were recruited to a trial to answer the question identified and prioritised by the public, i.e. 'Does reading a book in bed make a difference to sleep in comparison with not reading a book in bed?' We report the processes of The People's Trial in seven phases, paralleling the steps of a randomised trial, i.e. question identification and prioritisation, recruitment, randomisation, trial conduct, data analysis, and sharing of findings. We describe the decisions we made, the processes we used, the challenges we encountered, and the lessons we learned. CONCLUSION: The People's Trial involved the public successfully in the design, conduct, and dissemination of a randomised trial demonstrating the potential for such initiatives to help the public learn about randomised trials, to understand why they matter, and to be better equipped to think critically about health claims. TRIAL REGISTRATION: ClinicalTrials.gov NCT04185818 . Registered on 4 December 2019.

Trial Forge Guidance 3: randomised trials and how to recruit and retain individuals from ethnic minority groups-practical guidance to support better practice.

Randomised trials, especially those intended to directly inform clinical practice and policy, should be designed to reflect all those who could benefit from the intervention under test should it prove effective. This does not always happen. The UK National Institute for Health and Care Research (NIHR) INCLUDE project identified many groups in the UK that are under-served by trials, including ethnic minorities.This guidance document presents four key recommendations for designing and running trials that include the ethnic groups needed by the trial. These are (1) ensure eligibility criteria and recruitment pathway do not limit participation in ways you do not intend, (2) ensure your trial materials are developed with inclusion in mind, (3) ensure staff are culturally competent and (4) build trusting partnerships with community organisations that work with ethnic minority groups. Each recommendation comes with best practice advice, public contributor testimonials, examples of the inclusion problem tackled by the recommendation, or strategies to mitigate the problem, as well as a collection of resources to support implementation of the recommendations.We encourage trial teams to follow the recommendations and, where possible, evaluate the strategies they use to implement them. Finally, while our primary audience is those designing, running and reporting trials, we hope funders, grant reviewers and approvals agencies may also find our guidance useful.

An international core outcome set for evaluating interventions to improve informed consent to clinical trials: The ELICIT Study.

OBJECTIVE: To develop a core outcome set for the evaluation of interventions that aim to improve how people make decisions about whether to participate in randomized controlled trials (of healthcare interventions), the ELICIT Study. STUDY DESIGN: International mixed-method study involving a systematic review of existing outcomes, semi-structured interviews, an online Delphi survey, and a face-to-face consensus meeting. RESULTS: The literature review and stakeholder interviews (n = 25) initially identified 1045 reported outcomes that were grouped into 40 individually distinct outcomes. These 40 outcomes were scored for importance in two rounds of an online Delphi survey (n = 79), with 18 people attending the consensus meeting. Consensus was reached on 12 core outcomes: therapeutic misconception; comfort with decision; authenticity of decision; communication about the trial; empowerment; sense of altruism; equipoise; knowledge; salience of questions; understanding, how helpful the process was for decision making; and trial attrition. CONCLUSION: The ELICIT core outcome set is the first internationally agreed minimum set of outcomes deemed essential to be measured in all future studies evaluating interventions to improve decisions about participating in an randomized controlled trial. Use of the ELICIT core set will ensure that results from these trials are comparable and relevant to all stakeholders. REGISTRATION: COMET database - http://www.comet-initiative.org/Studies/Details/595.

Does reading a book in bed make a difference to sleep in comparison to not reading a book in bed? The People's Trial-an online, pragmatic, randomised trial.

BACKGROUND: The best way of comparing healthcare treatments is through a randomised trial. In a randomised trial, we compare something (a treatment or intervention) to something else, often another treatment. Who gets what is decided at random, meaning everyone has an equal chance of getting any of the treatments. This means any differences found can be put down to the treatment received rather than other things, such as where people live, or health conditions they might have. The People's Trial aimed to help the public better understand randomised trials by inviting them to design and carry out a trial. The question chosen by the public for The People's Trial was: 'Does reading a book in bed make a difference to sleep, in comparison to not reading a book in bed?' This paper describes that trial, called 'The Reading Trial'. METHODS: The Reading Trial was an online, randomised trial. Members of the public were invited to take part through social media campaigns. People were asked to either read a book in bed before going to sleep (intervention group) or not read a book in bed before going to sleep (control group). We asked everyone to do this for 7 days, after which they measured their sleep quality. RESULTS: During December 2019, a total of 991 people took part in The Reading Trial, half (496 (50%)) in the intervention group and half (495 (50%)) in the control group. Not everyone finished the trial: 127 (25.6%) people in the intervention group and 90 (18.18%) people in the control group. Of those providing data, 156/369 (42%) people in the intervention group felt their sleep improved, compared to 112/405 (28%) of those in the control group, a difference of 14%. When we consider how certain we are of this finding, we estimate that, in The Reading Trial, sleep improved for between 8 and 22% more people in the intervention group compared to the control group. CONCLUSIONS: Reading a book in bed before going to sleep improved sleep quality, compared to not reading a book in bed. TRIAL REGISTRATION: ClinicalTrials.gov NCT04185818. Registered on 4 December 2019.

Developing the INCLUDE Ethnicity Framework-a tool to help trialists design trials that better reflect the communities they serve.

BACKGROUND: Ensuring that a trial is designed so that its participants reflect those who might benefit from the results, or be spared harms, is key to the potential benefits of the trial reaching all they should. This paper describes the process, facilitated by Trial Forge, that was used between July 2019 and October 2020 to develop the INCLUDE Ethnicity Framework, part of the wider INCLUDE initiative from the National Institute for Health Research to improve inclusion of under-served groups in clinical research studies. METHODS: Development of the Framework was done in seven phases: (1) outline, (2) initial draft, (3) stakeholder meeting, (4) modify draft, (5) Stakeholder feedback, (6) applying the Framework and (7) packaging. Phases 2 and 3 were face-to-face meetings. Consultation with stakeholders was iterative, especially phases 4 to 6. Movement to the next phase was done once all or most stakeholders were comfortable with the results of the current phase. When there was a version of the Framework that could be considered final, the Framework was applied to six trials to create a set of examples (phase 6). Finally, the Framework, guidance and examples were packaged ready for dissemination (phase 7). RESULTS: A total of 40 people from stakeholder groups including patient and public partners, clinicians, funders, academics working with various ethnic groups, trial managers and methodologists contributed to the seven phases of development. The Framework comprises two parts. The first part is a list of four key questions: 1. Who should my trial apply to? 2. Are the groups identified likely to respond in different ways? 3. Will my study intervention make it harder for some groups to engage? 4. Will the way I have designed the study make it harder for some groups to engage? The second part is a set of worksheets to help trial teams address these questions. The Framework can be used for any stage of trial, for a healthcare intervention in any disease area. The Framework was launched on 1st October 2020 and is available open access at the Trial Forge website: https://www.trialforge.org/trial-forge-centre/include/ . CONCLUSION: Thinking about the number of people in our trials is not enough: we need to start thinking more carefully about who our participants are.

A systematic review of non-randomised evaluations of strategies to improve participant recruitment to randomised controlled trials.

Background: Recruitment to trials can be challenging. Currently, non-randomised evaluations of trial recruitment interventions are rejected due to poor methodological quality, but systematic assessment of this substantial body of work may inform trialists' decision-making about recruitment methods. Our objective was to quantify the effects of strategies to improve participant recruitment to randomised trials evaluated using non-randomised study designs. Methods: We searched relevant databases for non-randomised studies that included two or more interventions evaluating recruitment to trials. Two reviewers screened abstracts and full texts for eligible studies, then extracted data on: recruitment intervention, setting, participant characteristics, number of participants in intervention and comparator groups. The ROBINS-I tool was used to assess risk of bias. The primary outcome was the number of recruits to a trial. Results: We identified 92 studies for inclusion; 90 studies aimed to improve the recruitment of participants, one aimed to improve the recruitment of GP practices, and one aimed to improve recruitment of GPs. Of the 92 included studies, 20 were at high risk of bias due to confounding; the remaining 72 were at high risk of bias due to confounding and at least one other category of the ROBINS-I tool. The 20 studies at least risk of bias were synthesised narratively based on seven broad categories; Face to face recruitment initiatives, postal invitations and responses, language adaptations, randomisation methods, trial awareness strategies aimed at the recruitee, trial awareness strategies aimed at the recruiter, and use of networks and databases. The utility of included studies is substantially limited due to small sample sizes, inadequate reporting, and a lack of coordination around deciding what to evaluate and how. Conclusions: Careful thought around planning, conduct, and reporting of non-randomised evaluations of recruitment interventions is required to prevent future non-randomised studies contributing to research waste. Registration: PROSPERO CRD42016037718.

Non-randomised evaluations of strategies to increase participant retention in randomised controlled trials: a systematic review.

BACKGROUND: Retention of participants is essential to ensure the statistical power and internal validity of clinical trials. Poor participant retention reduces power and can bias the estimates of intervention effect. There is sparse evidence from randomised comparisons of effective strategies to retain participants in randomised trials. Currently, non-randomised evaluations of trial retention interventions embedded in host clinical trials are rejected from the Cochrane review of strategies to improve retention because it only included randomised evaluations. However, the systematic assessment of non-randomised evaluations may inform trialists' decision-making about retention methods that have been evaluated in a trial context.Therefore, we performed a systematic review to synthesise evidence from non-randomised evaluations of retention strategies in order to supplement existing randomised trial evidence. METHODS: We searched MEDLINE, EMBASE, and Cochrane CENTRAL from 2007 to October 2017. Two reviewers independently screened abstracts and full-text articles for non-randomised studies that compared two or more strategies to increase participant retention in randomised trials. The retention trials had to be nested in real 'host' trials ( including feasibility studies) but not hypothetical trials. Two investigators independently rated the risk of bias of included studies using the ROBINS-I tool and determined the certainty of evidence using GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework. RESULTS: Fourteen non-randomised studies of retention were included in this review. Most retention strategies (in 10 studies) aimed to increase questionnaire response rate. Favourable strategies for increasing questionnaire response rate were telephone follow-up compared to postal questionnaire completion, online questionnaire follow-up compared to postal questionnaire, shortened version of questionnaires versus longer questionnaires, electronically transferred monetary incentives compared to cash incentives, cash compared with no incentive and reminders to non-responders (telephone or text messaging). However, each retention strategy was evaluated in a single observational study. This, together with risk of bias concerns, meant that the overall GRADE certainty was low or very low for all included studies. CONCLUSIONS: This systematic review provides low or very low certainty evidence on the effectiveness of retention strategies evaluated in non-randomised studies. Some strategies need further evaluation to provide confidence around the size and direction of the underlying effect.

Using systematic data categorisation to quantify the types of data collected in clinical trials: the DataCat project.

BACKGROUND: Data collection consumes a large proportion of clinical trial resources. Each data item requires time and effort for collection, processing and quality control procedures. In general, more data equals a heavier burden for trial staff and participants. It is also likely to increase costs. Knowing the types of data being collected, and in what proportion, will be helpful to ensure that limited trial resources and participant goodwill are used wisely. AIM: The aim of this study is to categorise the types of data collected across a broad range of trials and assess what proportion of collected data each category represents. METHODS: We developed a standard operating procedure to categorise data into primary outcome, secondary outcome and 15 other categories. We categorised all variables collected on trial data collection forms from 18, mainly publicly funded, randomised superiority trials, including trials of an investigational medicinal product and complex interventions. Categorisation was done independently in pairs: one person having in-depth knowledge of the trial, the other independent of the trial. Disagreement was resolved through reference to the trial protocol and discussion, with the project team being consulted if necessary. KEY RESULTS: Primary outcome data accounted for 5.0% (median)/11.2% (mean) of all data items collected. Secondary outcomes accounted for 39.9% (median)/42.5% (mean) of all data items. Non-outcome data such as participant identifiers and demographic data represented 32.4% (median)/36.5% (mean) of all data items collected. CONCLUSION: A small proportion of the data collected in our sample of 18 trials was related to the primary outcome. Secondary outcomes accounted for eight times the volume of data as the primary outcome. A substantial amount of data collection is not related to trial outcomes. Trialists should work to make sure that the data they collect are only those essential to support the health and treatment decisions of those whom the trial is designed to inform.

Trial Forge Guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed.

The evidence base available to trialists to support trial process decisions-e.g. how best to recruit and retain participants, how to collect data or how to share the results with participants-is thin. One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. These are self-contained research studies embedded within a host trial that aim to evaluate or explore alternative ways of delivering or organising a particular trial process.SWATs are increasingly being supported by funders and considered by trialists, especially in the UK and Ireland. At some point, increasing SWAT evidence will lead funders and trialists to ask: given the current body of evidence for a SWAT, do we need a further evaluation in another host trial? A framework for answering such a question is needed to avoid SWATs themselves contributing to research waste.This paper presents criteria on when enough evidence is available for SWATs that use randomised allocation to compare different interventions.

Using evidence when planning for trial recruitment: An international perspective from time-poor trialists.

INTRODUCTION: Recruiting participants to trials is challenging. To date, research has focussed on improving recruitment once the trial is underway, rather than planning strategies to support it, e.g. developing trial information leaflets together with people like those to be recruited. We explored whether people involved with participant recruitment have explicit planning strategies; if so, how these are developed, and if not, what prevents effective planning. METHODS: Design: Individual qualitative semi-structured interviews. Data were analysed using a Framework approach, and themes linked through comparison of data within and across stakeholder groups. Participants: 23 international trialists (UK, Canada, South Africa, Italy, the Netherlands); 11 self-identifying as 'Designers'; those who design recruitment methods, and 12 self-identifying as 'Recruiters'; those who recruit participants. Interviewees' had recruitment experience spanning diverse interventions and clinical areas. Setting: Primary, secondary and tertiary-care sites involved in trials, academic institutions, and contract research organisations supporting pharmaceutical companies. RESULTS: To varying degrees, respondents had prospective strategies for recruitment. These were seldom based on rigorous evidence. When describing their recruitment planning experiences, interviewees identified a range of influences that they believe impacted success: The timing of recruitment strategy development relative to the trial start date, and who is responsible for recruitment planning.The methods used to develop trialists' recruitment strategy design and implementation skills, and when these skills are gained (i.e. before the trial or throughout).The perceived barriers and facilitators to successful recruitment planning; and how trialists modify practice when recruitment is poor. CONCLUSIONS: Respondents from all countries considered limited time and disproportionate approvals processes as major challenges to recruitment planning. Poor planning is a mistake that trialists live with throughout the trial. The experiences of our participants suggest that effective recruitment requires strategies to increase the time for trial planning, as well as access to easily implementable evidence-based strategies.

What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study.

BACKGROUND: One of the top three research priorities for the UK clinical trial community is to address the gap in evidence-based approaches to improving participant retention in randomised trials. Despite this, there is little evidence supporting methods to improve retention. This paper reports the PRioRiTy II project, a Priority Setting Partnership (PSP) that identified and prioritised unanswered questions and uncertainties around trial retention in collaboration with key stakeholders. METHODS: This PSP was conducted in collaboration with the James Lind Alliance, a non-profit making initiative, to support key stakeholders (researchers, patients, and the public) in jointly identifying and agreeing on priority research questions. There were three stages. (1) First an initial online survey was conducted consisting of six open-ended questions about retention in randomised trials. Responses were coded into thematic groups to create a longlist of questions. The longlist of questions was checked against existing evidence to ensure that they had not been answered by existing research. (2) An interim stage involved a further online survey where stakeholders were asked to select questions of key importance from the longlist. (3) A face-to-face consensus meeting was held, where key stakeholder representatives agreed on an ordered list of 21 unanswered research questions for methods of improving retention in randomised trials. RESULTS: A total of 456 respondents yielded 2431 answers to six open-ended questions, from which 372 questions specifically about retention were identified. Further analysis included thematically grouping all data items within answers and merging questions in consultation with the Steering Group. This produced 27 questions for further rating during the interim survey. The top 21 questions from the interim online survey were brought to a face-to-face consensus meeting in which key stakeholder representatives prioritised the order. The 'Top 10' of these are reported in this paper. The number one ranked question was 'What motivates a participant's decision to complete a clinical trial?' The entire list will be available at www.priorityresearch.ie . CONCLUSION: The Top 10 list can inform the direction of future research on trial methods and be used by funders to guide projects aiming to address and improve retention in randomised trials.

A protocol for a systematic review of non-randomised evaluations of strategies to increase participant retention to randomised controlled trials.

BACKGROUND: Randomised control trials are regarded as the gold standard for evaluating the effectiveness and efficacy of healthcare interventions with thousands of trials published every year. Despite significant investment in infrastructure, a staggering number of clinical trials continue to face challenges with retention. Dropouts could lead to negative consequences-from lengthy delays to missing data that can undermine the results and integrity of the trial. Summarising evidence from non-randomised evaluations of retention strategies could provide complementary information to randomised evaluations that could guide trialists to the most effective ways of increasing retention of participants in clinical trials. METHODS: The following electronic databases will be searched for relevant studies: EMBASE, MEDLINE, the Cochrane Controlled Trials Register, and Cochrane Methodology Register and the search will be limited to English-published studies during the last 10 years to increase relevance to current trials. Non-randomised studies (observational studies) including a comparison of two or more strategies to increase participant retention in randomised trials or comparing one or more strategies with no strategy will be included. The primary outcome will be the proportion of participants remained at the primary analysis as determined in each retention study. DISCUSSION: This review aims to gather and evaluate evidence on the effect of retention strategies examined in non-randomised studies. It is imperative to collect evidence from obseravational studies to infer whether or not these studies could be considered a practical way to complement or even replace a broadly favourable randomised design. If we find that non-randomised studies to be included in this review are of high quality with adequate control of biases, we will recommend to trialists and others not to rely exclusively on randomised studies and to give meticulous attention to the plentiful evidence that can be obtained from non-randomised studies. Should the results of this review suggest that evaluating retention strategies in observational studies provides insufficient evidence to trialists planning their retention strategies, we will be able to say that there is little point in conducting non-randomised studies and that they would do better to invest their time and resources in a randomised evaluation if possible. Where a non-randomised study design is chosen, the review authors will offer recommendations to trialists and others regarding how to ensure that these studies are conducted in a way that can minimise the risk of bias and increase confidence in the findings. SYSTEMATIC REVIEW REGISTRATION: PROSPERO 2017: CRD42017072775 .