Factors to Consider During Identification and Invitation of Individuals in a Multi-stakeholder Research Partnership.

BACKGROUND: Health research teams increasingly partner with stakeholders to produce research that is relevant, accessible, and widely used. Previous work has covered stakeholder group identification. OBJECTIVE: We aimed to develop factors for health research teams to consider during identification and invitation of individual representatives in a multi-stakeholder research partnership, with the aim of forming equitable and informed teams. DESIGN: Consensus development. PARTICIPANTS: We involved 16 stakeholders from the international Multi-Stakeholder Engagement (MuSE) Consortium, including patients and the public, providers, payers of health services/purchasers, policy makers, programme managers, peer review editors, and principal investigators. APPROACH: We engaged stakeholders in factor development and as co-authors of this manuscript. Using a modified Delphi approach, we gathered stakeholder views concerning a preliminary list of 18 factors. Over two feedback rounds, using qualitative and quantitative analysis, we concentrated these into ten factors. KEY RESULTS: We present seven highly desirable factors: 'expertise or experience', 'ability and willingness to represent the stakeholder group', 'inclusivity (equity, diversity and intersectionality)', 'communication skills', 'commitment and time capacity', 'financial and non-financial relationships and activities, and conflict of interest', 'training support and funding needs'. Additionally, three factors are desirable: 'influence', 'research relevant values', 'previous stakeholder engagement'. CONCLUSIONS: We present factors for research teams to consider during identification and invitation of individual representatives in a multi-stakeholder research partnership. Policy makers and guideline developers may benefit from considering the factors in stakeholder identification and invitation. Research funders may consider stipulating consideration of the factors in funding applications. We outline how these factors can be implemented and exemplify how their use has the potential to improve the quality and relevancy of health research.

Audio-/Videorecording Clinic Visits for Patient's Personal Use in the United States: Cross-Sectional Survey.

BACKGROUND: Few clinics in the United States routinely offer patients audio or video recordings of their clinic visits. While interest in this practice has increased, to date, there are no data on the prevalence of recording clinic visits in the United States. OBJECTIVE: Our objectives were to (1) determine the prevalence of audiorecording clinic visits for patients' personal use in the United States, (2) assess the attitudes of clinicians and public toward recording, and (3) identify whether policies exist to guide recording practices in 49 of the largest health systems in the United States. METHODS: We administered 2 parallel cross-sectional surveys in July 2017 to the internet panels of US-based clinicians (SERMO Panel) and the US public (Qualtrics Panel). To ensure a diverse range of perspectives, we set quotas to capture clinicians from 8 specialties. Quotas were also applied to the public survey based on US census data (gender, race, ethnicity, and language other than English spoken at home) to approximate the US adult population. We contacted 49 of the largest health systems (by clinician number) in the United States by email and telephone to determine the existence, or absence, of policies to guide audiorecordings of clinic visits for patients' personal use. Multiple logistic regression models were used to determine factors associated with recording. RESULTS: In total, 456 clinicians and 524 public respondents completed the surveys. More than one-quarter of clinicians (129/456, 28.3%) reported that they had recorded a clinic visit for patients' personal use, while 18.7% (98/524) of the public reported doing so, including 2.7% (14/524) who recorded visits without the clinician's permission. Amongst clinicians who had not recorded a clinic visit, 49.5% (162/327) would be willing to do so in the future, while 66.0% (346/524) of the public would be willing to record in the future. Clinician specialty was associated with prior recording: specifically oncology (odds ratio [OR] 5.1, 95% CI 1.9-14.9; P=.002) and physical rehabilitation (OR 3.9, 95% CI 1.4-11.6; P=.01). Public respondents who were male (OR 2.11, 95% CI 1.26-3.61; P=.005), younger (OR 0.73 for a 10-year increase in age, 95% CI 0.60-0.89; P=.002), or spoke a language other than English at home (OR 1.99; 95% CI 1.09-3.59; P=.02) were more likely to have recorded a clinic visit. None of the large health systems we contacted reported a dedicated policy; however, 2 of the 49 health systems did report an existing policy that would cover the recording of clinic visits for patient use. The perceived benefits of recording included improved patient understanding and recall. Privacy and medicolegal concerns were raised. CONCLUSIONS: Policy guidance from health systems and further examination of the impact of recordings-positive or negative-on care delivery, clinician-related outcomes, and patients' behavioral and health-related outcomes is urgently required.

Public involvement in UK health and care research 1995-2020: reflections from a witness seminar.

BACKGROUND: Public involvement is important to the relevance and impact of health and care research, as well as supporting the democratisation of research. In 2020, the National Institute for Health Research (NIHR) reorganized and eliminated INVOLVE, an internationally recognised group that had played a central role in public involvement in the UK since 1996. Its remit was subsumed within a new center tasked with public involvement, participant recruitment, and evidence dissemination. A year later, in 2021, interested parties came together to discuss the evolution of INVOLVE and consider how to retain some of the important historical details and learn lessons from its long and important tenure. METHODS: We hosted a witness seminar in 2022 that was one of four work groups and brought together public involvement leaders that had been part of the conception, development, and evolution of INVOLVE between 1995 and 2020. Witness seminars are a method used to capture the complexity and nuance of historical events or initiatives. They support critical thinking and reflection rather than simple commemoration. We identified those who had played a role in INVOLVE history, ensuring diversity of perspective, and invited them to attend and speak at the seminar. This took place during two sessions where witnesses provided their recollections and participated in a facilitated discussion. RESULTS: Across the two online sessions, 29 witnesses attended and contributed thoughts and recollections. Two authors (SS, MP) identified six themes that were described in the witness seminar report and have been discussed, elaborated, and illustrated with witness quotations. These are: the importance of historical perspective; INVOLVE as a social movement; how INVOLVE worked (e.g. its hospitality, kindness, and inclusivity); INVOLVE as a quiet disruptor; public involvement evidence, knowledge, and learning; the infrastructure, processes, and systems developed by INVOLVE; and the demise and loss of INVOLVE as an internationally recognized center of excellence. DISCUSSION: The authors of this commentary reflected on the discussions that took place during the witness seminar and the themes that emerged, and share six broad learnings for future practice; (1) it is important to create and nurture public involvement communities of practice; (2) collaborative ways of working support open discussion amongst diverse groups; (3) be aware of the tensions between activism and being part of the establishment; (4) continued efforts should be made to build an evidence base for public involvement practice; (5) there are both benefits and drawbacks to having a centralized organization leading public involvement; and (6) support for public involvement in research requires a fit-for-purpose tendering process that embeds robust public involvement.

BACKGROUND: Involving members of the public in research can improve the way that research is planned, managed, and shared. Between 1996 and 2020 an organization in the UK called INVOLVE had an important role in public involvement in research. When INVOLVE lost this role, some people who had been part of the group got together to think about how to save some of the important information and learn lessons from the time it had existed. METHODS: A meeting was arranged where people who have been part of an event or topic get together to share what it was like for them. This was called a witness seminar and it took place online over two days in 2022. Twenty-nine people attended and spoke about their experiences. RESULTS: The people who attended the witness seminar had different ideas about why INVOLVE was important and agree that it is now missed. People talked about INVOLVE as part of a certain time in history and said it was a social movement. They felt that it was kind and caring, brought together lots of people with different ideas, and supported changes in thinking. INVOLVE had a focus on evidence and learning and created structure and systems to support public involvement in research. Losing INVOLVE was difficult because a lot of people within the UK and beyond looked to them as a leader in public involvement. We share quotes on all of these topics. DISCUSSION: In this article we looked at how people remembered INVOLVE and thought about what information could be saved. We share lessons that will support thinking about the future of public involvement. These include things like how important it is for there to be spaces for people to come together to learn, discuss, and share, and that we have more work to do to understand public involvement and fully include it in research.

From a decentralized clinical trial to a decentralized and clinical-trial-in-a-box platform: Towards patient-centric and equitable trials.

Background/Objective: Despite the intuitive attractiveness of bringing research to participants rather than making them come to central study sites, widespread decentralized enrollment has not been common in clinical trials. Methods: The need for clinical research in the context of the COVID-19 pandemic, along with innovations in technology, led us to use a decentralized trial approach in our Phase 2 COVID-19 trial. We used real-time acquisition and transmission of health-related data using home-based monitoring devices and mobile applications to assess outcomes. This approach not only avoids spreading COVID-19 but it also can support inclusion of participants in more diverse socioeconomic circumstances and in rural settings. Results: Our team developed and deployed a decentralized trial platform to support patient engagement and adverse event reporting. Clinicians, engineers, and informaticians on our research team developed a Clinical-Trial-in-a-Box tool to optimally collect and analyze data from multiple decentralized platforms. Conclusion: Applying the decentralized model in Long COVID, using digital health technology and personal devices integrated with our telehealth platform, we share the lessons learned from our work, along with challenges and future possibilities.

The use of N-of-1 trials to generate real-world evidence for optimal treatment of individuals and populations.

Introduction: Ideally, real-world data (RWD) collected to generate real-world evidence (RWE) should lead to impact on the care and health of real-world patients. Deriving from care in which clinicians and patients try various treatments to inform therapeutic decisions, N-of-1 trials bring scientific methods to real-world practice. Methods: These single-patient crossover trials generate RWD and RWE by giving individual patients various treatments in a double-blinded way in sequential periods to determine the most effective treatment for a given patient. Results: This approach is most often used for patients with chronic, relatively stable conditions that provide the opportunity to make comparisons over multiple treatment periods, termed Type 1 N-of-1 trials. These are most helpful when there is heterogeneity of treatment effects among patients and no a priori best option. N-of-1 trials also can be done for patients with rare diseases, potentially testing only one treatment, to generate evidence for personalized treatment decisions, designated as Type 2 N-of-1 trials. With both types, in addition to informing individual's treatments, when uniform protocols are used for multiple patients with the same condition, the data collected in the individual N-of-1 trials can be aggregated to provide RWD/RWE to inform more general use of the treatments. Thereby, N-of-1 trials can provide RWE for the care of individuals and for populations. Conclusions: To fulfill this potential, we believe N-of-1 trials should be built into our current healthcare ecosystem. To this end, we are building the needed infrastructure and engaging the stakeholders who should receive value from this approach.

Quantitative assessment of the impact of standard agreement templates on multisite clinical trial start up time.

Contracting delays remain a challenge to the successful initiation of multisite clinical research in the US. The Clinical and Translational Science Awards (CTSA) Contracts Processing Study showed average contract negotiation duration of > 100 days for industry-sponsored or investigator-initiated contracts. Such delays create enormous costs to sponsors and to patients waiting to use new evidence-based treatments. With support from the National Institutes of Health's National Center for Advancing Translational Sciences, the Accelerated Clinical Trial Agreement (ACTA) was developed by 25 major academic institutions and medical centers engaged in clinical research in collaboration with the University-Industry Demonstration Partnership and with input from pharmaceutical companies. The ACTA also informed the development of subsequent agreements, including the Federal Demonstration Partnership Clinical Trial Subaward Agreement (FDP-CTSA); both ACTA and the FDP-CTSA are largely non-negotiable agreements that represent pre-negotiated compromises in contract terms agreed upon by industry and/or medical center stakeholders. When the involved parties agree to use the CTSA-developed and supported standard agreement templates as a starting point for negotiations, there can be significant time savings for trials. Use of the ACTA resulted in an average savings of 48 days and use of the FDP-CTSA saved an average of 57 days of negotiation duration.

Approaches for enhancing the informativeness and quality of clinical trials: Innovations and principles for implementing multicenter trials from the Trial Innovation Network.

One challenge for multisite clinical trials is ensuring that the conditions of an informative trial are incorporated into all aspects of trial planning and execution. The multicenter model can provide the potential for a more informative environment, but it can also place a trial at risk of becoming uninformative due to lack of rigor, quality control, or effective recruitment, resulting in premature discontinuation and/or non-publication. Key factors that support informativeness are having the right team and resources during study planning and implementation and adequate funding to support performance activities. This communication draws on the experience of the National Center for Advancing Translational Science (NCATS) Trial Innovation Network (TIN) to develop approaches for enhancing the informativeness of clinical trials. We distilled this information into three principles: (1) assemble a diverse team, (2) leverage existing processes and systems, and (3) carefully consider budgets and contracts. The TIN, comprised of NCATS, three Trial Innovation Centers, a Recruitment Innovation Center, and 60+ CTSA Program hubs, provides resources to investigators who are proposing multicenter collaborations. In addition to sharing principles that support the informativeness of clinical trials, we highlight TIN-developed resources relevant for multicenter trial initiation and conduct.

Decentralized clinical trials in the trial innovation network: Value, strategies, and lessons learned.

New technologies and disruptions related to Coronavirus disease-2019 have led to expansion of decentralized approaches to clinical trials. Remote tools and methods hold promise for increasing trial efficiency and reducing burdens and barriers by facilitating participation outside of traditional clinical settings and taking studies directly to participants. The Trial Innovation Network, established in 2016 by the National Center for Advancing Clinical and Translational Science to address critical roadblocks in clinical research and accelerate the translational research process, has consulted on over 400 research study proposals to date. Its recommendations for decentralized approaches have included eConsent, participant-informed study design, remote intervention, study task reminders, social media recruitment, and return of results for participants. Some clinical trial elements have worked well when decentralized, while others, including remote recruitment and patient monitoring, need further refinement and assessment to determine their value. Partially decentralized, or "hybrid" trials, offer a first step to optimizing remote methods. Decentralized processes demonstrate potential to improve urban-rural diversity, but their impact on inclusion of racially and ethnically marginalized populations requires further study. To optimize inclusive participation in decentralized clinical trials, efforts must be made to build trust among marginalized communities, and to ensure access to remote technology.

Development, implementation, and dissemination of operational innovations across the trial innovation network.

Improving the quality and conduct of multi-center clinical trials is essential to the generation of generalizable knowledge about the safety and efficacy of healthcare treatments. Despite significant effort and expense, many clinical trials are unsuccessful. The National Center for Advancing Translational Science launched the Trial Innovation Network to address critical roadblocks in multi-center trials by leveraging existing infrastructure and developing operational innovations. We provide an overview of the roadblocks that led to opportunities for operational innovation, our work to develop, define, and map innovations across the network, and how we implemented and disseminated mature innovations.

The Trial Innovation Network Liaison Team: building a national clinical and translational community of practice.

In 2016, the National Center for Advancing Translational Science launched the Trial Innovation Network (TIN) to address barriers to efficient and informative multicenter trials. The TIN provides a national platform, working in partnership with 60+ Clinical and Translational Science Award (CTSA) hubs across the country to support the design and conduct of successful multicenter trials. A dedicated Hub Liaison Team (HLT) was established within each CTSA to facilitate connection between the hubs and the newly launched Trial and Recruitment Innovation Centers. Each HLT serves as an expert intermediary, connecting CTSA Hub investigators with TIN support, and connecting TIN research teams with potential multicenter trial site investigators. The cross-consortium Liaison Team network was developed during the first TIN funding cycle, and it is now a mature national network at the cutting edge of team science in clinical and translational research. The CTSA-based HLT structures and the external network structure have been developed in collaborative and iterative ways, with methods for shared learning and continuous process improvement. In this paper, we review the structure, function, and development of the Liaison Team network, discuss lessons learned during the first TIN funding cycle, and outline a path toward further network maturity.

Key issues for stakeholder engagement in the development of health and healthcare guidelines.

Established in 2015, the Multi-Stakeholder Engagement (MuSE) Consortium is an international network of over 120 individuals interested in stakeholder engagement in research and guidelines. The MuSE group is developing guidance for stakeholder engagement in the development of health and healthcare guideline development. The development of this guidance has included multiple meetings with stakeholders, including patients, payers/purchasers of health services, peer review editors, policymakers, program managers, providers, principal investigators, product makers, the public, and purchasers of health services and has identified a number of key issues. These include: (1) Definitions, roles, and settings (2) Stakeholder identification and selection (3) Levels of engagement, (4) Evaluation of engagement, (5) Documentation and transparency, and (6) Conflict of interest management. In this paper, we discuss these issues and our plan to develop guidance to facilitate stakeholder engagement in all stages of the development of health and healthcare guideline development.

A group of international researchers, patient partners, and other stakeholders are working together to create a checklist for when and how to involve stakeholders in health guideline development. Health guidelines include clinical practice guidelines, which your healthcare provider uses to determine treatments for health conditions. While working on this checklist, the team identified key issues to work on, including: (1) Definitions, roles, and settings (2) Stakeholder identification and selection (3) Levels of engagement, (4) Evaluation of engagement, (5) Documentation and transparency, and (6) Conflict of interest management. This paper describes each issue and how the team plans to produce guidance papers to address them.

Leveraging the Expertise of the CTSA Program to Increase the Impact and Efficiency of Clinical Trials.

Importance: Multicenter clinical trials play a critical role in the translational processes that enable new treatments to reach all people and improve public health. However, conducting multicenter randomized clinical trials (mRCT) presents challenges. The Trial Innovation Network (TIN), established in 2016 to partner with the Clinical and Translational Science Award (CTSA) Consortium of academic medical institutions in the implementation of mRCTs, consists of 3 Trial Innovation Centers (TICs) and 1 Recruitment Innovation Center (RIC). This unique partnership has aimed to address critical roadblocks that impede the design and conduct of mRCTs, in expectation of accelerating the translation of novel interventions to clinical practice. The TIN's challenges and achievements are described in this article, along with examples of innovative resources and processes that may serve as useful models for other clinical trial networks providing operational and recruitment support. Observations: The TIN has successfully integrated more than 60 CTSA institution program hubs into a functional network for mRCT implementation and optimization. A unique support system for investigators has been created that includes the development and deployment of novel tools, operational and recruitment services, consultation models, and rapid communication pathways designed to reduce delays in trial start-up, enhance recruitment, improve engagement of diverse research participants and communities, and streamline processes that improve the quality, efficiency, and conduct of mRCTs. These resources and processes span the clinical trial spectrum and enable the TICs and RIC to serve as coordinating centers, data centers, and recruitment specialists to assist trials across the National Institutes of Health and other agencies. The TIN's impact has been demonstrated through its response to both historical operational challenges and emerging public health emergencies, including the national opioid public health crisis and the COVID-19 pandemic. Conclusions and Relevance: The TIN has worked to reduce barriers to implementing mRCTs and to improve mRCT processes and operations by providing needed clinical trial infrastructure and resources to CTSA investigators. These resources have been instrumental in more quickly and efficiently translating research discoveries into beneficial patient treatments.

Sharing data among clinical trials of therapeutics in COVID-19: Barriers and facilitators to collaborating in a crisis.

Background: The Clinical and Translational Science Award Program (CTSA) Trial Innovation Network (TIN) was launched in 2016 to increase the efficiency and effectiveness of multisite trials by supporting the development of national infrastructure. With the advent of the COVID-19 pandemic, it was therefore well-positioned to support clinical trial collaboration. The TIN was leveraged to support two initiatives: (1) to create and evaluate a mechanism for coordinating Data and Safety Monitoring Board (DSMB) activities among multiple ongoing trials of the same therapeutic agents, and (2) to share data across clinical trials so that smaller, likely underpowered studies, could be combined to produce meaningful and actionable data through pooled analyses. The success of these initiatives was understood to be dependent upon the willingness of investigators, study teams, and US National Institutes of Health research networks to collaborate and share information. Methods: To inform these two initiatives, we conducted semistructured interviews with members of CTSA hubs and clinical research stakeholders that probed barriers and facilitators to collaboration. Thematic analysis identified topics relevant across institutions, individuals, and DSMBs. Results: The DSMB coordination initiative was viewed as less controversial, while the data pooling initiative was seen as complex because of its potential impact on publication, authorship, and the rewards of discovery. Barriers related to resources, centralization, and technical work were significant, but interviewees suggested these could be handled by the provision of central funding and supportive frameworks. The more intractable findings were related to issues around credit and ownership of data. Conclusion: Based on our interviews, we conclude with nine recommended actions that can be implemented to support collaboration.

A Useful and Sustainable Role for N-of-1 Trials in the Healthcare Ecosystem.

Clinicians and patients often try a treatment for an initial period to inform longer-term therapeutic decisions. A more rigorous approach involves N-of-1 trials. In these single-patient crossover trials, typically conducted in patients with chronic conditions, individual patients are given candidate treatments in a double-blinded, random sequence of alternating periods to determine the most effective treatment for that patient. However, to date, these trials are rarely done outside of research settings and have not been integrated into general care where they could offer substantial benefit. Designating this classical, N-of-1 trial design as type 1, there also are new and evolving uses of N-of-1 trials that we designate as type 2. In these, rather than focusing on optimizing treatment for chronic diseases when multiple approved choices are available, as is typical of type 1, a type 2 N-of-1 trial tests treatments designed specifically for a patient with a rare disease, to facilitate personalized medicine. While the aims differ, both types face the challenge of collecting individual-patient evidence using standard, trusted, widely accepted methods. To fulfill their potential for producing both clinical and research benefits, and to be available for wide use, N-of-1 trials will have to fit into the current healthcare ecosystem. This will require generalizable and accepted processes, platforms, methods, and standards. This also will require sustainable value-based arrangements among key stakeholders. In this article, we review opportunities, stakeholders, issues, and possible approaches that could support general use of N-of-1 trials and deliver benefit to patients and the healthcare enterprise. To assess and expand the benefits of N-of-1 trials, we propose multistakeholder meetings, workshops, and the generation of methods, standards, and platforms that would support wider availability and the value of N-of-1 trials.

Financial Integration's Impact On Care Delivery And Payment Reforms: A Survey Of Hospitals And Physician Practices.

Health systems continue to grow in size. Financial integration-the ownership of hospitals or physician practices-often has anticompetitive effects that contribute to the higher prices for health care seen in the US. To determine whether the potential harms of financial integration are counterbalanced by improvements in quality, we surveyed nationally representative samples of hospitals (n = 739) and physician practices (n = 2,189), stratified according to whether they were independent or were owned by complex systems, simple systems, or medical groups. The surveys included nine scales measuring the level of adoption of diverse, quality-focused care delivery and payment reforms. Scores varied widely across hospitals and practices, but little of this variation was explained by ownership status. Quality scores favored financially integrated systems for four of nine hospital measures and one of nine practice measures, but in no case favored complex systems. Greater financial integration was generally not associated with better quality.

Development and validation of the Generalized Anxiety Disorder Inventory (GADI).

The psychometric tools used for the assessment of generalized anxiety disorder (GAD) either do not conform to the current concept of the condition or have important limitations. We aimed to develop and validate a new questionnaire for the assessment of symptom profile and severity of GAD. An original pool of potential scale items was subjected to a series of studies in non-clinical and clinical populations, in order to determine the final composition of the scale. The psychometric properties of the new scale, the Generalized Anxiety Disorder Inventory (GADI), were evaluated using a factor analytic model suitable for ordinal data and the Graded Response Model. The precision of measurement of the GADI was quantified through the item information functions.A total of 197 outpatients and 522 non-clinical subjects participated in four studies and completed the GADI. The final 18-item scale was derived from an original pool of 30 potential items. The GADI showed good reliability, convergent and divergent validity. The scale comprises three factors, relating to cognitive, somatic and sleep symptoms. It accurately distinguished GAD patients from non-patient controls. The cognitive factor also distinguished GAD from other anxiety disorders and depression. The GADI is a useful tool in the assessment of the breadth of symptoms and the severity of generalized anxiety disorder in clinical settings.