Very rapid insight generation to support UK health and care systems: An AHSN approach.

Introduction: COVID-19 challenges are well documented. Academic Health Science Networks (AHSNs) are a key partner to NHS and care organizations. In response to managing COVID-19 challenges, Wessex AHSN offered rapid insight generation and rapid evaluation to local NHS and care systems to capture learning during this period. This novel "Rapid Insight" approach involved one-off online deliberative events with stakeholders to generate insights linked to specific, priority areas of interest, followed by rapid analysis and dissemination of the findings. Context: Key objectives were to enable system leaders to build their adaptive leadership capability and learn from the experience of COVID-19 to inform recovery planning and system support. Rapid Insight (RI) gathered together health and care professionals into a tightly managed, virtual forum to share system intelligence. Approach: Focused questions asked about the systems' response to the pandemic, what changes to continue and sustain, or discontinue. Participants responded simultaneously to each question using the virtual chat function. Immediate thematic analysis of the chat conducted in 48-72 h by paired analysts for each question strengthened analytical integrity. Mind maps, the key output, provided easily assimilated information and showed linkages between themes. Telephone or virtual interviews of key informants (health and care professionals and patients) and routinely collected data were synthesized into short reports alongside several RI events. However, insufficient time limited the opportunities to engage diverse participants (e.g., mental health users). Data from RI can scope the problem and immediate system needs, to stimulate questions for future evaluative work. Impact: RI facilitated a shared endeavor to discover "clues in the system" by including diverse opinions and experience across NHS and care organizations. Although these rapid virtual events saved on travel time, digital exclusion might constrain participation for some stakeholders which needs other ways to ensure inclusion. Successful rapid engagement required Wessex AHSN's existing system relationships to champion RI and facilitate participant recruitment. RI events "opened the door" to conversations between up to 150 multi-professional clinicians to share their collective response to COVID-19. This paper focuses on the RI approach with a case example and its further development.

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR).

BACKGROUND: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. METHODS: We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. RESULTS: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. CONCLUSIONS: The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

Qualitative "trial-sibling" studies and "unrelated" qualitative studies contributed to complex intervention reviews.

OBJECTIVE: To compare the contribution of "trial-sibling" and "unrelated" qualitative studies in complex intervention reviews. STUDY DESIGN AND SETTING: Researchers are using qualitative "trial-sibling" studies undertaken alongside trials to provide explanations to understand complex interventions. In the absence of qualitative "trial-sibling" studies, it is not known if qualitative studies "unrelated" to trials are helpful. Trials, "trial-sibling," and "unrelated" qualitative studies looking at three health system interventions were identified. We looked for similarities and differences between the two types of qualitative studies, such as participants, intervention delivery, context, study quality and reporting, and contribution to understanding trial results. RESULTS: Reporting was generally poor in both qualitative study types. We detected no substantial differences in participant characteristics. Interventions in qualitative "trial-sibling" studies were delivered using standardized protocols, whereas interventions in "unrelated" qualitative studies were delivered in routine care. Qualitative "trial-sibling" studies alone provided insufficient data to develop meaningful transferrable explanations beyond the trial context, and their limited focus on immediate implementation did not address all phenomena of interest. Together, "trial-sibling" and "unrelated" qualitative studies provided larger, richer data sets across contexts to better understand the phenomena of interest. CONCLUSIONS: Findings support inclusion of "trial-sibling" and "unrelated" qualitative studies to explore complexity in complex intervention reviews.

Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed.

OBJECTIVE: To identify examples of how social theories are used in systematic reviews of complex interventions to inform production of Cochrane guidance. STUDY DESIGN AND SETTING: Secondary analysis of published/unpublished examples of theories of social phenomena for use in reviews of complex interventions identified through scoping searches, engagement with key authors and methodologists supplemented by snowballing and reference searching. Theories were classified (low-level, mid-range, grand). RESULTS: Over 100 theories were identified with evidence of proliferation over the last 5 years. New low-level theories (tools, taxonomies, etc) have been developed for classifying and reporting complex interventions. Numerous mid-range theories are used; one example demonstrated how control theory had changed the review's findings. Review-specific logic models are increasingly used, but these can be challenging to develop. New low-level and mid-range psychological theories of behavior change are evolving. No reviews using grand theory (e.g., feminist theory) were identified. We produced a searchable Wiki, Mendeley Inventory, and Cochrane guidance. CONCLUSIONS: Use of low-level theory is common and evolving; incorporation of mid-range theory is still the exception rather than the norm. Methodological work is needed to evaluate the contribution of theory. Choice of theory reflects personal preference; application of theory is a skilled endeavor.

Cochrane methods--twenty years experience in developing systematic review methods.

This year, The Cochrane Collaboration reached its 20th anniversary. It has played a pivotal role in the scientific development of systematic reviewing and in the development of review methods to synthesize research evidence, primarily from randomized trials, to answer questions about the effects of healthcare interventions. We introduce a series of articles, which form this special issue describing the development of systematic review methods within The Cochrane Collaboration. We also discuss the impact of Cochrane Review methods, and acknowledge the breadth and depth of methods development within The Cochrane Collaboration as part of the wider context of evidence synthesis. We conclude by considering the future development of methods for Cochrane Reviews.

The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework.

BACKGROUND: The case has been made for more and better theory-informed process evaluations within trials in an effort to facilitate insightful understandings of how interventions work. In this paper, we provide an explanation of implementation processes from one of the first national implementation research randomized controlled trials with embedded process evaluation conducted within acute care, and a proposed extension to the Promoting Action on Research Implementation in Health Services (PARIHS) framework. METHODS: The PARIHS framework was prospectively applied to guide decisions about intervention design, data collection, and analysis processes in a trial focussed on reducing peri-operative fasting times. In order to capture a holistic picture of implementation processes, the same data were collected across 19 participating hospitals irrespective of allocation to intervention. This paper reports on findings from data collected from a purposive sample of 151 staff and patients pre- and post-intervention. Data were analysed using content analysis within, and then across data sets. RESULTS: A robust and uncontested evidence base was a necessary, but not sufficient condition for practice change, in that individual staff and patient responses such as caution influenced decision making. The implementation context was challenging, in which individuals and teams were bounded by professional issues, communication challenges, power and a lack of clarity for the authority and responsibility for practice change. Progress was made in sites where processes were aligned with existing initiatives. Additionally, facilitators reported engaging in many intervention implementation activities, some of which result in practice changes, but not significant improvements to outcomes. CONCLUSIONS: This study provided an opportunity for reflection on the comprehensiveness of the PARIHS framework. Consistent with the underlying tenant of PARIHS, a multi-faceted and dynamic story of implementation was evident. However, the prominent role that individuals played as part of the interaction between evidence and context is not currently explicit within the framework. We propose that successful implementation of evidence into practice is a planned facilitated process involving an interplay between individuals, evidence, and context to promote evidence-informed practice. This proposal will enhance the potential of the PARIHS framework for explanation, and ensure theoretical development both informs and responds to the evidence base for implementation.

A pragmatic cluster randomised trial evaluating three implementation interventions.

BACKGROUND: Implementation research is concerned with bridging the gap between evidence and practice through the study of methods to promote the uptake of research into routine practice. Good quality evidence has been summarised into guideline recommendations to show that peri-operative fasting times could be considerably shorter than patients currently experience. The objective of this trial was to evaluate the effectiveness of three strategies for the implementation of recommendations about peri-operative fasting. METHODS: A pragmatic cluster randomised trial underpinned by the PARIHS framework was conducted during 2006 to 2009 with a national sample of UK hospitals using time series with mixed methods process evaluation and cost analysis. Hospitals were randomised to one of three interventions: standard dissemination (SD) of a guideline package, SD plus a web-based resource championed by an opinion leader, and SD plus plan-do-study-act (PDSA). The primary outcome was duration of fluid fast prior to induction of anaesthesia. Secondary outcomes included duration of food fast, patients' experiences, and stakeholders' experiences of implementation, including influences. ANOVA was used to test differences over time and interventions. RESULTS: Nineteen acute NHS hospitals participated. Across timepoints, 3,505 duration of fasting observations were recorded. No significant effect of the interventions was observed for either fluid or food fasting times. The effect size was 0.33 for the web-based intervention compared to SD alone for the change in fluid fasting and was 0.12 for PDSA compared to SD alone. The process evaluation showed different types of impact, including changes to practices, policies, and attitudes. A rich picture of the implementation challenges emerged, including inter-professional tensions and a lack of clarity for decision-making authority and responsibility. CONCLUSIONS: This was a large, complex study and one of the first national randomised controlled trials conducted within acute care in implementation research. The evidence base for fasting practice was accepted by those participating in this study and the messages from it simple; however, implementation and practical challenges influenced the interventions' impact. A set of conditions for implementation emerges from the findings of this study, which are presented as theoretically transferable propositions that have international relevance. TRIAL REGISTRATION: ISRCTN18046709--Peri-operative Implementation Study Evaluation (POISE).